From conditioning to category learning: An adaptive network model

Modelling the acquisition of Pavlovian conditioning

Pavlovian conditioning is a fundamental learning process that allows animals to anticipate and respond to significant environmental events. This review examines the key properties of the relationship between the conditioned stimulus (CS) and unconditioned stimulus (US) that influence learning, focussing on the temporal proximity of the CS and US, the spacing of trials (pairings of the CS and US), and the contingency between the CS and US. These properties have been touchstones for models of associative learning. Two primary theoretical approaches are contrasted here. Connection strength models, exemplified by the Rescorla-Wagner model (Rescorla & Wagner, 1972), describe learning as trial-by-trial changes in the strength of an associative bond based on prediction errors. In time-based models of learning (e.g., Gallistel & Gibbon, 2000) animals encode and remember temporal intervals and rates of reinforcement. The integration of Information Theory into time-based models (Balsam & Gallistel, 2009) provides a mathematical framework for quantifying the effects of proximity, trial spacing, and contingency in terms of how much the CS reduces uncertainty about the US. The present paper incorporates a trial-by-trial Bayesian updating process into the information theoretic account to describe how uncertainty about the CS-US interval changes across conditioning. This Bayesian process is shown to account for empirical evidence about the way that responding changes continuously over conditioning trials.

Humans forage for reward in reinforcement learning tasks

How do we make good decisions in uncertain environments? In psychology and neuroscience, the classic view is that we calculate the value of each option, compare them, and choose the most rewarding modulo exploratory noise. An ethologist, conversely, would argue that we commit to one option until its value drops below a threshold and then explore alternatives. Because the fields use incompatible methods, it remains unclear which view better describes human decision-making. Here, we found that humans use compare-to-threshold computations in classic compare-alternative tasks. Because compare-alternative computations are central to the reinforcement-learning (RL) models typically used in the cognitive and brain sciences, we developed a novel compare-to-threshold model ("foraging"). Compared to previous RL models, the foraging model better fit participant behavior, better predicted the tendency to repeat choices, and predicted held-out participants that were almost impossible under compare-alternative models. These results suggest that humans use compare-to-threshold computations in sequential decision-making.

Better generalization through distraction? Concurrent load reduces the size of the inverse base-rate effect

The inverse base-rate effect (IBRE) is an irrational phenomenon in predictive learning. It occurs when people try to generalize what they have experienced to novel and ambiguous events. This irrational generalization manifests as a preference for rare, unlikely outcomes in the face of ambiguity. At least two formal mathematical models of this irrational preference (EXIT, NNRAS) lead to a counter-intuitive prediction: the effect reduces under concurrent load. We tested this prediction across two experiments ( N 1 = 72, M age = 20.12; N 2 = 160, M age = 20.88). We confirm the prediction, but only when participants were under an obvious time constraint. This empirical confirmation is as surprising as the prediction itself-irrationality reduces under increased task demands. Further, our data are more consistent with the NNRAS model than with EXIT, the most prominent model of the IBRE to date.

Prototype Analysis in Hopfield Networks With Hebbian Learning

We discuss prototype formation in the Hopfield network. Typically, Hebbian learning with highly correlated states leads to degraded memory performance. We show that this type of learning can lead to prototype formation, where unlearned states emerge as representatives of large correlated subsets of states, alleviating capacity woes. This process has similarities to prototype learning in human cognition. We provide a substantial literature review of prototype learning in associative memories, covering contributions from psychology, statistical physics, and computer science. We analyze prototype formation from a theoretical perspective and derive a stability condition for these states based on the number of examples of the prototype presented for learning, the noise in those examples, and the number of nonexample states presented. The stability condition is used to construct a probability of stability for a prototype state as the factors of stability change. We also note similarities to traditional network analysis, allowing us to find a prototype capacity. We corroborate these expectations of prototype formation with experiments using a simple Hopfield network with standard Hebbian learning. We extend our experiments to a Hopfield network trained on data with multiple prototypes and find the network is capable of stabilizing multiple prototypes concurrently. We measure the basins of attraction of the multiple prototype states, finding attractor strength grows with the number of examples and the agreement of examples. We link the stability and dominance of prototype states to the energy profile of these states, particularly when comparing the profile shape to target states or other spurious states.

Toward the Emergence of Intelligent Control: Episodic Generalization and Optimization

Human cognition is unique in its ability to perform a wide range of tasks and to learn new tasks quickly. Both abilities have long been associated with the acquisition of knowledge that can generalize across tasks and the flexible use of that knowledge to execute goal-directed behavior. We investigate how this emerges in a neural network by describing and testing the Episodic Generalization and Optimization (EGO) framework. The framework consists of an episodic memory module, which rapidly learns relationships between stimuli; a semantic pathway, which more slowly learns how stimuli map to responses; and a recurrent context module, which maintains a representation of task-relevant context information, integrates this over time, and uses it both to recall context-relevant memories (in episodic memory) and to bias processing in favor of context-relevant features and responses (in the semantic pathway). We use the framework to address empirical phenomena across reinforcement learning, event segmentation, and category learning, showing in simulations that the same set of underlying mechanisms accounts for human performance in all three domains. The results demonstrate how the components of the EGO framework can efficiently learn knowledge that can be flexibly generalized across tasks, furthering our understanding of how humans can quickly learn how to perform a wide range of tasks-a capability that is fundamental to human intelligence.

Remote memory in a Bayesian model of context fear conditioning (BaconREM)

Here, we propose a model of remote memory (BaconREM), which is an extension of a previously published Bayesian model of context fear learning (BACON) that accounts for many aspects of recently learned context fear. BaconREM simulates most known phenomenology of remote context fear as studied in rodents and makes new predictions. In particular, it predicts the well-known observation that fear that was conditioned to a recently encoded context becomes hippocampus-independent and shows much-enhanced generalization ("hyper-generalization") when systems consolidation occurs (i.e., when memory becomes remote). However, the model also predicts that there should be circumstances under which the generalizability of remote fear may not increase or even decrease. It also predicts the established finding that a "reminder" exposure to a feared context can abolish hyper-generalization while at the same time making remote fear again hippocampus-dependent. This observation has in the past been taken to suggest that reminders facilitate access to detail memory that remains permanently in the hippocampus even after systems consolidation is complete. However, the present model simulates this result even though it totally moves all the contextual memory that it retains to the neo-cortex when context fear becomes remote.

Associative learning or Bayesian inference? Revisiting backwards blocking reasoning in adults

Causal reasoning is a fundamental cognitive ability that enables humans to learn about the complex interactions in the world around them. However, the cognitive mechanisms that underpin causal reasoning are not well understood. For instance, there is debate over whether Bayesian inference or associative learning best captures causal reasoning in human adults. The two experiments and computational models reported here were designed to examine whether adults engage in one form of causal inference called backwards blocking reasoning, whether the presence of potential distractors affects performance, and how adults' ratings align with the predictions of different computational models. The results revealed that adults engaged in backwards blocking reasoning regardless of whether distractor objects are present and that their causal judgements supported the predictions of a Bayesian model but not the predictions of two different associative learning models. Implications of these results are discussed.

Why is the Rescorla-Wagner model so influential?

The influence of the Rescorla-Wagner model cannot be overestimated, despite that (1) the model does not differ much computationally from its predecessors and competitors, and (2) its shortcomings are well-known in the learning community. Here we discuss the reasons behind its widespread influence in the cognitive and neural sciences, and argue that it is the constant search for general-process theories by learning scholars which eventually produced a model whose application spans many different areas of research to this day. We focus on the theoretical and empirical background of the model, the theoretical connections that it has with later developments across Marr's levels of analysis, as well as the broad variety of research that it has guided and inspired.

A source for category-induced global effects of feature-based attention in human prefrontal cortex

Global effects of feature-based attention (FBA) are generally limited to stimuli sharing the same or similar features, as hypothesized in the "feature-similarity gain model." Visual perception, however, often reflects categories acquired via experience/learning; whether the global-FBA effect can be induced by the categorized features remains unclear. Here, human subjects were trained to classify motion directions into two discrete categories and perform a classical motion-based attention task. We found a category-induced global-FBA effect in both the middle temporal area (MT+) and frontoparietal areas, where attention to a motion direction globally spread to unattended motion directions within the same category, but not to those in a different category. Effective connectivity analysis showed that the category-induced global-FBA effect in MT+ was derived by feedback from the inferior frontal junction (IFJ). Altogether, our study reveals a category-induced global-FBA effect and identifies a source for this effect in human prefrontal cortex, implying that FBA is of greater ecological significance than previously thought.

Similarity and structured representation in human and nonhuman apes

How we judge the similarity between objects in the world is connected ultimately to how we represent those objects. It has been argued extensively that object representations in humans are 'structured' in nature, meaning that both individual features and the relations between them can influence similarity. In contrast, popular models within comparative psychology assume that nonhuman species appreciate only surface-level, featural similarities. By applying psychological models of structural and featural similarity (from conjunctive feature models to Tversky's Contrast Model) to visual similarity judgements from adult humans, chimpanzees, and gorillas, we demonstrate a cross-species sensitivity to complex structural information, particularly for stimuli that combine colour and shape. These results shed new light on the representational complexity of nonhuman apes, and the fundamental limits of featural coding in explaining object representation and similarity, which emerge strikingly across both human and nonhuman species.

Incidental auditory category learning and visuomotor sequence learning do not compete for cognitive resources

The environment provides multiple regularities that might be useful in guiding behavior if one was able to learn their structure. Understanding statistical learning across simultaneous regularities is important, but poorly understood. We investigate learning across two domains: visuomotor sequence learning through the serial reaction time (SRT) task, and incidental auditory category learning via the systematic multimodal association reaction time (SMART) task. Several commonalities raise the possibility that these two learning phenomena may draw on common cognitive resources and neural networks. In each, participants are uninformed of the regularities that they come to use to guide actions, the outcomes of which may provide a form of internal feedback. We used dual-task conditions to compare learning of the regularities in isolation versus when they are simultaneously available to support behavior on a seemingly orthogonal visuomotor task. Learning occurred across the simultaneous regularities, without attenuation even when the informational value of a regularity was reduced by the presence of the additional, convergent regularity. Thus, the simultaneous regularities do not compete for associative strength, as in overshadowing effects. Moreover, the visuomotor sequence learning and incidental auditory category learning do not appear to compete for common cognitive resources; learning across the simultaneous regularities was comparable to learning each regularity in isolation.

A Statistical Foundation for Derived Attention

According to the theory of derived attention, organisms attend to cues with strong associations. Prior work has shown that - combined with a Rescorla-Wagner style learning mechanism - derived attention explains phenomena such as learned predictiveness, inattention to blocked cues, and value-based salience. We introduce a Bayesian derived attention model that explains a wider array of results than previous models and gives further insight into the principle of derived attention. Our approach combines Bayesian linear regression with the assumption that the associations of any cue with various outcomes share the same prior variance, which can be thought of as the inherent importance of that cue. The new model simultaneously estimates cue-outcome associations and prior variance through approximate Bayesian learning. A significant cue will develop large associations, leading the model to estimate a high prior variance and hence develop larger associations from that cue to novel outcomes. This provides a normative, statistical explanation for derived attention. Through simulation, we show that this Bayesian derived attention model not only explains the same phenomena as previous versions, but also retrospective revaluation. It also makes a novel prediction: inattention after backward blocking. We hope that further development of the Bayesian derived attention model will shed light on the complex relationship between uncertainty and predictiveness effects on attention.

An exploration of error-driven learning in simple two-layer networks from a discriminative learning perspective

Error-driven learning algorithms, which iteratively adjust expectations based on prediction error, are the basis for a vast array of computational models in the brain and cognitive sciences that often differ widely in their precise form and application: they range from simple models in psychology and cybernetics to current complex deep learning models dominating discussions in machine learning and artificial intelligence. However, despite the ubiquity of this mechanism, detailed analyses of its basic workings uninfluenced by existing theories or specific research goals are rare in the literature. To address this, we present an exposition of error-driven learning - focusing on its simplest form for clarity - and relate this to the historical development of error-driven learning models in the cognitive sciences. Although historically error-driven models have been thought of as associative, such that learning is thought to combine preexisting elemental representations, our analysis will highlight the discriminative nature of learning in these models and the implications of this for the way how learning is conceptualized. We complement our theoretical introduction to error-driven learning with a practical guide to the application of simple error-driven learning models in which we discuss a number of example simulations, that are also presented in detail in an accompanying tutorial.

An adaptive linear filter model of procedural category learning

We use a feature-based association model to fit grouped and individual level category learning and transfer data. The model assumes that people use corrective feedback to learn individual feature to categorization-criterion correlations and combine those correlations additively to produce classifications. The model is an Adaptive Linear Filter (ALF) with logistic output function and Least Mean Squares learning algorithm. Categorization probabilities are computed by a logistic function. Our data span over 31 published data sets. Both at grouped and individual level analysis levels, the model performs remarkably well, accounting for large amounts of available variances. When fitted to grouped data, it outperforms alternative models. When fitted to individual level data, it is able to capture learning and transfer performance with high explained variances. Notably, the model achieves its fits with a very minimal number of free parameters. We discuss the ALF's advantages as a model of procedural categorization, in terms of its simplicity, its ability to capture empirical trends and its ability to solve challenges to other associative models. In particular, we discuss why the model is not equivalent to a prototype model, as previously thought.

Neural Networks Track the Logical Complexity of Boolean Concepts

The language of thought hypothesis and connectionism provide two main accounts of category acquisition in the cognitive sciences. However, it is unclear to what extent their predictions agree. In this article, we tackle this problem by comparing the two accounts with respect to a common set of predictions about the effort required to acquire categories. We find that the two accounts produce similar predictions in the domain of Boolean categorization, however, with substantial variation depending on the operators in the language of thought.

Rethinking delusions: A selective review of delusion research through a computational lens

Delusions are rigid beliefs held with high certainty despite contradictory evidence. Notwithstanding decades of research, we still have a limited understanding of the computational and neurobiological alterations giving rise to delusions. In this review, we highlight a selection of recent work in computational psychiatry aimed at developing quantitative models of inference and its alterations, with the goal of providing an explanatory account for the form of delusional beliefs in psychosis. First, we assess and evaluate the experimental paradigms most often used to study inferential alterations in delusions. Based on our review of the literature and theoretical considerations, we contend that classic draws-to-decision paradigms are not well-suited to isolate inferential processes, further arguing that the commonly cited 'jumping-to-conclusion' bias may reflect neither delusion-specific nor inferential alterations. Second, we discuss several enhancements to standard paradigms that show promise in more effectively isolating inferential processes and delusion-related alterations therein. We further draw on our recent work to build an argument for a specific failure mode for delusions consisting of prior overweighting in high-level causal inferences about partially observable hidden states. Finally, we assess plausible neurobiological implementations for this candidate failure mode of delusional beliefs and outline promising future directions in this area.

What Is Behind the Buzzword for Experts and Laymen: Representation of "Artificial Intelligence" in the IT-Professionals' and Non-Professionals' Minds

The aim of the presented research was to define the differences between information technology (IT) professionals (ITP) and non-professionals (NP) in the way of understanding artificial intelligence (AI). The research was designed in the tradition of categorization research. In an online study participants were asked to make typicality and familiarity judgments for 50 AI exemplars. Two types of analyses were carried out, which made it possible to identify and compare the hierarchy of AI designates (graded structure) and the dimensions of their groupings. We have found that "invisible AI" exemplars were highly rated by ITP, but "visible AI" by NP. Expert knowledge allows ITP to systematize AI exemplars based on both structural and functional elements. On the other hand, laymen indicate the functions that AI-driven products perform, rather than their structures. For ITP, they are primarily algorithmic systems, while for NP they are systems that emulate the functions of living organisms.

A Functional Contextual Account of Background Knowledge in Categorization: Implications for Artificial General Intelligence and Cognitive Accounts of General Knowledge

Psychology has benefited from an enormous wealth of knowledge about processes of cognition in relation to how the brain organizes information. Within the categorization literature, this behavior is often explained through theories of memory construction called exemplar theory and prototype theory which are typically based on similarity or rule functions as explanations of how categories emerge. Although these theories work well at modeling highly controlled stimuli in laboratory settings, they often perform less well outside of these settings, such as explaining the emergence of background knowledge processes. In order to explain background knowledge, we present a non-similarity-based post-Skinnerian theory of human language called Relational Frame Theory (RFT) which is rooted in a philosophical world view called functional contextualism (FC). This theory offers a very different interpretation of how categories emerge through the functions of behavior and through contextual cues, which may be of some benefit to existing categorization theories. Specifically, RFT may be able to offer a novel explanation of how background knowledge arises, and we provide some mathematical considerations in order to identify a formal model. Finally, we discuss much of this work within the broader context of general semantic knowledge and artificial intelligence research.

Computational mechanisms of distributed value representations and mixed learning strategies

Learning appropriate representations of the reward environment is challenging in the real world where there are many options, each with multiple attributes or features. Despite existence of alternative solutions for this challenge, neural mechanisms underlying emergence and adoption of value representations and learning strategies remain unknown. To address this, we measure learning and choice during a multi-dimensional probabilistic learning task in humans and trained recurrent neural networks (RNNs) to capture our experimental observations. We find that human participants estimate stimulus-outcome associations by learning and combining estimates of reward probabilities associated with the informative feature followed by those of informative conjunctions. Through analyzing representations, connectivity, and lesioning of the RNNs, we demonstrate this mixed learning strategy relies on a distributed neural code and opponency between excitatory and inhibitory neurons through value-dependent disinhibition. Together, our results suggest computational and neural mechanisms underlying emergence of complex learning strategies in naturalistic settings.

Strategy Development and Feedback Processing During Complex Category Learning

In this study, 38 young adults participated in a probabilistic A/B prototype category learning task under observational and feedback-based conditions. The study compared learning success (testing accuracy) and strategy use (multi-cue vs. single feature vs. random pattern) between training conditions. The feedback-related negativity (FRN) and P3a event related potentials were measured to explore the relationships between feedback processing and strategy use under a probabilistic paradigm. A greater number of participants were found to utilize an optimal, multi-cue strategy following feedback-based training than observational training, adding to the body of research suggesting that feedback can influence learning approach. There was a significant interaction between training phase and strategy on FRN amplitude. Specifically, participants who used a strategy in which category membership was determined by a single feature (single feature strategy) exhibited a significant decrease in FRN amplitude from early training to late training, perhaps due to reduced utilization of feedback or reduced prediction error. There were no significant main or interaction effects between valence, training phase, or strategy on P3a amplitude. Findings are consistent with prior research suggesting that learners vary in their approach to learning and that training method influences learning. Findings also suggest that measures of feedback processing during probabilistic category learning may reflect changes in feedback utilization and may further illuminate differences among individual learners.

Symbolic Deep Networks: A Psychologically Inspired Lightweight and Efficient Approach to Deep Learning

The last two decades have produced unprecedented successes in the fields of artificial intelligence and machine learning (ML), due almost entirely to advances in deep neural networks (DNNs). Deep hierarchical memory networks are not a novel concept in cognitive science and can be traced back more than a half century to Simon's early work on discrimination nets for simulating human expertise. The major difference between DNNs and the deep memory nets meant for explaining human cognition is that the latter are symbolic networks meant to model the dynamics of human memory and learning. Cognition-inspired symbolic deep networks (SDNs) address several known issues with DNNs, including (1) learning efficiency, where a much larger number of training examples are required for DNNs than would be expected for a human; (2) catastrophic interference, where what is learned by a DNN gets unlearned when a new problem is presented; and (3) explainability, where there is no way to explain what is learned by a DNN. This paper explores whether SDNs can achieve similar classification accuracy performance to DNNs across several popular ML datasets and discusses the strengths and weaknesses of each approach. Simulations reveal that (1) SDNs provide similar accuracy to DNNs in most cases, (2) SDNs are far more efficient than DNNs, (3) SDNs are as robust as DNNs to irrelevant/noisy attributes in the data, and (4) SDNs are far more robust to catastrophic interference than DNNs. We conclude that SDNs offer a promising path toward human-level accuracy and efficiency in category learning. More generally, ML frameworks could stand to benefit from cognitively inspired approaches, borrowing more features and functionality from models meant to simulate and explain human learning.

Neural systems underlying the learning of cognitive effort costs

People balance the benefits of cognitive work against the costs of cognitive effort. Models that incorporate prospective estimates of the costs of cognitive effort into decision making require a mechanism by which these costs are learned. However, it remains an open question what brain systems are important for this learning, particularly when learning is not tied explicitly to a decision about what task to perform. In this fMRI experiment, we parametrically manipulated the level of effort a task requires by increasing task switching frequency across six task contexts. In a scanned learning phase, participants implicitly learned about the task switching frequency in each context. In a subsequent test phase, participants made selections between pairs of these task contexts. We modeled learning within a reinforcement learning framework, and found that effort expectations that derived from task-switching probability and response time (RT) during learning were the best predictors of later choice behavior. Prediction errors (PE) from these two models were associated with FPN during distinct learning epochs. Specifically, PE derived from expected RT was most correlated with the fronto-parietal network early in learning, whereas PE derived from expected task switching frequency was correlated with the fronto-parietal network late in learning. These results suggest that multiple task-related factors are tracked by the brain while performing a task that can drive subsequent estimates of effort costs.

On the Generalization of Simple Alternating Category Structures

A fundamental question in the study of human cognition is how people learn to predict the category membership of an example from its properties. Leading approaches account for a wide range of data in terms of comparison to stored examples, abstractions capturing statistical regularities, or logical rules. Across three experiments, participants learned a category structure in a low-dimension, continuous-valued space consisting of regularly alternating regions of class membership (A B A B). The dependent measure was generalization performance for novel items outside the range of the training space. Human learners often extended the alternation pattern--a finding of critical interest given that leading theories of categorization based on similarity or dimensional rules fail to predict this behavior. In addition, we provide novel theoretical interpretations of the observed phenomenon.

Hearing hooves, thinking zebras: A review of the inverse base-rate effect

People often fail to use base-rate information appropriately in decision-making. This is evident in the inverse base-rate effect, a phenomenon in which people tend to predict a rare outcome for a new and ambiguous combination of cues. While the effect was first reported in 1988, it has recently seen a renewed interest from researchers concerned with learning, attention and decision-making. However, some researchers have raised concerns that the effect arises in specific circumstances and is unlikely to provide insight into general learning and decision-making processes. In this review, we critically evaluate the evidence for and against the main explanations that have been proposed to explain the effect, and identify where this evidence is currently weak. We argue that concerns about the effect are not well supported by the data. Instead, the evidence supports the conclusion that the effect is a result of general mechanisms that provides a useful opportunity to understand the processes involved in learning and decision making. We discuss gaps in our knowledge and some promising avenues for future research, including the relevance of the effect to models of attentional change in learning, an area where the phenomenon promises to contribute new insights.

Trial-by-trial dynamics of reward prediction error-associated signals during extinction learning and renewal

Reward prediction errors (RPEs) have been suggested to drive associative learning processes, but their precise temporal dynamics at the single-neuron level remain elusive. Here, we studied the neural correlates of RPEs, focusing on their trial-by-trial dynamics during an operant extinction learning paradigm. Within a single behavioral session, pigeons went through acquisition, extinction and renewal - the context-dependent response recovery after extinction. We recorded single units from the avian prefrontal cortex analogue, the nidopallium caudolaterale (NCL) and found that the omission of reward during extinction led to a peak of population activity that moved backwards in time as trials progressed. The chronological order of these signal changes during the progress of learning was indicative of temporal shifts of RPE signals that started during reward omission and then moved backwards to the presentation of the conditioned stimulus. Switches from operant choices to avoidance behavior (and vice versa) coincided with changes in population activity during the animals' decision-making. On the single unit level, we found more diverse patterns where some neurons' activity correlated with RPE signals whereas others correlated with the absolute value during the outcome period. Finally, we demonstrated that mere sensory contextual changes during the renewal test were sufficient to elicit signals likely associated with RPEs. Thus, RPEs are truly expectancy-driven since they can be elicited by changes in reward expectation, without an actual change in the quality or quantity of reward.

Generics and Alternatives

In this paper we argue that for the (probabilistic) interpretation of generic sentences of the form "Gs are f," three types of alternatives play a role: (i) alternative features of f, (ii) alternative groups, or kinds, of G, and (iii) alternative causal background factors. In the first part of this paper we argue for the relevance of these alternatives. In the second part, we describe the results of some experiments that empirically tested in particular the second use of alternatives.

Cognitive Models in Cybersecurity: Learning From Expert Analysts and Predicting Attacker Behavior

Cybersecurity stands to benefit greatly from models able to generate predictions of attacker and defender behavior. On the defender side, there is promising research suggesting that Symbolic Deep Learning (SDL) may be employed to automatically construct cognitive models of expert behavior based on small samples of expert decisions. Such models could then be employed to provide decision support for non-expert users in the form of explainable expert-based suggestions. On the attacker side, there is promising research suggesting that model-tracing with dynamic parameter fitting may be used to automatically construct models during live attack scenarios, and to predict individual attacker preferences. Predicted attacker preferences could then be exploited for mitigating risk of successful attacks. In this paper we examine how these two cognitive modeling approaches may be useful for cybersecurity professionals via two human experiments. In the first experiment participants play the role of cyber analysts performing a task based on Intrusion Detection System alert elevation. Experiment results and analysis reveal that SDL can help to reduce missed threats by 25%. In the second experiment participants play the role of attackers picking among four attack strategies. Experiment results and analysis reveal that model-tracing with dynamic parameter fitting can be used to predict (and exploit) most attackers' preferences 40-70% of the time. We conclude that studies and models of human cognition are highly valuable for advancing cybersecurity.

Rescorla-Wagner Models with Sparse Dynamic Attention

The Rescorla-Wagner (R-W) model describes human associative learning by proposing that an agent updates associations between stimuli, such as events in their environment or predictive cues, proportionally to a prediction error. While this model has proven informative in experiments, it has been posited that humans selectively attend to certain cues to overcome a problem with the R-W model scaling to large cue dimensions. We formally characterize this scaling problem and provide a solution that involves limiting attention in a R-W model to a sparse set of cues. Given the universal difficulty in selecting features for prediction, sparse attention faces challenges beyond those faced by the R-W model. We demonstrate several ways in which a naive attention model can fail explain those failures and leverage that understanding to produce a Sparse Attention R-W with Inference framework (SAR-WI). The SAR-WI framework not only satisfies a constraint on the number of attended cues, it also performs as well as the R-W model on a number of natural learning tasks, can correctly infer associative strengths, and focuses attention on predictive cues while ignoring uninformative cues. Given the simplicity of proposed alterations, we hope this work informs future development and empirical validation of associative learning models that seek to incorporate sparse attention.

Mitigating cue competition effects in human category learning

When people learn perceptual categories, if one feature makes it easy to determine the category membership, learning about other features can be reduced. In three experiments, we asked whether this cue competition effect could be fully eradicated with simple instructions. For this purpose, in a pilot experiment, we adapted a classical overshadowing paradigm into a human category learning task. Unlike previous reports, we demonstrate a robust cue competition effect with human learners. In Experiments 1 and 2, we created a new warning condition that aimed at eradicating the cue competition effect through top-down instructions. With a medium-size overshadowing effect, Experiment 1 shows a weak mitigation of the overshadowing effect. We replaced the stimuli in Experiment 2 to obtain a larger overshadowing effect and showed a larger warning effect. Nevertheless, the overshadowing effect could not be fully eradicated. These experiments suggest that cue competition effects can be a stubborn roadblock in human category learning. Theoretical and practical implications are discussed.

Why Higher Working Memory Capacity May Help You Learn: Sampling, Search, and Degrees of Approximation

Algorithms for approximate Bayesian inference, such as those based on sampling (i.e., Monte Carlo methods), provide a natural source of models of how people may deal with uncertainty with limited cognitive resources. Here, we consider the idea that individual differences in working memory capacity (WMC) may be usefully modeled in terms of the number of samples, or "particles," available to perform inference. To test this idea, we focus on two recent experiments that report positive associations between WMC and two distinct aspects of categorization performance: the ability to learn novel categories, and the ability to switch between different categorization strategies ("knowledge restructuring"). In favor of the idea of modeling WMC as a number of particles, we show that a single model can reproduce both experimental results by varying the number of particles-increasing the number of particles leads to both faster category learning and improved strategy-switching. Furthermore, when we fit the model to individual participants, we found a positive association between WMC and best-fit number of particles for strategy switching. However, no association between WMC and best-fit number of particles was found for category learning. These results are discussed in the context of the general challenge of disentangling the contributions of different potential sources of behavioral variability.

Vector-Space Models of Semantic Representation From a Cognitive Perspective: A Discussion of Common Misconceptions

Models that represent meaning as high-dimensional numerical vectors—such as latent semantic analysis (LSA), hyperspace analogue to language (HAL), bound encoding of the aggregate language environment (BEAGLE), topic models, global vectors (GloVe), and word2vec—have been introduced as extremely powerful machine-learning proxies for human semantic representations and have seen an explosive rise in popularity over the past 2 decades. However, despite their considerable advancements and spread in the cognitive sciences, one can observe problems associated with the adequate presentation and understanding of some of their features. Indeed, when these models are examined from a cognitive perspective, a number of unfounded arguments tend to appear in the psychological literature. In this article, we review the most common of these arguments and discuss (a) what exactly these models represent at the implementational level and their plausibility as a cognitive theory, (b) how they deal with various aspects of meaning such as polysemy or compositionality, and (c) how they relate to the debate on embodied and grounded cognition. We identify common misconceptions that arise as a result of incomplete descriptions, outdated arguments, and unclear distinctions between theory and implementation of the models. We clarify and amend these points to provide a theoretical basis for future research and discussions on vector models of semantic representation.

How competitive is cue competition?

Cue competition refers to phenomena indicating that learning about the relationship between a cue and an outcome is influenced by learning about the predictive significance of other cues that are concurrently present. In two autoshaping experiments with pigeons, we investigated the strength of competition among cues for predictive value. In each experiment, animals received an overexpectation training (A+, D+ followed by AD+). In addition, the training schedule of each experiment comprised two control conditions-one condition to evaluate the presence of overexpectation (B+ followed by BY+) and a second one to assess the strength of competition among cues (C+ followed by CZ-). Training trials were followed by a test with individual stimuli (A, B, C). Experiment 1 revealed no evidence for cue competition as responding during the test mirrored the individual cue-outcome contingencies. The test results from Experiment 2, which included an outcome additivity training, showed cue competition in form of an overexpectation effect as responding was weaker for Stimulus A than Stimulus B. However, the test results from Experiment 2 also revealed that responding to Stimulus A was stronger than to Stimulus C, which indicates that competition among cues was not as strong as predicted by some influential theories of associative learning.

Relation-Based Categorization and Category Learning as a Result From Structural Alignment. The RoleMap Model

Relational categories are structure-based categories, defined not only by their internal properties but also by their extrinsic relations with other categories. For example, predator could not be defined without referring to hunt and prey. Even though they are commonly used, there are few models taking into account any relational information. A category learning and categorization model aiming to fill this gap is presented. Previous research addresses the hypothesis that the acquisition and the use of relational categories are underlined by structural alignment. That is why the proposed RoleMap model is based on mechanisms often studied as the analogy-making sub-processes, developed on a suitable for this cognitive architecture. RoleMap is conceived in such a way that relation-based category learning and categorization emerge while other tasks are performed. The assumption it steps on is that people constantly make structural alignments between what they experience and what they know. During these alignments various mappings and anticipations emerge. The mappings capture commonalities between the target (the representation of the current situation) and the memory, while the anticipations try to fill the missing information in the target, based on the conceptual system. Because some of the mappings are highly important, they are transformed into a distributed representation of a new concept for further use, which denotes the category learning. When some knowledge is missing in the target, meaning it is uncategorized, that knowledge is transferred from memory in the form of anticipations. The wining anticipation is transformed into a category member, denoting the act of categorization. The model’s behavior emerges from the competition between these two pressures – to categorize and to create new categories. Several groups of simulations demonstrate that the model can deal with relational categories in a context-dependent manner and to account for single-shot learning, challenging most of the existing approaches to category learning. The model also simulates previous empirical data pointing to the thematic categories and to the puzzling inverse base-rate effect. Finally, the model’s strengths and limitations are evaluated.

Cue Competition and Incidental Learning: No Blocking or Overshadowing in the Colour-Word Contingency Learning Procedure Without Instructions to Learn

Overshadowing and blocking are two important findings that are frequently used to constrain models of associative learning. Overshadowing is the finding that learning about a cue (referred to as X) is reduced when that cue is always accompanied by a second cue (referred to as A) during the learning phase (AX). Blocking is the finding that after learning a stimulus-outcome relation for one stimulus (A), learning about a second stimulus (X) is reduced when the second stimulus is always accompanied by the first stimulus (AX). It remains unclear whether overshadowing and blocking result from explicit decision processes (e.g., “I know that A predicts the outcome, so I am not sure whether X does, too”), or whether cue competition is built directly into low-level association formation processes. In that vein, the present work examined whether overshadowing and/or blocking are present in an incidental learning procedure, where the predictive stimuli (words or shapes) are irrelevant to the cover task and merely correlated with the task-relevant stimulus dimension (colour). In two large online studies, we observed no evidence for overshadowing or blocking in this setup: (a) no evidence for an overshadowing cost was observed with compound (word-shape) cues relative to single cue learning conditions, and (b) contingency learning effects for blocked stimuli did not differ from those for blocking stimuli. However, when participants were given the explicit instructions to learn contingencies, evidence for blocking and overshadowing was observed. Together, these results suggest that contingencies of blocked/overshadowed stimuli are learned incidentally, but are suppressed by explicit decision processes due to knowledge of the contingencies for the blocking/overshadowing stimuli.

Viewing Adaptive Social Choice Through the Lens of Associative Learning

Because humans live in a dynamic and evolving social world, modeling the factors that guide social behavior has remained a challenge for psychology. In contrast, much progress has been made on understanding some of the more basic elements of human behavior, such as associative learning and memory, which has been successfully modeled in other species. Here we argue that applying an associative learning approach to social behavior can offer valuable insights into the human moral experience. We propose that the basic principles of associative learning-conserved across a range of species-can, in many situations, help to explain seemingly complex human behaviors, including altruistic, cooperative, and selfish acts. We describe examples from the social decision-making literature using Pavlovian learning phenomena (e.g., extinction, cue competition, stimulus generalization) to detail how a history of positive or negative social outcomes influences cognitive and affective mechanisms that shape moral choice. Examining how we might understand social behaviors and their likely reliance on domain-general mechanisms can help to generate testable hypotheses to further understand how social value is learned, represented, and expressed behaviorally.

Computational and empirical simulations of selective memory impairments: Converging evidence for a single-system account of memory dissociations

Current theory has divided memory into multiple systems, resulting in a fractionated account of human behaviour. By an alternative perspective, memory is a single system. However, debate over the details of different single-system theories has overshadowed the converging agreement among them, slowing the reunification of memory. Evidence in favour of dividing memory often takes the form of dissociations observed in amnesia, where amnesic patients are impaired on some memory tasks but not others. The dissociations are taken as evidence for separate explicit and implicit memory systems. We argue against this perspective. We simulate two key dissociations between classification and recognition in a computational model of memory, A Theory of Nonanalytic Association. We assume that amnesia reflects a quantitative difference in the quality of encoding. We also present empirical evidence that replicates the dissociations in healthy participants, simulating amnesic behaviour by reducing study time. In both analyses, we successfully reproduce the dissociations. We integrate our computational and empirical successes with the success of alternative models and manipulations and argue that our demonstrations, taken in concert with similar demonstrations with similar models, provide converging evidence for a more general set of single-system analyses that support the conclusion that a wide variety of memory phenomena can be explained by a unified and coherent set of principles.

Associative and Normative Accounts of Negative Transfer

Subjects were given a prediction task in which they had to learn that one cue, P (positive), was followed by the outcome, and a compound of P and another cue, N (negative), was not followed by the outcome. Next, N was tested in compound with a transfer cue, T, which had signalled the outcome but had never been compounded with N. Experiment 1 confirmed an important assumption of the Rescorla–Wagner model (Wagner & Rescorla, 1972) that negation of T should depend on the specific P cue compound with N being positively contingent. Experiments 2 and 3 confirmed the model's prediction that no decrement in negative transfer should be observed following postlearning devaluation of P. Unfortunately, the model did not anticipate that a large proportion of devaluation trials relative to learning trials would attenuate negative transfer (Experiment 4), nor did it predict that negative transfer would occur when P was neutral during the learning stage and was only later made positive (Experiment 3). The results can be accommodated by the Rescorla–Wagner model if one assumes that absent cues have their associative strengths altered.

Conditioned Inhibition in a Multiple Category Learning Task

Four experiments investigated inhibition that might arise in a task in which cues are associated with more than one outcome. In each experiment, human subjects played a game called “Clues and Culprits” in which they were asked to judge the predictive strength of clues that had been associated with culprits in a series of hypothetical crimes. In a two-outcome version of the familiar conditioned inhibition paradigm (A+, AX-), one clue was paired with one culprit on its own, but it was paired with a second culprit when it was combined with a second clue (A-1, AX-2). According to the delta rule, X should acquire inhibition for the first culprit; it should also acquire more inhibition than a differential cue merely associated with a second culprit (e.g. A-1, X-2). Inhibition was found with both procedures. However, the amount of inhibition did not differ between them, suggesting that mere association with a second outcome was sufficient to inhibit performance based on the first. Other data suggested the presence of cue competition. Also, when a cue associated with one culprit was paired with a second culprit on other trials, there was little evidence of unlearning of the first association.

Contingency judgements on the fly

The present research was conducted to establish the validity of a novel procedure for measuring human contingency judgements aimed at shortening the length of conventional procedures. Cues and outcomes were simple geometric shapes that were presented in a rapid streaming fashion, reducing the length of a block of trials from several minutes to a few seconds. We establish the reliability of the procedure by replicating two central findings in the contingency judgement literature, and we elaborate on the importance of this method for future research.

Generalization in Human Category Learning: A Connectionist Account of Differences in Gradient after Discriminative and Non discriminative Training

Two experiments are reported that investigate the difference in gradient of generalization observed between one-category (non-discriminative) and two-category (discriminative) training. Extrapolating from the results of a number of animal learning studies, it was predicted that the gradient should be steeper under discriminative training. The first experiment confirms this basic prediction for the stimuli used, which were novel, prototype-structured, and constructed from 12 symbols positioned on a grid. An explanation for the effect, based on the Rescorla-Wagner theory of Pavlovian conditioning (Rescorla & Wagner, 1972), is that under non-discriminative training “incidental stimuli” have significant control over responding, whereas under discriminative training they do not. Incidental stimuli are those aspects of the stimulus, or the surrounding context, that are not differentially reinforced under discriminative training. This explanation leads to the prediction that a comparable effect of blocked versus intermixed discriminative training should also be found. This prediction is disconfirmed by the second experiment. An alternative model, still based on the Rescorla Wagner theory but with the addition of a decision mechanism comprising a threshold unit and a competitive network system, is proposed, and its ability to predict both the choice probabilities and the pattern of response times found is evaluated via simulation.

Inattentional blindness for negative relationships in human causal learning

The present study focuses on the effect of selective attention on causal learning. Three effects of the level of attention to predictive symptoms in positive and negative contingency learning tasks are reported. First, participants accurately detected a positive relationship between an incidental cue and a contingent outcome, although judgements were slightly lower than those for the attended cue. Second, participants were unable to detect negative relationships between incidental cues and outcomes, which suggests a major role of selective attention in this type of learning. Third, participants retrieved the frequency of each trial type more accurately in the attended conditions than in the incidental conditions. These findings show how attention guides and constrains human causal learning and reveal an inattentional blindness effect for negative contingency learning.

Evidence for Rule-Based Processes in the Inverse Base-Rate Effect

Three studies provide convergent evidence that the inverse base-rate effect (Medin & Edelson, 1988) is mediated by rule-based cognitive processes. Experiment 1 shows that, in contrast to adults, prior to the formal operational stage most children do not exhibit the inverse base-rate effect. Experiments 2 and 3 demonstrate that an adult sample is a mix of participants relying on associative processes who categorize according to the base-rate and participants relying on rule-based processes who exhibit a strong inverse base-rate effect. The distribution of the effect is bimodal, and removing participants independently classified as prone to rule-based processing effectively eliminates the inverse base-rate effect. The implications for current explanations of the inverse base-rate effect are discussed.

Associationism and cognition: Human contingency learning at 25

A major topic within human learning, the field of contingency judgement, began to emerge about 25 years ago following publication of an article on depressive realism by Alloy and Abramson (1979). Subsequently, associationism has been the dominant theoretical framework for understanding contingency learning but this has been challenged in recent years by an alternative cognitive or inferential approach. This article outlines the key conceptual differences between these approaches and summarizes some of the main methods that have been employed to distinguish between them.