From conditioning to category learning: An adaptive network model

The purpose of experiments: Ecological validity versus comparing hypotheses

As illustrated by research Koehler himself cites (Dawes et al. 1993), the purpose of experiments is to choose between contrasting explanations of past observations – rather than to seek statistical generalizations about the prevalence of effects. True external validity results not from sampling various problems that are representative of “real world” decision making, but from reproducing an effect in the laboratory with minimal contamination (including from real world factors).

Physicians neglect base rates, and it matters

A recent study showed physicians' reasoning about a realistic case to be ignorant of base rate. It also showed physicians interpreting information pertinent to base rate differently, depending on whether it was presented early or late in the case. Although these adult reasoners might do better if given hints through talk of relative frequencies, this would not prove that they had no problem of base rate neglect.

Why do frequency formats improve Bayesian reasoning? Cognitive algorithms work on information, which needs representation

In contrast to traditional research on base-rate neglect, an ecologically-oriented research program would analyze the correspondence between cognitive algorithms and the nature of information in the environment. Bayesian computations turn out to be simpler when information is represented in frequency formats as opposed to the probability formats used in previous research. Frequency formats often enable even uninstructed subjects to perform Bayesian reasoning.

Base rates, experience, and the big picture

The important question is how people process probabilistic information, not whether they process it in accordance with a normative model that we never should have expected them to be capable of following. Experience is not the cure, as widely thought, to problems with utilizing base rate information.

Base rates, stereotypes, and judgmental accuracy

The base rate literature has an opposite twin in the social psychological literature on stereotypes, which concludes that people use their preexisting beliefs about probabilistic category attributes too much, rather than not enough. This ironic discrepancy arises because beliefs about category attributes enhance accuracy when the beliefs are accurate and diminish accuracy when they are not. To determine the accuracy of base rate/stereotype beliefs requires research that addresses specific content.

Characteristics of dissociable human learning systems

A number of ways of taxonomizing human learning have been proposed. We examine the evidence for one such proposal, namely, that there exist independent explicit and implicit learning systems. This combines two further distinctions, (1) between learning that takes place with versus without concurrent awareness, and (2) between learning that involves the encoding of instances (or fragments) versus the induction of abstract rules or hypotheses. Implicit learning is assumed to involve unconscious rule learning. We examine the evidence for implicit learning derived from subliminal learning, conditioning, artificial grammar learning, instrumental learning, and reaction times in sequence learning. We conclude that unconscious learning has not been satisfactorily established in any of these areas. The assumption that learning in some of these tasks (e.g., artificial grammar learning) is predominantly based on rule abstraction is questionable. When subjects cannot report the “implicitly learned” rules that govern stimulus selection, this is often because their knowledge consists of instances or fragments of the training stimuli rather than rules. In contrast to the distinction between conscious and unconscious learning, the distinction between instance and rule learning is a sound and meaningful way of taxonomizing human learning. We discuss various computational models of these two forms of learning.

Cognitive algebra versus representativeness heuristic

Cognitive algebra strongly disproved the representativeness heuristic almost before it was published; and therewith it also disproved the base rate fallacy. Cognitive algebra provides a theoretical foundation for judgment-decision theory through its joint solution to the two fundamental problems – true measurement of subjective values, and cognitive rules for integration of multiple determinants.

Throwing out the baby with the bathwater? Let's not overstate the overselling of the base rate fallacy

Koehler's summary and critique of research on the base rate fallacy is cogent and persuasive. However, he may have overstated the case, and his suggestions for future research may be too restrictive. We agree that methodological approaches to this topic should be broadened, but we argue that experimental laboratory research and the Bayesian normative standard are useful and should not be abandoned.

Probabilistic fallacies

Two distinct issues are sometimes confused in the base rate literature: Why do people make logical mistakes in the assessment of probabilities? and why do subjects not use base rates the way experimenters do? The latter problem may often reflect differences in an implicit reference class rather than a disinclination to update a base rate by Bayes' theorem. Also important are considerations concerning the interaction of several potentially relevant base rates.

Which reference class is evoked?

Any instance (i.e., event, behavior, trait) belongs to infinitely many reference classes, hence there are infinitely many base rates from which to choose. People clearly do not entertain all possible reference classes, however, so something must be limiting the search space. We suggest some possible mechanisms that determine which reference class is evoked for the purpose of judgment and decision.

Critical and natural sensitivity to base rates

This commentary discusses three points: (1) The implications of the fact that it is rational to ignore base rates if probabilities are estimated by frequencies from samples without missing data (natural sampling); (2) second order probabilities distributions are a plausible way to model imprecise probabilities; and (3) Bayesian networks represent a normative reference for multi-cue models of probabilistic inference.

The base rate controversy: Is the glass half-full or half-empty?

Setting the two hypotheses of complete neglect and full use of base rates against each other is inappropriate. The proper question concerns the degree to which base rates are used (or neglected), and under what conditions. We outline alternative approaches and recommend regression analysis. Koehler's conclusion that we have been oversold on the base rate fallacy seems to be premature.

First things first: What is a base rate?

The fallacy beneath the base rate fallacy is that we know what a base rate is. We talk as if base rates and individuating information were two different kinds of information. From a Bayesian perspective, however, the only difference between base rate and individuating information is – which comes first.

Explanatory coherence

This target article presents a new computational theory of explanatory coherence that applies to the acceptance and rejection of scientific hypotheses as well as to reasoning in everyday life. The theory consists of seven principles that establish relations of local coherence between a hypothesis and other propositions. A hypothesis coheres with propositions that it explains, or that explain it, or that participate with it in explaining other propositions, or that offer analogous explanations. Propositions are incoherent with each other if they are contradictory. Propositions that describe the results of observation have a degree of acceptability on their own. An explanatory hypothesis is accepted if it coheres better overall than its competitors. The power of the seven principles is shown by their implementation in a connectionist program called ECHO, which treats hypothesis evaluation as a constraint satisfaction problem. Inputs about the explanatory relations are used to create a network of units representing propositions, while coherence and incoherence relations are encoded by excitatory and inhibitory links. ECHO provides an algorithm for smoothly integrating theory evaluation based on considerations of explanatory breadth, simplicity, and analogy. It has been applied to such important scientific cases as Lavoisier's argument for oxygen against the phlogiston theory and Darwin's argument for evolution against creationism, and also to cases of legal reasoning. The theory of explanatory coherence has implications for artificial intelligence, psychology, and philosophy.

Structure learning in a sensorimotor association task

Learning is often understood as an organism's gradual acquisition of the association between a given sensory stimulus and the correct motor response. Mathematically, this corresponds to regressing a mapping between the set of observations and the set of actions. Recently, however, it has been shown both in cognitive and motor neuroscience that humans are not only able to learn particular stimulus-response mappings, but are also able to extract abstract structural invariants that facilitate generalization to novel tasks. Here we show how such structure learning can enhance facilitation in a sensorimotor association task performed by human subjects. Using regression and reinforcement learning models we show that the observed facilitation cannot be explained by these basic models of learning stimulus-response associations. We show, however, that the observed data can be explained by a hierarchical Bayesian model that performs structure learning. In line with previous results from cognitive tasks, this suggests that hierarchical Bayesian inference might provide a common framework to explain both the learning of specific stimulus-response associations and the learning of abstract structures that are shared by different task environments.

The effects of feature-label-order and their implications for symbolic learning

Symbols enable people to organize and communicate about the world. However, the ways in which symbolic knowledge is learned and then represented in the mind are poorly understood. We present a formal analysis of symbolic learning-in particular, word learning-in terms of prediction and cue competition, and we consider two possible ways in which symbols might be learned: by learning to predict a label from the features of objects and events in the world, and by learning to predict features from a label. This analysis predicts significant differences in symbolic learning depending on the sequencing of objects and labels. We report a computational simulation and two human experiments that confirm these differences, revealing the existence of Feature-Label-Ordering effects in learning. Discrimination learning is facilitated when objects predict labels, but not when labels predict objects. Our results and analysis suggest that the semantic categories people use to understand and communicate about the world can only be learned if labels are predicted from objects. We discuss the implications of this for our understanding of the nature of language and symbolic thought, and in particular, for theories of reference.

Synaptic computation underlying probabilistic inference

We propose that synapses may be the workhorse of the neuronal computations that underlie probabilistic reasoning. We built a neural circuit model for probabilistic inference in which information provided by different sensory cues must be integrated and the predictive powers of individual cues about an outcome are deduced through experience. We found that bounded synapses naturally compute, through reward-dependent plasticity, the posterior probability that a choice alternative is correct given that a cue is presented. Furthermore, a decision circuit endowed with such synapses makes choices on the basis of the summed log posterior odds and performs near-optimal cue combination. The model was validated by reproducing salient observations of, and provides insights into, a monkey experiment using a categorization task. Our model thus suggests a biophysical instantiation of the Bayesian decision rule, while predicting important deviations from it similar to the 'base-rate neglect' observed in human studies when alternatives have unequal prior probabilities.

Induction with uncertain categories: When do people consider the category alternatives?

These three experiments examined how people make property inferences about exemplars whose category membership is uncertain. Participants were shown two categories and a novel exemplar with a feature that indicated that the exemplar was more likely to belong to one category (target) than to the other (nontarget). Participants then made categorization decisions and property inferences about the novel exemplar. In some conditions, property inferences could be made only by considering both target and nontarget categories. In other conditions, predictions could be based on both categories or on the target category alone. Consistent with previous studies (e.g., Murphy & Ross, 1994, 2005), we found that many people made predictions based only on consideration of the target category. However, the prevalence of such single-category reasoning was greatly reduced by highlighting the costs of neglecting nontarget alternatives and by asking for inferences before categorization decisions. The results suggest that previous work may have exaggerated the prevalence of single-category reasoning and that people may be more flexible in their use of multiple categories in property inference than has been previously recognized.

Implicit learning in aging: extant patterns and new directions

Research suggests that the striatum plays an important role in implicit learning (IL). The striatum exhibits marked age-related morphological and neurochemical losses. Yet, behavioral studies suggest that IL is generally well preserved in old age, and that age-related differences emerge only when highly complex IL tasks are used. In this review, we integrate behavioral and neuroimaging evidence on IL in aging. We suggest that relative stability of IL in old age may reflect neural reorganization that compensates for age-related losses in striatal functions. Specifically, there may be an age-related increase in reliance on extrastriatal regions (e.g., medial-temporal, frontal) during IL. This reorganization of function may be beneficial under less taxing performance conditions, but not when task demands become more challenging.

Do people treat missing information adaptively when making inferences?

When making inferences, people are often confronted with situations with incomplete information. Previous research has led to a mixed picture about how people react to missing information. Options include ignoring missing information, treating it as either positive or negative, using the average of past observations for replacement, or using the most frequent observation of the available information as a placeholder. The accuracy of these inference mechanisms depends on characteristics of the environment. When missing information is uniformly distributed, it is most accurate to treat it as the average, whereas when it is negatively correlated with the criterion to be judged, treating missing information as if it were negative is most accurate. Whether people treat missing information adaptively according to the environment was tested in two studies. The results show that participants were sensitive to how missing information was distributed in an environment and most frequently selected the mechanism that was most adaptive. From these results the authors conclude that reacting to missing information in different ways is an adaptive response to environmental characteristics.

A neurocomputational model of classical conditioning phenomena: a putative role for the hippocampal region in associative learning

Some existing models of hippocampal function simulate performance in classical conditioning tasks using the error backpropagation algorithm to guide learning (Gluck, M.A., and Myers, C.E., (1993). Hippocampal mediation of stimulus representation: a computational theory. Hippocampus, 3(4), 491-516.). This algorithm is not biologically plausible because it requires information to be passed backward through layers of nodes and assumes that the environment provides information to the brain about what correct outputs should be. Here, we show that the same information-processing function proposed for the hippocampal region in the Gluck and Myers (1993) model can also be implemented in a network without using the backpropagation algorithm. Instead, our newer instantiation of the theory uses only (a) Hebbian learning methods which match more closely with synaptic and associative learning mechanisms ascribed to the hippocampal region and (b) a more plausible representation of input stimuli. We demonstrate here that this new more biologically plausible model is able to simulate various behavioral effects, including latent inhibition, acquired equivalence, sensory preconditioning, negative patterning, and context shift effects. In addition, the newer model is able to address some new phenomena including the effect of the number of training trials on blocking and overshadowing.

Inferences from memory: strategy- and exemplar-based judgment models compared

What are the cognitive processes underlying people's inferences from memory? To provide an answer, the exemplar-based approach to predicting people's inferences is tested against the strategy-based approach. Exemplar models assume that people make inferences about objects by retrieving similar objects from memory. In contrast, the strategy-based approach assumes that people select cognitive strategies that make inferences based on abstracted knowledge and information the inference situation provides. In Experiment 1, in which dichotomous feedback on the level of pair-comparisons was provided, almost all participants were classified as using a simple lexicographic strategy. In Experiment 2, in which continuous feedback for single objects was provided, most participants were classified as using a compensatory strategy. Both experiments suggest that the strategy-based approach is more suitable for predicting people's inferences from memory than the exemplar-based approach. The strategy-based approach shows how people adapt to inference situations by selecting different cognitive strategies.

The role of the orbitofrontal cortex in human discrimination learning

Several lines of evidence implicate the prefrontal cortex in learning but there is little evidence from studies of human lesion patients to demonstrate the critical role of this structure. To this end, we tested patients with lesions of the frontal lobe (n=36) and healthy controls (n=35) on two learning tasks: the weather prediction task (WPT), and an eight-pair concurrent visual discrimination task ('Choose'). Performance of both tasks was previously shown to be disrupted in patients with Parkinson's disease; the Choose deficit was only present when patients were medicated. Patients with damage to the orbitofrontal cortex (OFC) were significantly impaired on Choose, compared to both healthy controls and non-OFC lesion patients. The OFC lesion patients showed a mild deficit on the first 50 trials of the WPT, compared to the control subjects but not non-OFC lesion patients. The selective deficit in the OFC patients on Choose performance could not be attributed to the larger lesion size in this group, and the deficit was not correlated with the volume of damage to adjacent prefrontal subregions (e.g. anterior cingulate cortex). These data support the notion that the OFC play a role in normal discrimination learning, and suggest qualitative similarities in learning performance of patients with OFC damage and medicated PD patients.

Saccade selection when reward probability is dynamically manipulated using Markov chains

Markov chains (stochastic processes where probabilities are assigned based on the previous outcome) are commonly used to examine the transitions between behavioral states, such as those that occur during foraging or social interactions. However, relatively little is known about how well primates can incorporate knowledge about Markov chains into their behavior. Saccadic eye movements are an example of a simple behavior influenced by information about probability, and thus are good candidates for testing whether subjects can learn Markov chains. In addition, when investigating the influence of probability on saccade target selection, the use of Markov chains could provide an alternative method that avoids confounds present in other task designs. To investigate these possibilities, we evaluated human behavior on a task in which stimulus reward probabilities were assigned using a Markov chain. On each trial, the subject selected one of four identical stimuli by saccade; after selection, feedback indicated the rewarded stimulus. Each session consisted of 200-600 trials, and on some sessions, the reward magnitude varied. On sessions with a uniform reward, subjects (n = 6) learned to select stimuli at a frequency close to reward probability, which is similar to human behavior on matching or probability classification tasks. When informed that a Markov chain assigned reward probabilities, subjects (n = 3) learned to select the greatest reward probability more often, bringing them close to behavior that maximizes reward. On sessions where reward magnitude varied across stimuli, subjects (n = 6) demonstrated preferences for both greater reward probability and greater reward magnitude, resulting in a preference for greater expected value (the product of reward probability and magnitude). These results demonstrate that Markov chains can be used to dynamically assign probabilities that are rapidly exploited by human subjects during saccade target selection.

Prior knowledge enhances the category dimensionality effect

A study of the combined influence of prior knowledge and stimulus dimensionality on category learning was conducted. Subjects learned category structures with the same number of necessary dimensions but with more or fewer additional, redundant dimensions and with either knowledge-related or knowledge-unrelated features. Minimal-learning models predict that all subjects, regardless of condition, either should learn the same number of dimensions or should respond more slowly to each dimension. Despite similar learning rates and response times, subjects learned more features in the high-dimensional than in the low-dimensional condition. Furthermore, prior knowledge interacted with dimensionality, increasing what was learned, especially in the high-dimensional case. A second experiment confirmed that the participants did, in fact, learn more features during the training phase, rather than simply inferring them at test. These effects can be explained by direct associations among features (representing prior knowledge), combined with feedback between features and the category label, as was shown by simulations of the knowledge resonance, or KRES, model of category learning.

A rose in any other font would not smell as sweet: Effects of perceptual fluency on categorization

Fluency--the ease with which people process information--is a central piece of information we take into account when we make judgments about the world. Prior research has shown that fluency affects judgments in a wide variety of domains, including frequency, familiarity, and confidence. In this paper, we present evidence that fluency also plays a role in categorization judgments. In Experiment 1, participants judged a variety of different exemplars to be worse category members if they were less fluent (because they were presented in a smaller typeface). In Experiment 2, we found that fluency also affected judgments of feature typicality. In Experiment 3, we demonstrated that the effects of fluency can be reversed when a salient attribution for reduced fluency is available (i.e., the stimuli are hard to read because they were printed by a printer with low toner). In Experiment 4 we replicated these effects using a within-subject design, which ruled out the possibility that the effects were a statistical artifact caused by aggregation of data. We propose a possible mechanism for these effects: if an exemplar and its category are closely related, activation of one will cause priming of the other, leading to increased fluency. Over time, feelings of fluency come to be used as a valid cue that can become confused with more traditional sources of information about category membership.

The importance of learning when making inferences

The assumption that people possess a repertoire of strategies to solve the inference problems they face has been made repeatedly. The experimental findings of two previous studies on strategy selection are reexamined from a learning perspective, which argues that people learn to select strategies for making probabilistic inferences. This learning process is modeled with the strategy selection learning (SSL) theory, which assumes that people develop subjective expectancies for the strategies they have. They select strategies proportional to their expectancies, which are updated on the basis of experience. For the study by Newell, Weston, and Shanks (2003) it can be shown that people did not anticipate the success of a strategy from the beginning of the experiment. Instead, the behavior observed at the end of the experiment was the result of a learning process that can be described by the SSL theory. For the second study, by Bröder and Schiffer (2006), the SSL theory is able to provide an explanation for why participants only slowly adapted to new environments in a dynamic inference situation. The reanalysis of the previous studies illustrates the importance of learning for probabilistic inferences.