On Two Types of Conditioned Reflex

Neurophysiology of male sexual arousal-Behavioral perspective

In the presented review, we analyzed the physiology of male sexual arousal and its relation to the motivational aspects of this behavior. We highlighted the distinction between these processes based on observable physiological and behavioral parameters. Thus, we proposed the experimentally applicable differentiation between sexual arousal (SA) and sexual motivation (SM). We propose to define sexual arousal as an overall autonomic nervous system response leading to penile erection, triggered selectively by specific sexual cues. These autonomic processes include both spinal and supraspinal neuronal networks, activated by sensory pathways including information from sexual partner and sexual context, as well as external and internal genital organs. To avoid misinterpretation of experimental data, we also propose to precise the term "sexual motivation" as all actions performed by the individual that increase the probability of sexual interactions or increase the probability of exposition to sexual context cues. Neuronal structures such as the amygdala, bed nucleus of stria terminalis, hypothalamus, nucleus raphe, periaqueductal gray, and nucleus paragigantocellularis play crucial roles in controlling the level of arousal and regulating peripheral responses via specific autonomic effectors. On the highest level of CNS, the activity of cortical structures involved in the regulation of the autonomic nervous system, such as the insula and anterior cingulate cortex, can visualize an elevated level of SA in both animal and human brains. From a preclinical perspective, we underlie the usefulness of the non-contact erection test (NCE) procedure in understanding factors influencing sexual arousal, including studies of sexual preference in animal models. Taken together results obtained by different methods, we wanted to focus attention on neurophysiological aspects that are distinctly related to sexual arousal and can be used as an objective parameter, leading to higher translational transparency between basic, preclinical, and clinical studies.

Teaching Behavior Analysis through Its History: Narrative and Stories

Four articles appear in a special section of the current issue of this journal. Each offers methods for introducing students to the history of behavior analysis. Their distinctive approaches vary from delineating a course addressed specifically to history, to combining issues in behavior analysis with those within related fields, or to splicing historical events or methods into various courses within behavior analysis. I sketch these briefly to encourage readers to read them directly before proposing that the history of our field can also be understood both as an overarching narrative and as a collection of stories. Boje (2008) distinguishes between the two by characterizing narrative as a rather formal, organized account, on the one hand, with stories, on the other hand, being more disorderly episodes of behavior-in-process. Each has its roles for introducing behavior analysis-and even for effectively understanding it ourselves-and thus, the best place of each within strategies of teaching, bears systematic examination. Although narrative supplies an organized account, stories more strongly engage the reader. Stories are especially effective at keeping the reader or listener engaged when they entail nested relations delineated by establishing stimuli. Besides offering a principle of organization, this formulation yields a strategy for using stories to enable the overarching narrative to sustain the reader's or listener's behavior.

Safety learning and the Pavlovian conditioned inhibition of fear in humans: Current state and future directions

Safety learning occurs when an otherwise neutral stimulus comes to signal the absence of threat, allowing organisms to use safety information to inhibit fear and anxiety in nonthreatening environments. Although it continues to emerge as a topic of relevance in biological and clinical psychology, safety learning remains inconsistently defined and under-researched. Here, we analyse the Pavlovian conditioned inhibition paradigm and its application to the study of safety learning in humans. We discuss existing studies; address outstanding theoretical considerations; and identify prospects for its further application. Though Pavlovian conditioned inhibition presents a theoretically sound model of safety learning, it has been investigated infrequently, with decade-long interims between some studies, and notable methodological variability. Consequently, we argue that the full potential of conditioned inhibition as a model for human safety learning remains untapped, and propose that it could be revisited as a framework for addressing timely questions in the behavioural and clinical sciences.

A-learning: A new formulation of associative learning theory

We present a new mathematical formulation of associative learning focused on non-human animals, which we call A-learning. Building on current animal learning theory and machine learning, A-learning is composed of two learning equations, one for stimulus-response values and one for stimulus values (conditioned reinforcement). A third equation implements decision-making by mapping stimulus-response values to response probabilities. We show that A-learning can reproduce the main features of: instrumental acquisition, including the effects of signaled and unsignaled non-contingent reinforcement; Pavlovian acquisition, including higher-order conditioning, omission training, autoshaping, and differences in form between conditioned and unconditioned responses; acquisition of avoidance responses; acquisition and extinction of instrumental chains and Pavlovian higher-order conditioning; Pavlovian-to-instrumental transfer; Pavlovian and instrumental outcome revaluation effects, including insight into why these effects vary greatly with training procedures and with the proximity of a response to the reinforcer. We discuss the differences between current theory and A-learning, such as its lack of stimulus-stimulus and response-stimulus associations, and compare A-learning with other temporal-difference models from machine learning, such as Q-learning, SARSA, and the actor-critic model. We conclude that A-learning may offer a more convenient view of associative learning than current mathematical models, and point out areas that need further development.

Learning, Reward, and Decision Making

In this review, we summarize findings supporting the existence of multiple behavioral strategies for controlling reward-related behavior, including a dichotomy between the goal-directed or model-based system and the habitual or model-free system in the domain of instrumental conditioning and a similar dichotomy in the realm of Pavlovian conditioning. We evaluate evidence from neuroscience supporting the existence of at least partly distinct neuronal substrates contributing to the key computations necessary for the function of these different control systems. We consider the nature of the interactions between these systems and show how these interactions can lead to either adaptive or maladaptive behavioral outcomes. We then review evidence that an additional system guides inference concerning the hidden states of other agents, such as their beliefs, preferences, and intentions, in a social context. We also describe emerging evidence for an arbitration mechanism between model-based and model-free reinforcement learning, placing such a mechanism within the broader context of the hierarchical control of behavior.

An embodied biologically constrained model of foraging: from classical and operant conditioning to adaptive real-world behavior in DAC-X

Animals successfully forage within new environments by learning, simulating and adapting to their surroundings. The functions behind such goal-oriented behavior can be decomposed into 5 top-level objectives: 'how', 'why', 'what', 'where', 'when' (H4W). The paradigms of classical and operant conditioning describe some of the behavioral aspects found in foraging. However, it remains unclear how the organization of their underlying neural principles account for these complex behaviors. We address this problem from the perspective of the Distributed Adaptive Control theory of mind and brain (DAC) that interprets these two paradigms as expressing properties of core functional subsystems of a layered architecture. In particular, we propose DAC-X, a novel cognitive architecture that unifies the theoretical principles of DAC with biologically constrained computational models of several areas of the mammalian brain. DAC-X supports complex foraging strategies through the progressive acquisition, retention and expression of task-dependent information and associated shaping of action, from exploration to goal-oriented deliberation. We benchmark DAC-X using a robot-based hoarding task including the main perceptual and cognitive aspects of animal foraging. We show that efficient goal-oriented behavior results from the interaction of parallel learning mechanisms accounting for motor adaptation, spatial encoding and decision-making. Together, our results suggest that the H4W problem can be solved by DAC-X building on the insights from the study of classical and operant conditioning. Finally, we discuss the advantages and limitations of the proposed biologically constrained and embodied approach towards the study of cognition and the relation of DAC-X to other cognitive architectures.

Electrophysiological CNS-processes related to associative learning in humans

The neurophysiology of human associative memory has been studied with electroencephalographic techniques since the 1930s. This research has revealed that different types of electrophysiological processes in the human brain can be modified by conditioning: sensory evoked potentials, sensory induced gamma-band activity, periods of frequency-specific waves (alpha and beta waves, the sensorimotor rhythm and the mu-rhythm) and slow cortical potentials. Conditioning of these processes has been studied in experiments that either use operant conditioning or repeated contingent pairings of conditioned and unconditioned stimuli (classical conditioning). In operant conditioning, the appearance of a specific brain process is paired with an external stimulus (neurofeedback) and the feedback enables subjects to obtain varying degrees of control of the CNS-process. Such acquired self-regulation of brain activity has found practical uses for instance in the amelioration of epileptic seizures, Autism Spectrum Disorders (ASD) and Attention Deficit Hyperactivity Disorder (ADHD). It has also provided communicative means of assistance for tetraplegic patients through the use of brain computer interfaces. Both extra and intracortically recorded signals have been coupled with contingent external feedback. It is the aim for this review to summarize essential results on all types of electromagnetic brain processes that have been modified by classical or operant conditioning. The results are organized according to type of conditioned EEG-process, type of conditioning, and sensory modalities of the conditioning stimuli.

The learning of fear extinction

Recent work on the extinction of fear-motivated learning places emphasis on its putative circuitry and on its modulation. Extinction is the learned inhibition of retrieval of previously acquired responses. Fear extinction is used as a major component of exposure therapy in the treatment of fear memories such as those of the posttraumatic stress disorder (PTSD). It is initiated and maintained by interactions between the hippocampus, basolateral amygdala and ventromedial prefrontal cortex, which involve feedback regulation of the latter by the other two areas. Fear extinction depends on NMDA receptor activation. It is positively modulated by d-serine acting on the glycine site of NMDA receptors and blocked by AP5 (2-amino-5-phosphono propionate) in the three structures. In addition, histamine acting on H2 receptors and endocannabinoids acting on CB1 receptors in the three brain areas mentioned, and muscarinic cholinergic fibers from the medial septum to hippocampal CA1 positively modulate fear extinction. Importantly, fear extinction can be made state-dependent on circulating epinephrine, which may play a role in situations of stress. Exposure to a novel experience can strongly enhance the consolidation of fear extinction through a synaptic tagging and capture mechanism; this may be useful in the therapy of states caused by fear memory like PTSD.

Avoidance behavior: a free-operant lever-press avoidance task for the assessment of the effects of safety signals

This protocol details a free-operant avoidance paradigm that has been developed to evaluate the relative contribution of different sources of reinforcement of avoidance behavior that may play an important role in the development and maintenance of human anxiety disorders. The task enables the assessment of the effects of safety cues that signal a period free from danger on lever-press avoidance behavior. Avoidance behavior trained using this protocol has been shown to be sensitive to both behavioral and pharmacological manipulations and has been optimized so that it takes approximately 1 month for rats to perform at high levels of stable avoidance responding.

Learning from the spinal cord: how the study of spinal cord plasticity informs our view of learning

The paper reviews research examining whether and how training can induce a lasting change in spinal cord function. A framework for the study of learning, and some essential issues in experimental design, are discussed. A core element involves delayed assessment under common conditions. Research has shown that brain systems can induce a lasting (memory-like) alteration in spinal function. Neurons within the lower (lumbosacral) spinal cord can also adapt when isolated from the brain by means of a thoracic transection. Using traditional learning paradigms, evidence suggests that spinal neurons support habituation and sensitization as well as Pavlovian and instrumental conditioning. At a neurobiological level, spinal systems support phenomena (e.g., long-term potentiation), and involve mechanisms (e.g., NMDA mediated plasticity, protein synthesis) implicated in brain-dependent learning and memory. Spinal learning also induces modulatory effects that alter the capacity for learning. Uncontrollable/unpredictable stimulation disables the capacity for instrumental learning and this effect has been linked to the cytokine tumor necrosis factor (TNF). Predictable/controllable stimulation enables learning and counters the adverse effects of uncontrollable stimulation through a process that depends upon brain-derived neurotrophic factor (BDNF). Finally, uncontrollable, but not controllable, nociceptive stimulation impairs recovery after a contusion injury. A process-oriented approach (neurofunctionalism) is outlined that encourages a broader view of learning phenomena.

Invertebrate learning and cognition: relating phenomena to neural substrate

Diverse invertebrate species have been used for studies of learning and comparative cognition. Although we have gained invaluable information from this, in this study we argue that our approach to comparative learning research is rather deficient. Generally invertebrate learning research has focused mainly on arthropods, and most of that within the Hymenoptera and Diptera. Any true comparative analysis of the distribution of comparative cognitive abilities across phyla is hampered by this bias, and more fundamentally by a reporting bias toward positive results. To understand the limits of learning and cognition for a species, knowing what animals cannot do is at least as important as reporting what they can. Finally, much more effort needs to be focused on the neurobiological analysis of different types of learning to truly understand the differences and similarities of learning types. In this review, we first give a brief overview of the various forms of learning in invertebrates. We also suggest areas where further study is needed for a more comparative understanding of learning. Finally, using what is known of learning in honeybees and the well-studied honeybee brain, we present a model of how various complex forms of learning may be accounted for with the same neural circuitry required for so-called simple learning types. At the neurobiological level, different learning phenomena are unlikely to be independent, and without considering this it is very difficult to correctly interpret the phylogenetic distribution of learning and cognitive abilities. WIREs Cogn Sci 2013, 4:561-582. doi: 10.1002/wcs.1248 For further resources related to this article, please visit the WIREs website.

A study in the founding of applied behavior analysis through its publications

This article reports a study of the founding of applied behavior analysis through its publications. Our methods included hand searches of sources (e.g., journals, reference lists), search terms (i.e., early, applied, behavioral, research, literature), inclusion criteria (e.g., the field's applied dimension), and (d) challenges to their face and content validity. Our results were 36 articles published between 1959 and 1967 that we organized into 4 groups: 12 in 3 programs of research and 24 others. Our discussion addresses (a) limitations in our method (e.g., the completeness of our search), (b) challenges to the validity of our methods and results (e.g., convergent validity), and (c) priority claims about the field's founding. We conclude that the claims are irresolvable because identification of the founding publications depends significantly on methods and because the field's founding was an evolutionary process. We close with suggestions for future research.

The response-stimulus contingency and reinforcement learning as a context for considering two non-behavior-analytic views of contingency learning

This paper introduces a special section on the contingency. Bower and Watson were invited to present their views of contingency learning in human infants from outside the context of behavior analysis, and Cigales, Marr, and Lattal and Shahan provided commentaries that point out some of the more interesting and controversial aspects of those views from a behavior-analytic perspective. The debate turns on how to conceptualize the response-stimulus contingency of operant learning. The present paper introduces the contingency concept and contingency detection by subjects, as well as research practices in behavior analysis, in a context in which the dependency between infant responding and the presentation of environmental consequences may be disrupted through procedures in which ordinarily consequent events occur before the response or in its absence. These points can relate to and serve as an introduction to the Bower and Watson papers on infant contingency learning as well as to the three commentaries that follow.

Contingency and behavior analysis

The concept of contingency is central to theoretical discussions of learned behavior and in the application of learning research to problems of social significance. This paper reviews three aspects of the contingency concept as it has been developed by behavior analysts. The first is the empirical analysis of contingency through experimental studies of both human and nonhuman behavior. The second is the synthesis of experimental studies in theoretical and conceptual frameworks to yield a more general account of contingency and to integrate the concept with other behavioral processes. The third aspect is one of practical considerations in the application of the contingency concept in both laboratory and applied settings.

A cocaine cue is more preferred and evokes more frequency-modulated 50-kHz ultrasonic vocalizations in rats prone to attribute incentive salience to a food cue

RATIONALE

Individuals vary considerably in the extent to which they attribute incentive salience to food-associated cues.

OBJECTIVES

We asked whether individuals prone to attribute incentive salience to a food cue are also prone to attribute incentive properties to a stimulus associated with a drug of abuse-cocaine.

METHODS

We first identified those rats that attributed incentive salience to a food cue by quantifying the extent to which they came to approach and engage a food cue. We then used a conditioned place preference procedure to pair an injection of 10 mg/kg cocaine (i.p.) with one distinct floor texture (grid or holes) and saline with another. Following 8 days of conditioning, each rat was given a saline injection and placed into a chamber that had both floors present. We measured the time spent on each floor, and also 50-kHz ultrasonic vocalizations, which have been associated with positive affective states.

RESULTS

Rats that vigorously engaged the food cue ("sign trackers") expressed a preference for the cocaine-paired floor compared to those that did not ("goal trackers"). In addition, sign trackers made substantially more frequency-modulated 50-kHz vocalizations when injected with cocaine and when later exposed to the cocaine cue.

CONCLUSIONS

Rats prone to attribute incentive salience to a food cue are also prone to attribute incentive motivational properties to a tactile cue associated with cocaine. We suggest that individuals prone to attribute incentive salience to reward cues will have difficulty resisting them and, therefore, may be especially vulnerable to develop impulse control disorders, including addiction.

The biology of psychology: 'Simple' conditioning?

Operant (instrumental) and classical (Pavlovian) conditioning are taught as the simplest forms of associative learning. Recent research in several invertebrate model systems has now accumulated evidence that the dichotomy is not as simple as it seemed. During operant learning in the fruit fly Drosophila, at least two genetically distinct learning systems interact dynamically. Inspired by analogous results in three other research fields, we propose to term one of these systems world-learning (assigning value to sensory stimuli) and the other self-learning (assigning value to a specific action or movement). During the goal-directed phase of operant learning, world-learning inhibits self-learning (in Drosophila via the mushroom-body neuropil), to allow for flexible generalization. Extended training overcomes this inhibition in a phase transition akin to habit formation in vertebrates, allowing self-learning to transform spontaneous actions to habitual responses. In part, these insights were achieved by reducing operant experiments beyond the traditional set-ups (i.e., 'pure' operant learning) and using modern, molecular and/or genetic model systems.

The operant-respondent distinction: Future directions

The operant-respondent distinction has provided a major organizing framework for the data generated through the experimental analysis of behavior. Problems have been encountered, however, in using it as an explanatory concept for such phenomena as avoidance and conditioned suppression. Data now exist that do not fit neatly into the framework. Moreover, the discovery of autoshaping has highlighted difficulties in isolating the two types of behavior and conditioning. Despite these problems, the operant-respondent framework remains the most successful paradigm currently available for organizing behavioral data. Research and theoretical efforts should therefore probably be directed to modifying the framework to account for disparate data.

Classical conditioning: The new hegemony

Converging data from different disciplines are showing the role of classical conditioning processes in the elaboration of human and animal behavior to be larger than previously supposed. Restricted views of classically conditioned responses as merely secretory, reflexive, or emotional are giving way to a broader conception that includes problem-solving, and other rule-governed behavior thought to be the exclusive province of either operant conditiońing or cognitive psychology. These new views have been accompanied by changes in the way conditioning is conducted and evaluated. Data from a number of seemingly unrelated phenomena such as relapse to drug abuse by postaddicts, the placebo effect, and the immune response appear to involve classical conditioning processes. Classical conditioning, moreover, has been found to occur in simpler and simpler organisms and recently even demonstrated in brain slices and in utero. This target article will integrate the several research areas that have used the classical conditioning process as an explanatory model; it will challenge teleological interpretations of the classically conditioned CR and offer some basic principles for testing conditioning in diverse areas.

The integrative function of the basal ganglia in instrumental conditioning

Recent research in instrumental conditioning has focused on the striatum, particularly the role of the dorsal striatum in the learning processes that contribute to instrumental performance in rats. This research has found evidence of what appear to be parallel, functionally and anatomically distinct circuits involving dorsomedial striatum (DMS) and dorsolateral striatum (DLS) that contribute to two independent instrumental learning processes. Evidence suggests that the formation of the critical action-outcome associations mediating goal-directed action are localized to the dorsomedial striatum, whereas the sensorimotor connections that control the performance of habitual actions are localized to the dorsolateral striatum. In addition to the dorsal striatum, these learning processes appear to engage distinct cortico-striatal networks and to be embedded in a complex of converging and partially segregated loops that constitute the cortico-striatal thalamo-cortical feedback circuit. As the entry point for the basal ganglia, cortical circuits involving the dorsal striatum are clearly in a position to control a variety of motor functions but, as recent studies of various neurodegenerative disorders have made clear, they are also involved in a number of cognitive and executive functions including action selection, planning, and decision-making.

Determinants of imitation of hand-to-body gestures in 2- and 3-year-old children

Twenty children, ten 2-year-olds and ten 3-year-olds, participated in an AB procedure. In the baseline phase, each child was trained the same four matching relations to criterion under intermittent reinforcement. During the subsequent imitation test, the experimenter modeled a total of 20 target gestures (six trials each) interspersed with intermittently reinforced baseline trials. In each session, target gestures were selected in a pre-randomized sequence from: Set 1--ear touches; Set 2--shoulder touches; Set 3--midarm touches; and Set 4--wrist touches; subjects' responses to targets were not reinforced. In each target set, half the gestures featured in nursery matching games and were termed common targets whereas the remainder, which were topographically similar but did not feature in the games, served as uncommon targets. The children produced significantly more matching responses to common target models than to uncommon ones. Common responses were also produced as mismatches to uncommon target models more often than vice versa. Response accuracy did not improve over trials, suggesting that "parity" did not serve as a conditioned reinforcer. All children showed a strong bias for "mirroring"--responding in the same hemispace as the modeler. The 2-year-olds produced more matching errors than the 3-year-olds and most children showed a bias for responding with their right hands. The strong effects of training environment (nursery matching games) are consistent with a Skinnerian account, but not a cognitive goal theory account, of imitation in young children.

Autoshaping and automaintenance: a neural-network approach

This article presents an interpretation of autoshaping, and positive and negative automaintenance, based on a neural-network model. The model makes no distinction between operant and respondent learning mechanisms, and takes into account knowledge of hippocampal and dopaminergic systems. Four simulations were run, each one using an A-B-A design and four instances of feedfoward architectures. In A, networks received a positive contingency between inputs that simulated a conditioned stimulus (CS) and an input that simulated an unconditioned stimulus (US). Responding was simulated as an output activation that was neither elicited by nor required for the US. B was an omission-training procedure. Response directedness was defined as sensory feedback from responding, simulated as a dependence of other inputs on responding. In Simulation 1, the phenomena were simulated with a fully connected architecture and maximally intense response feedback. The other simulations used a partially connected architecture without competition between CS and response feedback. In Simulation 2, a maximally intense feedback resulted in substantial autoshaping and automaintenance. In Simulation 3, eliminating response feedback interfered substantially with autoshaping and automaintenance. In Simulation 4, intermediate autoshaping and automaintenance resulted from an intermediate response feedback. Implications for the operant-respondent distinction and the behavior-neuroscience relation are discussed.

Instrumental learning within the spinal cord: underlying mechanisms and implications for recovery after injury

Using spinally transected rats, research has shown that neurons within the L4-S2 spinal cord are sensitive to response-outcome (instrumental) relations. This learning depends on a form of N-methyl-D-aspartate (NMDA)-mediated plasticity. Instrumental training enables subsequent learning, and this effect has been linked to the expression of brain-derived neurotrophic factor. Rats given uncontrollable stimulation later exhibit impaired instrumental learning, and this deficit lasts up to 48 hr. The induction of the deficit can be blocked by prior training with controllable shock, the concurrent presentation of a tonic stimulus that induces antinociception, or pretreatment with an NMDA or gamma-aminobutyric acid-A antagonist. The expression of the deficit depends on a kappa opioid. Uncontrollable stimulation enhances mechanical reactivity (allodynia), and treatments that induce allodynia (e.g., inflammation) inhibit learning. In intact animals, descending serotonergic neurons exert a protective effect that blocks the adverse consequences of uncontrollable stimulation. Uncontrollable, but not controllable, stimulation impairs the recovery of function after a contusion injury.

A day of great illumination: B. F. Skinner's discovery of shaping

Despite the seminal studies of response differentiation by the method of successive approximation detailed in chapter 8 of The Behavior of Organisms (1938), B. F. Skinner never actually shaped an operant response by hand until a memorable incident of startling serendipity on the top floor of a flour mill in Minneapolis in 1943. That occasion appears to have been a genuine eureka experience for Skinner, causing him to appreciate as never before the significance of reinforcement mediated by biological connections with the animate social environment, as opposed to purely mechanical connections with the inanimate physical environment. This insight stimulated him to coin a new term (shaping), and also led directly to a shift in his perspective on verbal behavior from an emphasis on antecedents and molecular topographical details to an emphasis on consequences and more molar, functional properties in which the social dyad inherent to the shaping process became the definitive property of verbal behavior. Moreover, the insight seems to have emboldened Skinner to explore the greater implications of his behaviorism for human behavior writ large, an enterprise that characterized the bulk of his post-World War II scholarship.

On the origin and preservation of cumulative record in its struggle for life as a favored term

This paper offers a case study of the origins, emergence, and evolution of the term cumulative record as the name for the means by which B. F. Skinner brought his behavior under the control of his subject matter. Our methods included on-line searches, reviews of Skinner's publications, and journal codings and counts. The results reveal that the term is not originally attributable to Skinner, but emerged earlier in ordinary language and in another discipline--education. It was not even original to Skinner in print in his own science. Still, the term was once original to him, which we address with additional analyses of his having originated and advanced it. We conclude with a discussion the constraints of our methods, suggestions for future research, and the variable appreciation of technology and terminology in science studies.

The classical origins of Pavlov’s conditioning

This article presents a brief description of the scientific discovery of classical conditioning both in the United States and in Russia. The incorporation of classical conditioning as a scientific method in the United States is described. Particular attention is given to how and why the terminologies used to identify the components of classical conditioning were modified over the years. I then trace the curious evolution of the terminology associated with Pavlov's form of conditioning, from its introduction to the United States as "the Pawlow salivary reflex method" to its present appellation as classical conditioning. Finally I conclude by developing a theory as to when and why the term classical conditioning was adopted.

Extending in vitro conditioning in Aplysia to analyze operant and classical processes in the same preparation

Operant and classical conditioning are major processes shaping behavioral responses in all animals. Although the understanding of the mechanisms of classical conditioning has expanded significantly, the understanding of the mechanisms of operant conditioning is more limited. Recent developments in Aplysia are helping to narrow the gap in the level of understanding between operant and classical conditioning, and have raised the possibility of studying the neuronal processes underlying the interaction of operant and classical components in a relatively complex learning task. In the present study, we describe a first step toward realizing this goal, by developing a single in vitro preparation in which both operant and classical conditioning can be studied concurrently. The new paradigm reproduced previously published results, even under more conservative and homogenous selection criteria and tonic stimulation regime. Moreover, the observed learning was resistant to delay, shortening, and signaling of reinforcement.

The Impact of Pavlov on the Psychology of Learning in English-Speaking Countries

The translation of Pavlov's lectures (Pavlov, 1927) provided English-speaking psychologists with access to the full scope of Pavlov's research and theoretical ideas. The impact this had on their study of the psychology of learning can be assessed by examining influential books in this area. This reveals that Watson (1924) had been highly effective in promoting the misleading idea that Pavlov was a fellow S-R theorist. This assumption was not questioned by Tolman (1932), Hilgard and Marquis (1940) or by Hull (1943). However, this mistake was not made by Skinner (1938), who also provided the strongest arguments against Pavlov's belief that behavioral effects required explanation in terms of physiological processes. Post-1927 most learning research in the English-speaking countries continued to use instrumental, rather than Pavlovian, conditioning procedures. Nevertheless, many of the issues addressed by this research were ones that Pavlov had been the first to raise, so that his major influence can be seen as that of defining a research program for subsequent students of learning.

Instrumental learning within the spinal cord: V. Evidence the behavioral deficit observed after noncontingent nociceptive stimulation reflects an intraspinal modification

Spinally transected rats given leg shock whenever one hindlimb is extended learn to maintain the leg in a flexed position, which minimizes net shock exposure. Yoked rats, that receive an equal amount of shock independent of leg position (noncontingent shock), do not exhibit an increase in flexion duration. Yoked rats also fail to learn when response contingent shock is applied to the previously shocked leg, a behavioral deficit that resembles learned helplessness. This deficit could reflect either a peripheral (e.g. muscle fatigue) or central effect. Experiment 1 showed that spinalized rats given noncontingent shock to one hind limb fail to learn when response-contingent shock is applied to the contralateral leg. Experiment 2 demonstrated that blocking the afferent input to the spinal cord, by cutting the sciatic nerve, blocked the development of the deficit. Experiment 3 found that intrathecal lidocaine has a protective effect and prevents the deficit. These findings suggest that noncontingent nociceptive stimulation induces an intraspinal modification that undermines behavioral potential.

Instrumental learning within the spinal cord: IV. Induction and retention of the behavioral deficit observed after noncontingent shock

Spinalized rats given shock whenever 1 hind leg is extended learn to maintain that leg in a flexed position, a simple form of instrumental learning. Rats given shock independent of leg position do not exhibit an increase in flexion duration. Experiment 1 showed that 6 min of intermittent legshock can produce this deficit. Intermittent tailshock undermines learning (Experiments 2-3), and this effect lasts at least 2 days (Experiment 4). Exposure to continuous shock did not induce a deficit (Experiment 5) but did induce antinociception (Experiment 6). Intermittent shock did not induce antinociception (Experiment 6). Experiment 7 addressed an alternative interpretation of the results, and Experiment 8 showed that presenting a continuous tailshock while intermittent legshock is applied can prevent the deficit.

A evolução do conceito de operante

O conceito de operante surge nos anos 30 como necessidade, dada a dificuldade encontrada por Skinner para analisar seus dados usando como ferramenta o conceito de reflexo. O presente trabalho apresenta a evolução do conceito até o presente, quando a unidade de análise não tem mais a ver com a estrutura do comportamento: a nova ferramenta é o conceito de contingência tríplice.

The operant and the classical in conditioned orientation of Drosophila melanogaster at the flight simulator

Ever since learning and memory have been studied experimentally, the relationship between operant and classical conditioning has been controversial. Operant conditioning is any form of conditioning that essentially depends on the animal's behavior. It relies on operant behavior. A motor output is called operant if it controls a sensory variable. The Drosophila flight simulator, in which the relevant behavior is a single motor variable (yaw torque), fully separates the operant and classical components of a complex conditioning task. In this paradigm a tethered fly learns, operantly or classically, to prefer and avoid certain flight orientations in relation to the surrounding panorama. Yaw torque is recorded and, in the operant mode, controls the panorama. Using a yoked control, we show that classical pattern learning necessitates more extensive training than operant pattern learning. We compare in detail the microstructure of yaw torque after classical and operant training but find no evidence for acquired behavioral traits after operant conditioning that might explain this difference. We therefore conclude that the operant behavior has a facilitating effect on the classical training. In addition, we show that an operantly learned stimulus is successfully transferred from the behavior of the training to a different behavior. This result unequivocally demonstrates that during operant conditioning classical associations can be formed.

The operant control of vocalization in the dog

Control over the vocal responses of three dogs was established using operant-conditioning procedures. Several points of interest were observed in the data. First, fixed-ratio schedules of reinforcement generated a vocal response topography which was similar in detail to that of a "motor" bar-nosing response. Second, vocal responding was brought under the control of external visual stimuli as a result of differential reinforcement. Third, good stimulus control was maintained on a multiple schedule containing a vocal-response component and a bar-response component. Fourth, the stimulus control on the multiple schedule transferred with minimal disruption to a chain schedule requiring a sequence of 10 bar responses followed by 10 vocal responses. Fifth, because vocal and bar responses are not mutually exclusive, concurrent responding tended to develop on the chain schedule. These results were discussed with reference to the advisability of applying the terms operant and respondent to unconditioned behavior, and, particularly, to unconditioned verbal behavior.

Application of Pavlovian higher-order conditioning to the analysis of the neural substrates of fear conditioning

In Pavlovian first-order conditioning, a conditioned response is acquired by pairing a neutral stimulus (S1) with a stimulus that has innate motivational value. In higher-order conditioning, a neutral stimulus (S2) is paired with S1 either after (second-order conditioning) or before (sensory preconditioning) first-order conditioning has been acquired. Thus, in higher-order conditioning the motivational value of the reinforcer is acquired rather than innate. This review describes some of the potential uses of higher-order conditioning in investigating the neural substrates of fearful memories. First, because in second-order fear conditioning S2 is not paired directly with a painful stimulus, any effect of a treatment on the acquisition of fear cannot be attributed to the treatment's possible effects on transmission of nociceptive information. Second, higher-order conditioning provides opportunities for analyzing where and how different types of events, or different aspects of the same events, are represented in the brain.

Avoidance and classical conditioning of leg flexion in dogs

This study of leg flexion conditioning in dogs, which were trained under both avoidance and classical contingencies consecutively, confirmed and extended the results from the Wahlsten and Cole study (In Classical Conditioning II: Current Research and Theory, Appleton Century-Crofts, New York, 1972). Briefly, dogs were trained to asymptotic behavioral levels under either avoidance or classical contingencies with a CS-US interval of either three (3) or five (5) seconds where the unconditioned stimulus (US) was shock to the foreleg and the conditioned stimulus (CS) was 1000 Hz tone. The dogs were then switched to the other contingency (without any modification in the stimulus situation other than the shock contingency) and trained to asymptotic behavioral levels. The CS remained on for the entire CS-US period and terminated with the end of the scheduled interval. Under the classical contingency, the US occurred as scheduled on every trial regardless of the dog's behavior. Under the avoidance contingency, the US was prevented from occurring if the subject responded with a criterion leg lift during the CS-US interval. The only feedback to the dog of a successful performance was the leg lift itself. The results indicated that there were two different conditioned responses produced, one just after CS onset under the avoidance contingency, and one just before US onset under the classical contingency for both CS-US intervals. The findings were interpreted as supporting a single-factor informational view of learning and a neural model was presented.