Feedforward versus feedbackward: An ethological alternative to the law of effect

Contingency management and cognitive behavior therapy for smoking cessation among veterans with posttraumatic stress disorder: Design and methodology of a randomized clinical trial

BACKGROUND

Smoking is a prevalent concern among Veterans, especially those with posttraumatic stress disorder (PTSD). Despite the availability of interventions for smoking cessation, these treatments have not been as effective among Veteran populations, particularly Veterans with PTSD. The present study seeks to describe the methods of a randomized clinical trial examining the efficacy of CPT-SMART, a multidimensional treatment combining cognitive processing therapy (CPT) for PTSD, smoking cessation counseling, pharmacotherapy, and contingency management (CM) compared to a yoked comparison group.

METHODS

One hundred twenty Veterans with PTSD who smoke cigarettes will be enrolled. All participants will receive CPT in addition to counseling and pharmacotherapy for smoking cessation. Participants will be randomized to the CPT-SMART condition, which includes monetary reinforcement that is contingent on bioverification of smoking abstinence (i.e., contingency management), or a yoked comparison with monetary reinforcement matched to the participant to whom they are yoked. The primary outcome is bioverified smoking abstinence at the 6-month follow-up appointment.

CONCLUSION

If shown efficacious, a combined PTSD and smoking treatment plus incentive-based approach for smoking could be implemented into specialty PTSD programs. The positive public health impact of reducing smoking among Veterans with PTSD could be enormous as it would prevent significant smoking-related morbidity and mortality.

The Development of the Three Rs Concept

In 1955–57, in the course of a study initiated by Charles Hume and the Universities Federation for Animal Welfare, Rex Burch and I conceived of the Three Rs of humane experimental technique — replacement, reduction and refinement. Except for the foundation, in 1969, and the early work, of FRAME, the next two decades saw little development of the Three Rs, and widespread scepticism about the possibilities of replacement, especially in activity and toxicity testing. In the 1980s and early 1990s, advances in tissue culture technique led to spectacular progress in fundamental pharmacology and toxicology. There have also been important recent advances in aspects of refinement discussed in our book published in 1959, notably in anaesthesia and analgesia, and in the application of ethology. However, there has been less progress in reduction, except for one ingenious new statistical technique for reducing numbers of animals used for testing vaccines. Developments we scarcely touched on in 1959 include the application of the Three Rs to the production and testing of vaccines, the education of experimenters in laboratory animal science (both of these developments pioneered in The Netherlands), and the replacement of animals in teaching by the use of computers and audiovisual displays. A priority for the future is the application of the Three Rs to work with primates. New uses for experimental animals continue to arise, but, after recent successes, we can now have considerable confidence in the progress of humane experimental technique.

Rethinking reinforcement: allocation, induction, and contingency

The concept of reinforcement is at least incomplete and almost certainly incorrect. An alternative way of organizing our understanding of behavior may be built around three concepts: allocation, induction, and correlation. Allocation is the measure of behavior and captures the centrality of choice: All behavior entails choice and consists of choice. Allocation changes as a result of induction and correlation. The term induction covers phenomena such as adjunctive, interim, and terminal behavior-behavior induced in a situation by occurrence of food or another Phylogenetically Important Event (PIE) in that situation. Induction resembles stimulus control in that no one-to-one relation exists between induced behavior and the inducing event. If one allowed that some stimulus control were the result of phylogeny, then induction and stimulus control would be identical, and a PIE would resemble a discriminative stimulus. Much evidence supports the idea that a PIE induces all PIE-related activities. Research also supports the idea that stimuli correlated with PIEs become PIE-related conditional inducers. Contingencies create correlations between "operant" activity (e.g., lever pressing) and PIEs (e.g., food). Once an activity has become PIE-related, the PIE induces it along with other PIE-related activities. Contingencies also constrain possible performances. These constraints specify feedback functions, which explain phenomena such as the higher response rates on ratio schedules in comparison with interval schedules. Allocations that include a lot of operant activity are "selected" only in the sense that they generate more frequent occurrence of the PIE within the constraints of the situation; contingency and induction do the "selecting."

Brain organization and the roots of anticipation in Drosophila olfactory conditioning

Defining learning at the molecular and physiological level has been one of the greatest challenges in biology. Recent research suggests that by studying fruit fly (Drosophila melanogaster) brain organization we can now begin to unravel some of these mysteries. The fruit fly brain is organized into executive centers that regulate anatomically separate behavioral systems. The mushroom body is an example of an executive center which is modified by olfactory conditioning. During this simple form of learning, an odor is paired with either food or shock. Either experience alters distinguishable specific circuitry within the mushroom body. Results suggest that after conditioning an odor to food, the mushroom body will activate a feeding system via a subset of its circuitry. After conditioning an odor to shock, the mushroom body will instead activate an avoidance system with other subsets of mushroom body neurons. The results of these experiments demonstrate a mechanism for flies to display anticipation of their environment after olfactory conditioning has occurred. However, these results fail to provide evidence for reinforcement, a consequence of action, as part of this mechanism. Instead, specific subsets of dopaminergic and octopaminergic neurons provide a simple pairing signal, in contrast to a reinforcement signal, which allows for prediction of the environment after experience. This view has implications for models of conditioning.

An unexpected reduction in sucrose concentration activates the HPA axis on successive post shift days without attenuation by discriminative contextual stimuli

Previous studies have shown that the successive negative contrast procedure, in which food-restricted rats entrained to once daily, brief presentations of 32% sucrose are unexpectedly shifted to a 4% solution, results in an adrenocortical response on the second, but not the first postshift day. We attempted to generalize that finding in our own procedure. In Experiment 1, two groups of rats were given a 32% sucrose solution once daily in their home cages for 14 days before being shifted to a 4% solution. One group was killed 10 min after the first 4% solution and one was killed after the second 4% solution. In addition, two groups receiving either 32% or 4% sucrose throughout the experiment served as unshifted controls. In contrast to previous findings, both shifted groups exhibited prominent adrenocorticotropin hormone (ACTH) and adrenocortical (B) responses on both postshift days compared to unshifted controls, which did not differ from one another. Experiment 2a employed distinctive contexts to test if the lack of generality of the delayed HPA axis response was due to suppressive effects of S(+) on the first postshift day. Rats were given once daily 32% sucrose in S(+) and equal exposure time in S(-). Half of these rats were shifted to 4% sucrose in S(+) and half were shifted in S(-). These two groups were compared to home cage controls. Half of each group was killed after their first 4% sucrose, and half after the second 4% sucrose. All rats showed ACTH and B responses comparable to shifted rats in Experiment 1. S(+) failed to suppress the HPA axis, and the stress response was higher on the first compared to the second day of the shift. Experiment 2b established that distinctive contexts predicting sucrose, S(+), or not predicting sucrose, S(-), controlled behavioral choice and contextual discrimination. Thus, there was no evidence that issues of stimulus control could explain the lack of generality of previous findings. The data indicate that thwarting sucrose expectancies is stressful, and that this stress response habituates across days.

From Malthus to motive: how the HPA axis engineers the phenotype, yoking needs to wants

The hypothalamo-pituitary-adrenal (HPA) axis is the critical mediator of the vertebrate stress response system, responding to environmental stressors by maintaining internal homeostasis and coupling the needs of the body to the wants of the mind. The HPA axis has numerous complex drivers and highly flexible operating characterisitics. Major drivers include two circadian drivers, two extra-hypothalamic networks controlling top-down (psychogenic) and bottom-up (systemic) threats, and two intra-hypothalamic networks coordinating behavioral, autonomic, and neuroendocrine outflows. These various networks jointly and flexibly control HPA axis output of periodic (oscillatory) functions and a range of adventitious systemic or psychological threats, including predictable daily cycles of energy flow, actual metabolic deficits over many time scales, predicted metabolic deficits, and the state-dependent management of post-prandial responses to feeding. Evidence is provided that reparation of metabolic derangement by either food or glucocorticoids results in a metabolic signal that inhibits HPA activity. In short, the HPA axis is intimately involved in managing and remodeling peripheral energy fluxes, which appear to provide an unidentified metabolic inhibitory feedback signal to the HPA axis via glucocorticoids. In a complementary and perhaps a less appreciated role, adrenocortical hormones also act on brain to provide not only feedback, but feedforward control over the HPA axis itself and its various drivers, as well as coordinating behavioral and autonomic outflows, and mounting central incentive and memorial networks that are adaptive in both appetitive and aversive motivational modes. By centrally remodeling the phenotype, the HPA axis provides ballistic and predictive control over motor outflows relevant to the type of stressor. Evidence is examined concerning the global hypothesis that the HPA axis comprehensively induces integrative phenotypic plasticity, thus remodeling the body and its governor, the brain, to yoke the needs of the body to the wants of the mind. Adverse side effects of this yoking under conditions of glucocorticoid excess are discussed.

The role of the feedforward paradigm in cognitive psychology

Feedforward control is a process adjusting behaviour in a continuative way. Feedforward takes place when an equilibrium state is disrupted and the system has to automatically retrieve the homeostatic stable state. It also occurs when a perturbation is previewed and must be eliminated in order to achieve a desired goal. According to the most general definition, a feedforward process operates by fixing the future representation of the desired state, the achieving of which stops the process. Then, feedforward works by means of the refinement determined by successive comparisons between the actual and target products. In its applications, a feedforward process is thought to be modulated by the subject's purpose and the environmental state. Over the years, the feedforward process has assumed different connotations in several contests of cognitive psychology. An overview of the research fields in psychology that significantly progressed with the introduction of a feedforward paradigm is provided by: (a) reviewing models in which the feedforward concept plays a fundamental role in the system control; (b) examining critical experiments related to the interaction of feedforward and feedback processes; (c) evidencing practical applications for some of the presented feedforward-based architectures.

Recent use of signs by chimpanzees (Pan troglodytes) in interactions with humans

In light of the controversy about the linguistic properties of chimpanzee signing behavior, the recent sign use of 5 chimpanzees (Pan troglodytes) with long histories of sign use was analyzed while they interacted with longtime human companions. Four corpora from 1992 to 1999 consisting of 3,448 sign utterances were examined. The chimpanzees predominantly used object and action signs. There was no evidence for semantic or syntactic structure in combinations of signs. Longer combinations showed repetition and stringing of object and action signs. The chimpanzees mostly signed with an acquisitive motivation. Requests for objects and actions were the predominant communicative intentions of the sign utterances, though naming and answering also occurred. This recent sign use shows multiple differences with (early) human language.

Function transfer in human operant experiments: the role of stimulus pairings

Although function transfer often has been studied in complex operant procedures (such as matching to sample), whether operant reinforcement actually produces function transfer in such settings has not been established. The present experiments, with high school students as subjects, suggest that stimulus pairings can promote function transfer in conditions that closely approximate those of matching to sample. In Experiment 1, the subjects showed transfer of operant responding from three geometric figures (C1, C2, C3) to three colored shapes (B1, B2, B3) when the latter were paired with the former. Experiment 2 involved two groups of subjects. In the matching group, subjects matched the colored shapes with the geometric figures; in the yoked group, the shapes were merely paired with the geometric figures, and the schedule of stimulus pairing was yoked to the performance of the subjects in the matching group. Both groups of subjects showed function transfer. Experiment 3 documented function transfer from C stimuli to B stimuli through indirect stimulus pairings (A-B, A-C). In Experiment 4, function transfer was obtained even though the subjects vocalized continuously during the pairing trials, presumably preventing covert verbalization that might mediate transfer effects. Our results are consistent with a Pavlovian account and raise difficulties for current operant theories of function transfer.

The cerebellum in action: a simulation and robotics study

The control or prediction of the precise timing of events are central aspects of the many tasks assigned to the cerebellum. Despite much detailed knowledge of its physiology and anatomy, it remains unclear how the cerebellar circuitry can achieve such an adaptive timing function. We present a computational model pursuing this question for one extensively studied type of cerebellar-mediated learning: the classical conditioning of discrete motor responses. This model combines multiple current assumptions on the function of the cerebellar circuitry and was used to investigate whether plasticity in the cerebellar cortex alone can mediate adaptive conditioned response timing. In particular, we studied the effect of changes in the strength of the synapses formed between parallel fibres and Purkinje cells under the control of a negative feedback loop formed between inferior olive, cerebellar cortex and cerebellar deep nuclei. The learning performance of the model was evaluated at the circuit level in simulated conditioning experiments as well as at the behavioural level using a mobile robot. We demonstrate that the model supports adaptively timed responses under real-world conditions. Thus, in contrast to many other models that have focused on cerebellar-mediated conditioning, we investigated whether and how the suggested underlying mechanisms could give rise to behavioural phenomena.

Response acquisition with delayed reinforcement: a comparison of two-lever procedures

Groups of 8 experimentally naive rats were exposed during 8-hr sessions to resetting delay procedures in which responses on one lever (the reinforcement lever) produced water after a delay of 8, 16, 32, or 64 s. For rats in one condition, responses on a second (no-consequences) lever had no programmed consequences. For rats in another condition, responses on a second (cancellation) lever during a delay initiated by a response on the reinforcement lever prevented delivery of the scheduled reinforcer; responses on the cancellation lever at other times had no programmed consequences. Under both conditions and at all delays, most subjects emitted more responses on the reinforcement lever than did control rats that never received water emitted on either lever. At 8-s delays, both conditions engendered substantially more responding on the reinforcement lever than on the other lever, and performance closely resembled that of immediate-reinforcement controls. At delays of 16 and 32 s, however, there was clear differential responding on the two levers under the cancellation condition but not under the other condition. When the delay was 64 s, differential responding on the two levers did not occur consistently under either condition. These findings provide strong evidence that the behavior of rats is sensitive to consequences delayed by 8, 16, and 32 s, but only equivocal evidence of such sensitivity to consequences delayed 64 s. They also indicate that acquisition depends, in part, on the measure of performance used to index it.

Free food or earned food? A review and fuzzy model of contrafreeloading

Animals will work (e.g. lever press) for 'earned' food even though identical 'free' food can easily be obtained from a nearby dish. This phenomenon, called contrafreeloading, appears to contradict a basic tenet of most learning, motivation and optimal foraging theories; namely that animals strive to maximize the ratio of reward, or benefit, to effort, or cost. This paper reviews the factors that have been found to affect the level of contrafreeloading, to try to explain the behaviour. In experiments involving intensive training, contrafreeloading may be explained on the basis of secondary reinforcement and/or differential exposure to the alternative food sources. However, contrafreeloading also occurs without prior training. Contrafreeloading declines with increasing hunger and with increases in the effort required to obtain the earned food: it also has an inverted-U relationship with the degree of stimulus change associated with the earned food. A fuzzy logic model is developed to predict the outcome of interactions between these factors. The model successfully simulates previous empirical findings and provides novel, testable predictions. It is argued that contrafreeloading does not contradict reinforcement theory, provided that the sensory reinforcement obtained from stimuli associated with the earned food is also taken into account. A functional explanation of why such stimuli are reinforcing, and of contrafreeloading itself, is based upon the advantage of gathering information for animals living in changing environments (i.e. an information primacy model). Animals work for earned food in order to update their estimate of a currently sub-optimal food source because, in the longer term, it may unpredictably become the optimal place to feed. Contrafreeloading is therefore a behaviour that, under natural conditions, is adaptive.

Investigation of long-term recognition and association memory in unit responses from inferotemporal cortex

We investigated recognition and association memory in the responses of single units isolated in the inferior temporal cortex of a macaque while it performed a visual discrimination task. The unit responses showed significant recognition memory (a decreased response upon image repetition). Furthermore, a recognition memory appeared to be a permanent feature in these units. Such memory was evident in responses recorded at least 1 h after the most recent presentations of the more familiar images and may have been built up over the months of training. For these cells, the shorter-term recognition memory (seconds) and the longer-term recognition memory (hour plus) were significantly correlated (0.68). In these same cells associative memory was investigated with ten abstract images which had been randomly and permanently paired. The monkey had been taught to discriminate these five pairs from other similar pairs of images. Neither the spike count nor temporal response shape (as determined by a principal-components analysis) showed increased similarity for the images that had been paired. The cells that had both short-term and long-term recognition memory had responses to previously paired stimuli that were no more similar than expected by chance.

Behavior systems and reinforcement: an integrative approach

Most traditional conceptions of reinforcement are based on a simple causal model in which responding is strengthened by the presentation of a reinforcer. I argue that reinforcement is better viewed as the outcome of constraint of a functioning causal system comprised of multiple interrelated causal sequences, complex linkages between causes and effects, and a set of initial conditions. Using a simplified system conception of the reinforcement situation, I review the similarities and drawbacks of traditional reinforcement models and analyze the recent contributions of cognitive, regulatory, and ecological approaches. Finally, I show how the concept of behavior systems can begin to incorporate both traditional and recent conceptions of reinforcement in an integrative approach.

Role of prefeedings, plasma amino acid ratios and brain serotonin levels in carbohydrate and protein selection

The relationship between carbohydrate and protein consumption and brain serotonin levels was investigated using Sprague-Dawley rats trained to press two levers. Each response on one lever was followed by a carbohydrate-rich 45-mg food pellet and each response on another lever was followed by a 45-mg pellet relatively high in protein. Access to the carbohydrate pellets immediately prior to the daily session depressed the number of carbohydrate pellets consumed but had little effect on the number of protein pellets consumed. Prior access to standard rat chow, which is relatively high in protein, had the opposite effect. Prior access to carbohydrates in solution did not affect pellet consumption. Plasma amino acid levels and brain serotonin levels were altered by intraperitoneal injections of insulin (2.4 U/kg) and tryptophan (40 mg/kg), but consumption of carbohydrate pellets relative to protein was not changed by these injections. Saline and glucose injections did not change relative consumption either. The results emphasize the importance of oral-sensory cues in food intake and selection but are not supportive of a major role for serotonin.