Reinforced variability in animals and people: implications for adaptive action

Prolegomena to any future philosophy of behavior analysis as a science

This tribute to Howard Rachlin speculates about scholarly work that might have been. It explores how behavioral data might bear on philosophical issues, with examples that might be called case studies in experimental philosophy. In 1964, an issue of the Proceedings of the American Philosophical Society served a similar function. It was entitled "Psychology: A Behavioral Interpretation"; the papers included "Will," "Experience," "Appetite," "Humors," "Anxiety," and "Man." This presentation imagines what a contemporary project devoted to philosophical and behavior analytic perspectives on the topics of causation, freedom and volition, good and evil, time, words, and mind might have looked like. Along the way it notes how the project would have benefitted from Howard Rachlin's seminal contributions to both behavior analysis and philosophy. If ever such a project comes to pass, it will inevitably bear the stamp of his contributions.

Faustian bargains: Short-term and long-term contingencies in phylogeny, ontogeny, and sociogeny

Rachlin's interpretations of self-control depend on the short-term versus the long-term consequences of behavior. Sometimes these effects support each other (typing an abstract produces a written product now and is later read by others). Sometimes they conflict (procrastination now is incompatible with finishing the abstract by deadline). We usually reserve the language of self-control for human cases where long-term consequences are chosen over short-term ones. Rachlin made this distinction salient in ontogeny, but it also applies to selection in phylogeny (Darwinian evolution) and sociogeny (behavior passed from one organism to another). Our account examines relations between short-term and long-term consequences at each level of selection. For example, sexual selection has adaptive, short-term mating consequences but may drive species to extreme specializations that jeopardize long-term survival. In sociogeny, as in the Tragedy of the Commons, group members may get immediate economic benefits from exploiting resources but exhaust those resources over the long term. Whatever the level, when short-term and long-term consequences have opposing effects, adaptive behavior may depend on whether temporally extended contingencies exert more control than more immediate benefits.

Reinforced behavioral variability in the valproate rat model of autism spectrum disorder

Individuals diagnosed with autism spectrum disorder (ASD) tend to display restricted, repetitive behaviors and deficits in social interaction. Rats exposed to valproate (VPA) in utero have been shown to model symptoms of ASD. In previous research, VPA rats engaged in less social interaction and more repetitive responding than controls. The purpose of the present study was to further investigate behavioral variability in the VPA rat model of ASD by testing VPA and control rats in a reinforced-behavioral-variability operant task. In this procedure, rats emitted sequences of lever presses, some of which produced food. During baseline, food was delivered probabilistically, and variability was not required. Next, rats were exposed either to a variability contingency, in which food was only delivered following sequences that differed sufficiently from previous sequences (i.e., variability required), or to a yoked contingency, in which variability was not required. We hypothesized that VPA rats would behave less variably than controls in this task. However, VPA and control rats responded similarly variably when variability was required. Furthermore, VPA rats behaved slightly more variably than controls during baseline and yoked conditions, when variability was not required. These findings contribute to the complex literature surrounding the VPA rat model of ASD.

A Push For Examining Subjective Experience in Value-Based Decision-Making

Subjective experience is a powerful contributor to value-based decision-making. Not every decision is the same, nor made in isolation. Rather, decision-making relies on historical information and internal states for adaptive control. Hence, it is inherently continuous with respect to time - one decision or action evolves into the next. However, forays into the neurobiological underpinnings of decision-making have too frequently ignored the contribution of such continuous subjective experience, instead tying circuit activity and brain area involvement to discrete averaged behaviors and task parameters. While much information has been gained through these investigations, recent works have demonstrated the potential for a greater understanding of neural mechanisms when the continuous, experiential nature of behavior is integrated into the investigation. Such integration has important implications for disease states with disordered decision-making such as addiction, where subjective experience is a large contributor to the disorder.

Using a Lag Schedule of Reinforcement to Increase Response Variability in Children With Autism Spectrum Disorders

Lag reinforcement schedules have been shown in previous research to be an effective intervention for teaching verbal and nonverbal response variability to individuals with developmental disabilities. In more recent research, variability itself has been considered a reinforceable behavior in its own right (Susa & Schlinger, The Analysis of Verbal Behavior, 18, 125-130, 2012). Lag x schedules of reinforcement can be used to teach variability by using contingencies that require responses to differ from previous responses. The present study extended Susa and Schlinger's, The Analysis of Verbal Behavior, 18, 125-130, (2012) research by using 3 social questions instead of 1 in a random rotation and included probes to test for generality. A changing-criterion design was used to evaluate the results with one 11-year-old female participant diagnosed with autism. During baseline, the participant provided little variability, with rote responses. During the Lag 1 and Lag 2 phases, appropriate variable verbal responding increased with the use of echoic prompts, visual aids, and an error correction procedure. Further, the results also showed that the participant learned to vary her responses by demonstrating the ability to emit 11 novel prompted responses and 13 spontaneous responses. In addition, the participant was able to retain the skills learned in a maintenance probe conducted 4 weeks postintervention.

Tactical contingencies in the experimental analysis of reinforcement and operant classes

In his "Tactics of Scientific Research" (1960), his work on avoidance, his discovery of equivalence classes and his cautions on applications of coercion, Murray Sidman created high standards for behavior analytic research. I illustrate his influence in the context of three examples he might have characterized as pilot studies. Each examined trial N+1 response probabilities depending on whether trial N responding had produced a reinforcer. Differentially reinforced interresponse times, keys pecked in arbitrary matching, and two-key response sequences provided no robust evidence that reinforcing some response property on trial N raises the probability of responding with that property on trial N+1. These negative findings shed light on the nature of operant classes and on the relation of reinforcers to the responses that produced them. Through selection, reinforcers create operant classes and engender variations of the responses within those classes; operant classes are held together by common contingencies. Sidman extended our understanding of operant classes by expanding them to include equivalence relations.

An investigation of resurgence of reinforced behavioral variability in humans

The present study examined resurgence of reinforced variability in college students, who completed a 3-phase computer-based variability task. In the first phase, baseline, points were delivered for drawing rectangles that sufficiently differed from previous rectangles in terms of a target dimension (size or location, counterbalanced) but were sufficiently similar in terms of the alternative dimension. In the second phase, alternative, points were only delivered for rectangles that were sufficiently different in terms of the alternative dimension, but repetitive in terms of the target dimension. In the third phase, extinction, no points were delivered. In baseline, participants made rectangles that were highly varied in terms of the target dimension and less varied in terms of the alternative dimension, and vice versa in the alternative phase. During extinction, levels of variability increased for the target dimension, providing evidence for resurgence of reinforced variability of a specific dimension of behavior. However, levels of variability also remained high for the alternative dimension, indicating that extinction-induced response variability may also have impacted the results. Although future research is needed to explore other explanations, the results of this study replicate prior research with pigeons and provide some support for the notion of variability as an operant.

A Critical Review of the Support for Variability as an Operant Dimension

There is abundant evidence that behavioral variability is more predominant when reinforcement is contingent on it than when it is not, and the interpretation of direct reinforcement of variability suggested by Page and Neuringer, Journal of Experimental Psychology: Animal Behavior Processes, 11(3), 429-452 (1985) has been widely accepted. Even so, trying to identify the underlying mechanisms in the emergence of stochastic-like variability in a variability contingency is intricate. There are several challenges to characterizing variability as directly reinforced, most notably because reinforcement traditionally has been found to produce repetitive responding, but also because directly reinforced variability does not always relate to independent variables the same way as more commonly studied repetitive responding does. The challenging findings in variability experiments are discussed, along with alternative hypotheses on how variability contingencies may engender the high variability that they undeniably do. We suggest that the typical increase in behavioral variability that is often demonstrated when reinforcement is contingent on it may be better explained in terms of a dynamic interaction of reinforcement and extinction working on several specific responses rather than as directly reinforced.

Dugnad: A Fact and a Narrative of Norwegian Prosocial Behavior

Evolved mechanisms of phenotypic plasticity, which are evolutionary processes in their own right, enable species to respond adaptively to their environments. The Scandinavian countries, and Norway in particular, have for many years scored exceptionally high on lists of life quality, economic indicators, and measures of happiness. We propose that learning prosocial and cooperative behavior, which is central in a particular Norwegian cultural practice, dugnad, plays a role in the country's success story. Dugnad is a Norwegian term for a type of voluntary work carried out as a community or collective and traditionally involving a social gathering. Dugnad has a long history in Norway, and it is a well-established cultural practice that has led to and still maintains significant social benefits. Dugnad is arranged in virtually all communities such as kindergartens, neighborhoods, schools, and organizations. Participation in dugnad gatherings is generally expected. Children from a young age are involved in dugnad. Dugnad activities are based on cooperation and can include anything from arranging a spring cleaning in the local community to building a club house for your children's sports club. This paper discusses dugnad as a cultural practice that creates an environment that nurtures prosocial and cooperative activities. From a behavior analytic, selectionist perspective, we propose a non-domain-specific learning mechanism for dugnad-typical prosocial and cooperative behavior analogous to the phylogenetic evolutionary mechanism of group selection. Contingencies can lead to and maintain dugnad activities when extended behavioral patterns are selected as wholes.

The effect of learning on heart rate and behavior of European starlings (Sturnus vulgaris)

Wild-caught European starlings (Sturnus vulgaris) were exposed to a learning task to determine whether heart rate (HR) and behavior responses to the learning activated the sympathetic nervous system. Birds learned to discriminate between images of opposite convexity (concave and convex) based on shading cues in a closed economy (food only available through task completion). Once learned, the task was changed in three ways: (a) manipulating the angle and shape of the image; (b) altering the availability of the task; and (c) reversing the positive stimulus. HR, used as an index of catecholamine release, was measured during each change to determine whether having to alter previously established behaviors to learn new behaviors elicited a sympathetic response. Starlings decreased their HR during the initial discrimination training and did not alter their HR when presented with modified images or when the positive stimulus was reversed. However, HR increased when the task became unavailable and decreased upon its return, suggesting that preventing task performance was perceived as stressful. Birds also modified their behavior when tasks were changed. The number of trials per minute decreased during the reversal treatment, as did the success rate, suggesting that starlings may try to conserve energy when access to food diminishes. This is also supported by the decrease in perch hops per minute when the task was unavailable and the subsequent increase upon its return. Overall, these results suggest that learning per se does not activate the sympathetic nervous system and, therefore, is not a stressor for wild birds.

Empirical Evaluations of Skinner's Analysis of Problem Solving

We reviewed 12 studies in which the researcher taught problem-solving strategies, such as self-questioning and visual imagining, to children and adolescents with and without disabilities to facilitate the learning of math, spelling, play/social, and communication skills. We analyzed these studies in terms of types of problem-solving strategies, the multiple control involved in problem solving, the extent to which problem solving occurred at the overt or covert level. In addition to suggesting limitations of the literature, we recommend areas for future research and practice.

Generalization of learned variability across multiple dimensions in humans

This study examined whether trained variability would generalize across dimensions of the target response. Two experiments used a computerized rectangle drawing task that required participants to click and drag a mouse cursor to create rectangles on a computer screen. In Experiment 1, one group received points when successive rectangles varied in their size, shape and location (VAR), another group were yoked to the VAR group and received points that were allocated to them using a yoking procedure (YOKE), regardless of the variability in the size, shape or location of the rectangle drawn. Variability was higher for a dimension when variability on that dimension was directly reinforced. In Experiment 2, three groups of participants received points when rectangles varied on two dimensions; each group differed in the two dimensions that required variation. Variability was again higher for the reinforced dimensions for two of the three groups. Comparison with the YOKE group showed that the variability on those dimensions where variability was not directly reinforced was affected by reinforcement for variability on the other dimensions. Specifically, the variability in Shape and Location was significantly higher when these two dimensions occurred with other dimensions where variability was reinforced (as in Experiment 2) compared to when they were not required to vary (as in the YOKE group). This suggests that, for these two groups, the reinforced variability on the other two dimensions generalized to the third dimension. Implications of this finding to our understanding of factors that promote behavioral variability are discussed.

Midsession shifts in reward probability and the control of behavioral variability

Extensive research has shown that the variability of organismal behavior is great when contingent reinforcement is delayed, small, or improbable. This research has generally employed stable response-reinforcer relationships, and therefore is limited in its explanatory scope with respect to a dynamic environment. We conducted two experiments to investigate whether pigeons' conditioned pecking behavior shows anticipatory or perseverative patterns of behavioral variability when the reinforcement probability reliably changes within experimental sessions. In Experiment 1, three pigeons received alternating sessions in which the reinforcement probability (35% or 4.2%) was shifted at the midpoint of each session in the presence of the same discrete cue. Experiment 2 featured a similar design, but with the inclusion of a discrete visual cue that changed at the session midpoint, and thus unambiguously indicated reinforcement probability. Local reinforcement rates only reliably controlled response variability when a discrete visual cue was available. Without this, pigeons did not discriminate between trial types in the first halves of sessions, and showed evidence of perseveration of response variability following a within-session shift. Critically, this is the first experimental demonstration that the relationship between reinforcement probability and behavioral variability is moderated by another factor (i.e., trial position within a session). This study thus expands our understanding of the control of behavioral variability as a function of experiential factors.

Intra-Individual Variability from a Lifespan Perspective: A Comparison of Latency and Accuracy Measures

Within-task variability across trials (intra-individual variability (IIV)) has been mainly studied using latency measures but rarely with accuracy measures. The aim of the Geneva Variability Study was to examine IIV in both latency and accuracy measures of cognitive performance across the lifespan, administering the same tasks to children, younger adults, and older adults. Six processing speed tasks (Response Time (RT) tasks, 8 conditions) and two working memory tasks scored in terms of the number of correct responses (Working Memory (WM)—verbal and visuo-spatial, 6 conditions), as well as control tasks, were administered to over 500 individuals distributed across the three age periods. The main questions were whether age differences in IIV would vary throughout the lifespan according (i) to the type of measure used (RTs vs. accuracy); and (ii) to task complexity. The objective of this paper was to present the general experimental design and to provide an essentially descriptive picture of the results. For all experimental tasks, IIV was estimated using intra-individual standard deviation (iSDr), controlling for the individual level (mean) of performance and for potential practice effects. As concerns RTs, and in conformity with a majority of the literature, younger adults were less variable than both children and older adults, and the young children were often the most variable. In contrast, IIV in the WM accuracy scores pointed to different age trends—age effects were either not observed or, when found, they indicated that younger adults were the more variable group. Overall, the findings suggest that IIV provides complementary information to that based on a mean performance, and that the relation of IIV to cognitive development depends on the type of measure used.

Differential effects of 3,4-methylenedioxymethamphetamine, methamphetamine, meta-Chlorophenylpiperazine, and scopolamine on behavioral repetition versus variation in rats

Acute administration of drugs of abuse, such as MDMA and methamphetamine, disrupts performance on many operant tasks, for example, those used to study memory. This might occur in part because drugs make behavior, in general, more repetitive or more variable, or because they produce a more global disruption to performance. The current study explored this across two experiments by employing Neuringer's 'reinforced variability' procedure. Varied behavior was reinforced at some times during the session and repetitive behavior at other times; lights signalled the behavior required. This procedure allowed an investigation of whether a particular drug made behavior more variable (affected behavior when repetition was required), more repetitive (affected behavior when variability was required), or produced a global disruption (affected both components). In Experiment 1, MDMA increased variability while methamphetamine affected both components. In Experiment 2, m-CPP affected both components while scopolamine affected both components at lower doses and increased variability at higher doses. These results indicate both that the reinforced variability procedure can be used to isolate the specific effects of drugs of abuse on the variability of behavior, and that the specific impact of a given drug needs to be considered when interpreting pharmacological disruptions to operant task performance.

Elicited versus emitted behavior: Time to abandon the distinction

The concept of emitted behavior was formulated as a part of the original argument for the validity of a new kind of learning called operant conditioning. The rationale for operant conditioning contrasted it with Pavlovian or classical conditioning, which was (and remains) fundamentally based on responses to conditioned and unconditioned stimuli. Classical conditioned responses were said to be elicited. In contrast, operant behavior was viewed as emitted and controlled primarily by response consequences rather than antecedents. I argue that the distinction between emitted and elicited behavior is no longer warranted for three major reasons. First, the distinction was based on a view of Pavlovian conditioning that is no longer viable. Second, the distinction is incompatible with both empirical data and contemporary conceptualizations of operant behavior. Third, the only way to overcome these problems is to define emitted and elicited in terms of the type of conditioning (operant and classical) that produces these behaviors, but that approach makes the definitions circular and does not avoid implications of the terms that are misleading and counterproductive in light of contemporary research and thinking.

Repetitive Behavior in Neurodevelopmental Disorders: Clinical and Translational Findings

Repetitive behavior refers to a highly heterogeneous set of responses associated with a wide range of conditions, including normative development. Treatment studies for aberrant repetitive behavior are limited although one promising approach involves conceptualizing such behavior as a generalized inflexibility or lack of variability in responding. Relatively little is known about the neurobiological mechanisms that mediate the development and expression of repetitive behavior, information critical to the design of effective pharmacotherapies, early interventions, and prevention strategies. We will review clinical findings in repetitive behavior as well as findings from animal models highlighting environmental factors and the role of cortical-basal ganglia circuitry in mediating the development and expression of these behaviors. Findings from animal models have included identification of a specific neural pathway important in mediating repetitive behavior. Moreover, pharmacological studies that support the importance of this pathway have led to the identification of novel potential therapeutic targets. Expanding the evidence base for environmental enrichment-derived interventions and focusing on generalized variability in responding will aid in addressing the broader problem of rigidity or inflexibility.

Reward expectation modulates variability in path choice in rats

The relationship between behavioral variability and reward expectation has been examined in recent years. This relationship is predictive: when an animal has a low expectation of reinforcement for a particular behavioral set, they engage in high levels of variability in their actions. We conducted two experiments to further investigate this relationship using a novel measure of behavioral variability. In Experiment 1, two groups of rats were trained to travel through a column maze, with many possible reinforced pathways, to receive either their maintenance diet (i.e., chow) or a highly palatable sugary reward (cereal). We hypothesized that animals trained with a maintenance-diet food source (chow) would demonstrate more variation in the pathways taken to the goal location than those animals trained with the highly palatable alternative. In Experiment 2, all rats were trained to travel through the maze to receive alternating outcomes of chow or cereal in a within-subjects design. Results from both experiments indicated that rats emitted greater variability in paths taken to the goal when the reinforcer was the maintenance chow. These results corroborate the relationship between reward and behavioral variability in a new behavioral measure.

Operant variability: a conceptual analysis

Some researchers claim that variability is an operant dimension of behavior. The present paper reviews the concept of operant behavior and emphasizes that differentiation is the behavioral process that demonstrates an operant relation. Differentiation is conceived as change in the overlap between two probability distributions: the distribution of reinforcement probability as a function of some response property (S distribution) and the probability distribution of the response property itself (R distribution). This concept implies that the differentiation process can be measured only if S distribution and R distribution are both established on the same response property. To determine whether the differentially reinforced behavioral variability fits the proposed concept of operant behavior, I examine the main procedures (lag n and threshold procedures) and the main dependent variable (U value) employed in the studies of operant variability. Because lag n and threshold procedures establish their S distributions on properties distinct from U value, differentiation cannot be measured over the change in U value. I conclude that studies of operant variability have failed to provide a direct demonstration that variability is an operant dimension of behavior. Hence, studies in which measures of variability provide a basis to measure differentiation can better support the claim that variability is an operant dimension of behavior.

Habits, action sequences and reinforcement learning

It is now widely accepted that instrumental actions can be either goal-directed or habitual; whereas the former are rapidly acquired and regulated by their outcome, the latter are reflexive, elicited by antecedent stimuli rather than their consequences. Model-based reinforcement learning (RL) provides an elegant description of goal-directed action. Through exposure to states, actions and rewards, the agent rapidly constructs a model of the world and can choose an appropriate action based on quite abstract changes in environmental and evaluative demands. This model is powerful but has a problem explaining the development of habitual actions. To account for habits, theorists have argued that another action controller is required, called model-free RL, that does not form a model of the world but rather caches action values within states allowing a state to select an action based on its reward history rather than its consequences. Nevertheless, there are persistent problems with important predictions from the model; most notably the failure of model-free RL correctly to predict the insensitivity of habitual actions to changes in the action-reward contingency. Here, we suggest that introducing model-free RL in instrumental conditioning is unnecessary, and demonstrate that reconceptualizing habits as action sequences allows model-based RL to be applied to both goal-directed and habitual actions in a manner consistent with what real animals do. This approach has significant implications for the way habits are currently investigated and generates new experimental predictions.

Behaviorism, latent learning, and cognitive maps: needed revisions in introductory psychology textbooks

This paper critically assesses the scholarship in introductory psychology textbooks in relation to the topic of latent learning. A review of the treatment of latent learning in 48 introductory psychology textbooks published between 1948 and 2004, with 21 of these texts published since 1999, reveals that the scholarship on the topic of latent learning demonstrated in introductory textbooks warrants improvement. Errors that persist in textbooks include the assertion that the latent learning experiments demonstrate unequivocally that reinforcement was not necessary for learning to occur, that behavioral theories could not account for the results of the latent learning experiments, that B. F. Skinner was an S-R association behaviorist who argued that reinforcement is necessary for learning to occur, and that because behavioral theories (including that of B. F. Skinner) were unable explain the results of the latent learning experiments the cognitive map invoked by Edward Tolman is the only explanation for latent learning. Finally, the validity of the cognitive map is typically accepted without question. Implications of the presence of these errors for students and the discipline are considered. Lastly, remedies are offered to improve the scholarship found in introductory psychology textbooks.

Reinforced variability enhances object exploration in shy and bold rats

In an open-field test, the Long-Evans (LE) strain of rats was identified as "bold" and the PVG strain as "shy." Some members of each strain then experienced 14 sessions of a common enrichment procedure, namely exposure to a series of novel objects (Exposed). Others in each strain were explicitly reinforced with food pellets for variable interactions with the same objects (Reinforced). Both experience and strain influenced object interactions. In particular, Reinforced rats interacted more variably with the objects - contacting, probing, pushing and so forth - than did the Exposed; and LEs interacted more variably than PVGs. Foraging proficiency in the same rats was then studied in a transfer-of-training test. Food pellets were hidden among never-before experienced objects and the rats were permitted to explore freely. Reinforced rats discovered and consumed more pellets than Exposed; and LEs discovered and consumed more than PVGs. Thus a bold genetic strain and reinforcement of variability independently contributed to successful foraging behavior.

Evidence-based behavioral interventions for repetitive behaviors in autism

Restricted and repetitive behaviors (RRBs) are a core symptom of autism spectrum disorders (ASD). There has been an increased research emphasis on repetitive behaviors; however, this research primarily has focused on phenomenology and mechanisms. Thus, the knowledge base on interventions is lagging behind other areas of research. The literature suggests there are evidence-based practices to treat "lower order" RRBs in ASD (e.g., stereotypies); yet, there is a lack of a focused program of intervention research for "higher order" behaviors (e.g., insistence on sameness). This paper will (a) discuss barriers to intervention development for RRBs; (b) review evidence-based interventions to treat RRBs in ASD, with a focus on higher order behaviors; and (c) conclude with recommendations for practice and research.

Effects of a family-implemented treatment on the repetitive behaviors of children with autism

The restricted and repetitive behaviors of children with autism can interfere with family functioning as well as learning and socialization opportunities for the child. To date, neither pharmacological nor comprehensive behavioral treatments have been found to be consistently effective at significantly reducing children's engagement in repetitive behaviors. We developed Family-Implemented Treatment for Behavioral Inflexibility (FITBI) to target the full variety of repetitive behaviors found in autism. For the current study, a therapist and parents of five children with autism (mean age = 48 months) co-implemented FITBI in a clinic setting over a 12-week treatment period. Using single case design methodology, significant reductions in repetitive behaviors were found for all participants and maintenance of treatment effects for 4 of 5 participants.

The Modulation of Operant Variation by the Probability, Magnitude, and Delay of Reinforcement

Recent studies have demonstrated that the expectation of reward delivery has an inverse relationship with operant behavioral variation (e.g., Stahlman, Roberts, & Blaisdell, 2010). Research thus far has largely focused on one aspect of reinforcement - the likelihood of food delivery. In two experiments with pigeons, we examined the effect of two other aspects of reinforcement: the magnitude of the reward and the temporal delay between the operant response and outcome delivery. In the first experiment, we found that a large reward magnitude resulted in reduced spatiotemporal variation in pigeons' pecking behavior. In the second experiment, we found that a 4-s delay between response-dependent trial termination and reward delivery increased variation in behavior. These results indicate that multiple dimensions of the reinforcer modulate operant response variation.

A behavioral analysis of altruism

Altruistic acts have been defined, in economic terms, as "…costly acts that confer economic benefits on other individuals" (Fehr and Fischbacher, 2003). In multi-player, one-shot prisoner's dilemma games, a significant number of players behave altruistically; their behavior benefits each of the other players but is costly to them. We consider three potential explanations for such altruism. The first explanation, following a suggestion by the philosopher Derek Parfit, assumes that players devise a strategy to avoid being free-loaders-and that in the present case this strategy dictates cooperation. The second explanation says that cooperators reject the one-shot aspect of the game and behave so as to maximize reward over a series of choices extending beyond the present situation (even though reward is not maximized in the present case). This explanation assumes that people may learn to extend the boundaries of their selves socially (beyond their own skin) as well as temporally (beyond the present moment). We propose a learning mechanism for such behavior analogous to the biological, evolutionary mechanism of group selection. The third explanation assumes that people's altruism is based on a straightforward balancing of undiscounted costs to themselves against discounted benefits to others (social discounting). The three proposed explanations of altruism complement each other.

Increasing response diversity in children with autism

Repetitive and invariant behavior is a diagnostic feature of autism. We implemented a lag reinforcement schedule to increase response diversity for 6 participants with autism aged 6 to 10 years, 4 of whom also received prompting plus additional training. These procedures appeared to increase the variety of building-block structures, demonstrating that an intervention that includes differential reinforcement can increase response diversity for children with an autism spectrum disorder.

Towards a scientific concept of free will as a biological trait: spontaneous actions and decision-making in invertebrates

Until the advent of modern neuroscience, free will used to be a theological and a metaphysical concept, debated with little reference to brain function. Today, with ever increasing understanding of neurons, circuits and cognition, this concept has become outdated and any metaphysical account of free will is rightfully rejected. The consequence is not, however, that we become mindless automata responding predictably to external stimuli. On the contrary, accumulating evidence also from brains much smaller than ours points towards a general organization of brain function that incorporates flexible decision-making on the basis of complex computations negotiating internal and external processing. The adaptive value of such an organization consists of being unpredictable for competitors, prey or predators, as well as being able to explore the hidden resource deterministic automats would never find. At the same time, this organization allows all animals to respond efficiently with tried-and-tested behaviours to predictable and reliable stimuli. As has been the case so many times in the history of neuroscience, invertebrate model systems are spearheading these research efforts. This comparatively recent evidence indicates that one common ability of most if not all brains is to choose among different behavioural options even in the absence of differences in the environment and perform genuinely novel acts. Therefore, it seems a reasonable effort for any neurobiologist to join and support a rather illustrious list of scholars who are trying to wrestle the term 'free will' from its metaphysical ancestry. The goal is to arrive at a scientific concept of free will, starting from these recently discovered processes with a strong emphasis on the neurobiological mechanisms underlying them.

Measuring and reducing college students' procrastination

We examined college students' procrastination when studying for weekly in-class quizzes. Two schedules of online practice quiz delivery were compared using a multiple baseline design. When online study material was made available noncontingently, students usually procrastinated. When access to additional study material was contingent on completing previous study material, studying was more evenly distributed. Overall, the mean gain in percentage correct scores on weekly in-class quizzes relative to pretests was greater during contingent access than during noncontingent access conditions.

Uncertainty compensation in human attention: evidence from response times and fixation durations

Background

Uncertainty and predictability have remained at the center of the study of human attention. Yet, studies have only examined whether response times (RT) or fixations were longer or shorter under levels of stimulus uncertainty. To date, no study has examined patterns of stimuli and responses through a unifying framework of uncertainty.

Methodology/Principal Findings

We asked 29 college students to generate repeated responses to a continuous series of visual stimuli presented on a computer monitor. Subjects produced these responses by pressing on a keypad as soon a target was detected (regardless of position) while the durations of their visual fixations were recorded. We manipulated the level of stimulus uncertainty in space and time by changing the number of potential stimulus locations and time intervals between stimulus presentations. To allow the analyses to be conducted using uncertainty as common description of stimulus and response we calculated the entropy of the RT and fixation durations. We tested the hypothesis of uncertainty compensation across space and time by fitting the RT and fixation duration entropy values to a quadratic surface. The quadratic surface accounted for 80% of the variance in the entropy values of both RT and fixation durations. RT entropy increased as a function of spatial and temporal uncertainty of the stimulus, alongside a symmetric, compensatory decrease in the entropy of fixation durations as the level of spatial and temporal uncertainty of the stimuli was increased.

Conclusions/Significance

Our results demonstrate that greater uncertainty in the stimulus leads to greater uncertainty in the response, and that the effects of spatial and temporal uncertainties are compensatory. We also observed compensatory relationship across the entropies of fixation duration and RT, suggesting that a more predictable visual search strategy leads to more uncertain response patterns and vice versa.

Métodos de pesquisa em análise do comportamento

Apresenta-se neste artigo a discussão sobre o status da análise do comportamento como disciplina científica, bem como sua relação com outras ciências e as implicações desta discussão para a caracterização dos métodos de pesquisa em análise do comportamento. Várias classificações dos métodos de pesquisa são apresentadas, com destaque para o método experimental, tendo em vista seu papel na construção da disciplina. Os supostos e as principais características dos delineamentos de sujeito único são apresentados. E, por fim, são brevemente discutidas algumas questões que se originam da interação e aparente confronto entre conceituação e prática de pesquisa em análise do comportamento, tais como: reversibilidade comportamental, variabilidade comportamental, procedimentos de tentativa e procedimentos de operante livre, análise molecular e análise molar, tratamento estatístico e quantificação, pesquisa experimental com humanos.

The importance of being active

The successful stimulus-response approach to the organization of behavior has been the dominating paradigm for much of the psychology and neuroscience of the 20th century. Martin Heisenberg is a pioneer in championing the idea that all brains, even comparatively simple ones such as those of insects, instead operate according to output-input principles. Since the 1970s, his research produces evidence that the fruit fly, Drosophila melanogaster, is capable of spontaneous behavioral activity, and that the flies use it to control sensory input (i.e., operant behavior). Today, more and more evidence is accumulating also from fields outside of neuroscience that, indeed, one of the common, defining principles of all brains is this concept of operant behavior. Drawing from this evidence, it becomes clear that the conceptually simple process of generating activity and evaluating its consequences forms one of the fundamental cornerstones not only for all of our human nature, but also for our social coherence. This review recapitulates Heisenberg's most critical experiments and provides an overview over the current literature on the role of spontaneous activity in the ecology and evolution of brains. I conclude that spontaneous activity is both a necessary prerequisite and an inevitable consequence of evolution.