Discrimination of variable schedules is controlled by interresponse times proximal to reinforcement

Proprioceptive stimuli and habit formation: Interresponse time mediated behavior in CD-1 mice

The consolidation of behavioral sequences into relatively ballistic habits is thought to involve the formation of stimulus - response associations. Typically, the stimuli in these associations are assumed to be exteroceptive, i.e., external to the organism. However, responses, themselves, also possess stimulus properties that can mediate behavior. Indeed, these "proprioceptive cues" have long been hypothesized to underlie habit formation (Hull, 1934a, 1934b). One such stimulus involves the time durations between responses - a stimulus termed interresponse time (IRT). We hypothesize that IRTs can come to serve as stimuli that differentially control response elements during habit formation. To examine this hypothesis we report on two experiments that asked whether CD-1 mice utilize IRTs to structure behavior in a two-choice environment. In experiment 1, eight mice were exposed to a free-operant concurrent variable-interval (VI) 30-s VI 60-s reinforcement schedule. We found that switch and stay responses were differentially correlated with IRT durations. In Experiment 2 we directly and differentially reinforced stay/switch responses based on IRT durations in a two-lever procedure. For four of the subjects, the probability of receiving reinforcement after switch responses was proportional to IRT duration. For five of the subjects, these reinforcement probabilities were inversely proportional to IRT duration. Regardless, all of our subjects learned to emit IRT-mediated switching behavior that matched the reinforcement contingencies. Together, Experiments 1 and 2 provide the first evidence of which we are aware that IRTs can come to control sequential choice behavior in mice.

Neural substrates underlying effort, time, and risk-based decision making in motivated behavior

All mobile organisms rely on adaptive motivated behavior to overcome the challenges of living in an environment in which essential resources may be limited. A variety of influences ranging from an organism's environment, experiential history, and physiological state all influence a cost-benefit analysis which allows motivation to energize behavior and direct it toward specific goals. Here we review the substantial amount of research aimed at discovering the interconnected neural circuits which allow organisms to carry-out the cost-benefit computations which allow them to behave in adaptive ways. We specifically focus on how the brain deals with different types of costs, including effort requirements, delays to reward and payoff riskiness. An examination of this broad literature highlights the importance of the extended neural circuits which enable organisms to make decisions about these different types of costs. This involves Cortical Structures, including the Anterior Cingulate Cortex (ACC), the Orbital Frontal Cortex (OFC), the Infralimbic Cortex (IL), and prelimbic Cortex (PL), as well as the Baso-Lateral Amygdala (BLA), the Nucleus Accumbens (NAcc), the Ventral Pallidal (VP), the Sub Thalamic Nucleus (STN) among others. Some regions are involved in multiple aspects of cost-benefit computations while the involvement of other regions is restricted to information relating to specific types of costs.