1
|
Cho SA, Cho YS. Attentional Orienting by Non-informative Cue Is Shaped via Reinforcement Learning. Front Psychol 2020; 10:2884. [PMID: 32010011 PMCID: PMC6974624 DOI: 10.3389/fpsyg.2019.02884] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2019] [Accepted: 12/05/2019] [Indexed: 11/13/2022] Open
Abstract
It has been demonstrated that a reward-associated stimulus feature captures attention involuntarily. The present study tested whether spatial attentional orienting is biased via reinforcement learning. Participants were to identify a target stimulus presented in one of two placeholders, preceded by a non-informative arrow cue at the center of the display. Importantly, reward was available when the target occurred at a location cued by a reward cue, defined as a specific color (experiments 1 and 3) or a color-direction combination (experiment 2). The attentional bias of the reward cue was significantly increased as trials progressed, resulting in a greater cue-validity effect for the reward cue than the no-reward cue. This attentional bias was still evident even when controlling for the possibility that the incentive salience of the reward cue color modulates the cue-validity effect (experiment 2) or when the reward was withdrawn after reinforcement learning (experiment 3). However, it disappeared when the reward was provided regardless of cue validity (experiment 4), implying that the reinforcement contingency between reward and attentional orienting is a critical determinant of reinforcement learning-based spatial attentional modulation. Our findings highlight that a spatial attentional bias is shaped by value via reinforcement learning.
Collapse
Affiliation(s)
| | - Yang Seok Cho
- Department of Psychology, Korea University, Seoul, South Korea
| |
Collapse
|
2
|
Edgar D, Hall G, Pearce JM. Enhancement of Food-Rewarded Instrumental Responding by an Appetitive Conditioned Stimulus. ACTA ACUST UNITED AC 2018. [DOI: 10.1080/14640748108400825] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
Four experiments are reported in which a stimulus (with a minimum duration of 60 s) signalling the delivery of “free” food was presented to rats lever-pressing for food available on a variable interval schedule. It was found that responding was enhanced in the presence of the stimulus when the baseline schedule of reinforcement was lean (Experiment I) and that the enhancement was dependent upon the pairing of the stimulus with free food (Experiments II and III). Experiment IV showed that an enhancement could be found after initial training in which stimulus-food pairings were given to subjects that were not concurrently lever pressing for food. It is argued that these results are consistent with the suggestion that an appetitive conditioned stimulus can energise appetitive instrumental behaviour.
Collapse
Affiliation(s)
- David Edgar
- Department of Psychology, University of York, Heslington, York, England
| | - Geoffrey Hall
- Department of Psychology, University of York, Heslington, York, England
| | - John M. Pearce
- Psychological Laboratory, University of Cambridge, Downing Street, Cambridge, England
| |
Collapse
|
3
|
Holman JG, Mackintosh NJ. The Control of Appetitive Instrumental Responding does not Depend on Classical Conditioning to the Discriminative Stimulus. ACTA ACUST UNITED AC 2018. [DOI: 10.1080/14640748108400826] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
In Experiment I, rats were exposed to a classical relationship between a clicker-light compound and response-independent food. Conditioning to the light was blocked if the clicker had previously served as a classical signal for food, but not if it had been established as a discriminative stimulus for food-reinforced lever pressing. In Experiment II, a tone-light compound served as a discriminative stimulus for lever pressing. Control by the light was blocked if the tone was independently trained as a discriminative stimulus, but not if it was trained as a classical signal for response-independent food. These results suggest that discriminative stimuli do not come to control appetitive instrumental responding by virtue of their implicit classical relationship to the instrumental reinforcer.
Collapse
Affiliation(s)
- J. G. Holman
- Laboratory of Experimental Psychology, University of Sussex Brighton BN1 9QG, England
| | - N. J. Mackintosh
- Laboratory of Experimental Psychology, University of Sussex Brighton BN1 9QG, England
| |
Collapse
|
4
|
Differential conditioning of conditioned enhancement and positive conditioned suppression. ACTA ACUST UNITED AC 2013. [DOI: 10.3758/bf03336757] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
5
|
The effect of the magnitude of response-independent food on conditioned enhancement. ACTA ACUST UNITED AC 2013. [DOI: 10.3758/bf03329354] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
6
|
Models of trace decay, eligibility for reinforcement, and delay of reinforcement gradients, from exponential to hyperboloid. Behav Processes 2011; 87:57-63. [PMID: 21215304 DOI: 10.1016/j.beproc.2010.12.016] [Citation(s) in RCA: 45] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/15/2010] [Revised: 12/24/2010] [Accepted: 12/27/2010] [Indexed: 11/24/2022]
Abstract
Behavior such as depression of a lever or perception of a stimulus may be strengthened by consequent behaviorally significant events (BSEs), such as reinforcers. This is the Law of Effect. As time passes since its emission, the ability for the behavior to be reinforced decreases. This is trace decay. It is upon decayed traces that subsequent BSEs operate. If the trace comes from a response, it constitutes primary reinforcement; if from perception of an extended stimulus, it is classical conditioning. This paper develops simple models of these processes. It premises exponentially decaying traces related to the richness of the environment, and conditioned reinforcement as the average of such traces over the extended stimulus, yielding an almost-hyperbolic function of duration. The models account for some data, and reinforce the theories of other analysts by providing a sufficient account of the provenance of these effects. It leads to a linear relation between sooner and later isopreference delays whose slope depends on sensitivity to reinforcement, and intercept on that and the steepness of the delay gradient. Unlike human prospective judgments, all control is vested in either primary or secondary reinforcement processes; therefore the use of the term discounting, appropriate for humans, may be less descriptive of the behavior of nonverbal organisms.
Collapse
|
7
|
Woodruff G, Conner N, Gamzu E, Williams DR. Associative interaction: joint control of key pecking by stimulus-reinforcer and response-reinforcer relationships. J Exp Anal Behav 2010; 28:133-44. [PMID: 16812020 PMCID: PMC1333625 DOI: 10.1901/jeab.1977.28-133] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
Abstract
The joint control of rate of key pecking in pigeons by stimulus-reinforcer and response-reinforcer relationships was studied in the context of a two-component multiple schedule of reinforcement. Food presentation was always associated with one component and extinction with the other. The stimulus-reinforcer relationship was manipulated by varying the relative durations of the two components. In the food-presentation component, a fixed rate of reinforcement, independent of rate of responding, was generated by a schedule referred to as "T*". One aspect of the response-reinforcer relationship, contiguity, was manipulated by varying the percentage of delayed reinforcers. With the multiple T* extinction schedule, stimulus-reinforcer and response-reinforcer relationships could be varied independently of one another. Rate of key pecking was sensitive to manipulations of both relationships. However, significant differential effects due to either the stimulus-reinforcer or response-reinforcer relationship were obtained only when the other relationship was weak: stimulus-reinforcer and response-reinforcer relationships interacted in the joint control of responding.
Collapse
|
8
|
Holmes NM, Marchand AR, Coutureau E. Pavlovian to instrumental transfer: a neurobehavioural perspective. Neurosci Biobehav Rev 2010; 34:1277-95. [PMID: 20385164 DOI: 10.1016/j.neubiorev.2010.03.007] [Citation(s) in RCA: 174] [Impact Index Per Article: 12.4] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/09/2009] [Revised: 03/16/2010] [Accepted: 03/31/2010] [Indexed: 11/27/2022]
Abstract
Pavlovian-to-instrumental transfer (PIT) is a key concept in developing our understanding of cue-controlled behaviours. Here we have reviewed the literature on behavioural and neurobiological factors that influence PIT. Meta-analyses of the data for individual groups in PIT studies revealed that PIT is related to both the order and amounts of instrumental and Pavlovian training, and that it is critically determined by competition between instrumental and Pavlovian responses. We directly addressed the role of response competition in PIT in two experiments which showed that extensive Pavlovian conditioning produced more Pavlovian magazine visits and weaker PIT than moderate Pavlovian conditioning (Experiment 1); and that PIT lost after extensive Pavlovian conditioning was restored by Pavlovian extinction training (Experiment 2). These findings confirm that response competition is indeed an important determinant of PIT. This has significant implications for lesion and inactivation studies that assess the neurobiological substrates of PIT, as well as attempts to demonstrate PIT in the drug self-administration paradigm where the effect is yet to be reliably shown.
Collapse
|
9
|
Van Hest A, Van Haaren F, Kop P, Van Der Schoot F. Operant-Pavlovian interactions: Ratio-schedules and the effects of the duration and location of a stimulus preceding response-independent food. Behav Processes 1986; 13:149-58. [DOI: 10.1016/0376-6357(86)90022-7] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/18/1986] [Indexed: 11/29/2022]
|
10
|
|
11
|
Van Haaren F, Kop PF, van der Schoot F. Operant-Pavlovian interactions: The effect of the number of stimuli with free-food depends on the operant baseline. Behav Processes 1985; 10:53-61. [DOI: 10.1016/0376-6357(85)90117-2] [Citation(s) in RCA: 17] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/09/1984] [Indexed: 10/27/2022]
|
12
|
Hemmes NS, Rubinsky HJ. Conditional acceleration and external disinhibition of operant lever pressing by prereward, neutral, and reinforcing stimuli. J Exp Anal Behav 1982; 38:157-68. [PMID: 16812294 PMCID: PMC1347811 DOI: 10.1901/jeab.1982.38-157] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
Abstract
Rats responding under a differential-reinforcement-of-low-rate schedule increased their rates of lever pressing during a 20-second click/flash stimulus that preceded the delivery of a response-independent food pellet. The increase could not be attributed to suppression of collateral behavior that has been said to mediate temporally-spaced responding. We propose that the prereward stimulus functioned as an external disinhibitor of lever pressing that had been inhibited by the constraints of the operant schedule. Support is derived from the observed disinhibitory effects of a 10-second unpaired click/flash stimulus and of unsignaled, response-independent pellets that were presented while the animals were responding under the same schedule.
Collapse
|
13
|
Lovibond PF. Effects of long- and variable-duration signals for food on activity, instrumental responding, and eating. LEARNING AND MOTIVATION 1980. [DOI: 10.1016/0023-9690(80)90011-9] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
14
|
|
15
|
Wessells MG. The effects of the stimulus-reinforcer correlation in a discrete-trials IRT>t procedure. J Exp Anal Behav 1979; 31:307-20. [PMID: 16812133 PMCID: PMC1332861 DOI: 10.1901/jeab.1979.31-307] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
The correlation between a keylight and food in a discrete-trials, interresponse-time-greater-than 6-sec (IRT>6-sec) procedure was varied by manipulating the rate of response-independent food presentation in the intertrial interval. When the correlation was positive, the rates of pecking in the IRT>6-sec condition were high and food was obtained on only about 5% of the trials. Likewise, responding was maintained at a high rate in yoked birds that received the same presentations of the light and food as the birds in the IRT>6-sec condition. When the rate of reinforcement between trials was equated to or made greater than the rate of reinforcement within trials, the response rate decreased for all birds, and those decreases were considerably larger for the yoked birds. However, the percentage of trials in which reinforced responses occurred under the IRT>6-sec procedure did not increase substantially when the light and food were either uncorrelated or negatively correlated. The percentage of trials in which a reinforcer was obtained increased when the keylight was left on continuously and the discriminative stimulus was not presented on the key. The results show that the stimulus-reinforcer correlation affects responding in the discrete-trials IRT>6-sec procedure, but that the effects of the stimulus-reinforcer correlation vary as a function of whether reinforcement is response-dependent or response-independent. The differences between the effects of response-independent and response-dependent pairings and nonpairings of the light and food are best accounted for in terms of differences in the control of responding by background stimuli.
Collapse
|
16
|
Stubbs DA, Hughes JE, Cohen SL. Positive conditioned suppression: an explanation in terms of multiple and concurrent schedules. J Exp Anal Behav 1978; 30:329-43. [PMID: 16812113 PMCID: PMC1332777 DOI: 10.1901/jeab.1978.30-329] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022]
Abstract
Rats performed under a baseline variable-interval schedule of food presentation. A response-independent food schedule was then superimposed on the baseline schedule for different periods of time across different conditions. The response-independent schedule operated for the whole session in some conditions, intermittently for sixty second periods in some, and intermittently for ten-second periods in others. Under these latter two sets of conditions, the response-independent food schedule was stimulus correlated and alternated with the baseline schedule according to a multiple schedule. Response-independent food presentations always suppressed responding. The degree of suppression tended to increase the longer the period of response-independent food. Control conditions, in which the superimposed schedule was response-dependent, rather than response-independent, did not produce response suppression. The results fit an analysis of positive conditioned suppression phenomena in the context of multiple and concurrent schedule effects.
Collapse
|
17
|
Gutman A, Maier SF. Operant and Pavlovian factors in cross-response transfer of inhibitory stimulus control. LEARNING AND MOTIVATION 1978. [DOI: 10.1016/0023-9690(78)90008-5] [Citation(s) in RCA: 10] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
18
|
Green L. Are there two classes of classically-conditioned responses? THE PAVLOVIAN JOURNAL OF BIOLOGICAL SCIENCE 1978; 13:154-62. [PMID: 750966 DOI: 10.1007/bf03001388] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/24/2022]
Abstract
It is proposed that two response classes interact during classical conditioning: (1) a specific effect which directs responding towards or away from the CS, depending upon whether a positive or negative outcome is signaled, and (2) a general, emotional effect which interferes with responding. The interaction between these two response classes can account for the varied results obtained under CER and autoshaping procedures, and must be considered in any analysis which attempts to account for the complex nature of the conditioned response.
Collapse
|
19
|
|
20
|
Osborne SR, Killeen PR. Temporal properties of responding during stimuli that precede response-independent food. LEARNING AND MOTIVATION 1977. [DOI: 10.1016/0023-9690(77)90050-9] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|
21
|
Green L, Rachlin H. On the directionality of key pecking during signals for appetitive and aversive events. LEARNING AND MOTIVATION 1977. [DOI: 10.1016/0023-9690(77)90051-0] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/26/2022]
|