1
|
Miller RR. The Illusion of Pure Reason in Science: A Cautionary Note. Behav Processes 2023; 207:104863. [PMID: 36965606 DOI: 10.1016/j.beproc.2023.104863] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2022] [Revised: 02/09/2023] [Accepted: 02/13/2023] [Indexed: 03/27/2023]
Abstract
Introspection tells people that their behavior is both consciously reasoned and functional (i.e., rational), at least based on the evidence available to them. In contrast, research has found that much human behavior reported to be consciously determined, is strongly influenced by heuristics and the mechanistic principles of associative learning that usually function unconsciously and are sometimes sub-optimal. Scientists are trained to base their conclusions on a rational analysis of evidence, which enhances the scientific validity of their conclusions. But scientific training appears to do little to constrain the role of unconscious heuristics. The present point is that scientists are humans and, as such, they are subject to the influence of heuristics in their scientific conclusions just as laypeople are in their everyday behavior. As an example, the availability heuristic and how it seemingly feeds the repetition-induced truth effect are described. One consequence of this is that failures to replicate frequently cited papers do little to devalue the irreplicable reports. Although unconscious heuristics influence the scientific thinking of researchers, scientists are typically unaware of the role of these heuristics due to their operating below the horizon of introspection. This appears to explain the persistence, in light of overwhelming evidence to the contrary, of the views by many researchers that 'a prediction error is necessary for learning' and that 'reactivated memories have to be reconsolidated to be retained for future access.'
Collapse
|
2
|
Seitz BM, Hoang IB, DiFazio LE, Blaisdell AP, Sharpe MJ. Dopamine errors drive excitatory and inhibitory components of backward conditioning in an outcome-specific manner. Curr Biol 2022; 32:3210-3218.e3. [PMID: 35752165 DOI: 10.1016/j.cub.2022.06.035] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 04/29/2022] [Accepted: 06/13/2022] [Indexed: 01/06/2023]
Abstract
For over two decades, phasic activity in midbrain dopamine neurons was considered synonymous with the prediction error in temporal-difference reinforcement learning.1-4 Central to this proposal is the notion that reward-predictive stimuli become endowed with the scalar value of predicted rewards. When these cues are subsequently encountered, their predictive value is compared to the value of the actual reward received, allowing for the calculation of prediction errors.5,6 Phasic firing of dopamine neurons was proposed to reflect this computation,1,2 facilitating the backpropagation of value from the predicted reward to the reward-predictive stimulus, thus reducing future prediction errors. There are two critical assumptions of this proposal: (1) that dopamine errors can only facilitate learning about scalar value and not more complex features of predicted rewards, and (2) that the dopamine signal can only be involved in anticipatory cue-reward learning in which cues or actions precede rewards. Recent work7-15 has challenged the first assumption, demonstrating that phasic dopamine signals across species are involved in learning about more complex features of the predicted outcomes, in a manner that transcends this value computation. Here, we tested the validity of the second assumption. Specifically, we examined whether phasic midbrain dopamine activity would be necessary for backward conditioning-when a neutral cue reliably follows a rewarding outcome.16-20 Using a specific Pavlovian-to-instrumental transfer (PIT) procedure,21-23 we show rats learn both excitatory and inhibitory components of a backward association, and that this association entails knowledge of the specific identity of the reward and cue. We demonstrate that brief optogenetic inhibition of VTADA neurons timed to the transition between the reward and cue reduces both of these components of backward conditioning. These findings suggest VTADA neurons are capable of facilitating associations between contiguously occurring events, regardless of the content of those events. We conclude that these data may be in line with suggestions that the VTADA error acts as a universal teaching signal. This may provide insight into why dopamine function has been implicated in myriad psychological disorders that are characterized by very distinct reinforcement-learning deficits.
Collapse
Affiliation(s)
- Benjamin M Seitz
- Department of Psychology, University of California, Los Angeles, Portola Plaza, Los Angeles, CA 91602, USA
| | - Ivy B Hoang
- Department of Psychology, University of California, Los Angeles, Portola Plaza, Los Angeles, CA 91602, USA
| | - Lauren E DiFazio
- Department of Psychology, University of California, Los Angeles, Portola Plaza, Los Angeles, CA 91602, USA
| | - Aaron P Blaisdell
- Department of Psychology, University of California, Los Angeles, Portola Plaza, Los Angeles, CA 91602, USA
| | - Melissa J Sharpe
- Department of Psychology, University of California, Los Angeles, Portola Plaza, Los Angeles, CA 91602, USA.
| |
Collapse
|
3
|
Dhamija P, Wong A, Gilboa A. Early Auditory Event Related Potentials Distinguish Higher-Order From First-Order Aversive Conditioning. Front Behav Neurosci 2022; 16:751274. [PMID: 35221944 PMCID: PMC8879319 DOI: 10.3389/fnbeh.2022.751274] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Accepted: 01/03/2022] [Indexed: 11/17/2022] Open
Abstract
Stimuli in reality rarely co-occur with primary reward or punishment to allow direct associative learning of value. Instead, value is thought to be inferred through complex higher-order associations. Rodent research has demonstrated that the formation and maintenance of first-order and higher-order associations are supported by distinct neural substrates. In this study, we explored whether this pattern of findings held true for humans. Participants underwent first-order and subsequent higher-order conditioning using an aversive burst of white noise or neutral tone as the unconditioned stimuli. Four distinct tones, initially neutral, served as first-order and higher-order conditioned stimuli. Autonomic and neural responses were indexed by pupillometry and evoked response potentials (ERPs) respectively. Conditioned aversive values of first-order and higher-order stimuli led to increased autonomic responses, as indexed by pupil dilation. Distinct temporo-spatial auditory evoked response potentials were elicited by first-order and high-order conditioned stimuli. Conditioned first-order responses peaked around 260 ms and source estimation suggested a primary medial prefrontal and amygdala source. Conversely, conditioned higher-order responses peaked around 120 ms with an estimated source in the medial temporal lobe. Interestingly, pupillometry responses to first-order conditioned stimuli were diminished after higher order training, possibly signifying concomitant incidental extinction, while responses to higher-order stimuli remained. This suggests that once formed, higher order associations are at least partially independent of first order conditioned representations. This experiment demonstrates that first-order and higher-order conditioned associations have distinct neural signatures, and like rodents, the medial temporal lobe may be specifically involved with higher-order conditioning.
Collapse
Affiliation(s)
- Prateek Dhamija
- Department of Psychology, University of Toronto, Toronto, ON, Canada
- Rotman Research Institute, Baycrest, Toronto, ON, Canada
- *Correspondence: Prateek Dhamija,
| | - Allison Wong
- Department of Psychology, University of Toronto, Toronto, ON, Canada
- Rotman Research Institute, Baycrest, Toronto, ON, Canada
| | - Asaf Gilboa
- Department of Psychology, University of Toronto, Toronto, ON, Canada
- Rotman Research Institute, Baycrest, Toronto, ON, Canada
- Asaf Gilboa,
| |
Collapse
|
4
|
Prével A, Krebs RM. Higher-Order Conditioning With Simultaneous and Backward Conditioned Stimulus: Implications for Models of Pavlovian Conditioning. Front Behav Neurosci 2021; 15:749517. [PMID: 34858147 PMCID: PMC8632485 DOI: 10.3389/fnbeh.2021.749517] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Accepted: 10/18/2021] [Indexed: 11/23/2022] Open
Abstract
In a new environment, humans and animals can detect and learn that cues predict meaningful outcomes, and use this information to adapt their responses. This process is termed Pavlovian conditioning. Pavlovian conditioning is also observed for stimuli that predict outcome-associated cues; a second type of conditioning is termed higher-order Pavlovian conditioning. In this review, we will focus on higher-order conditioning studies with simultaneous and backward conditioned stimuli. We will examine how the results from these experiments pose a challenge to models of Pavlovian conditioning like the Temporal Difference (TD) models, in which learning is mainly driven by reward prediction errors. Contrasting with this view, the results suggest that humans and animals can form complex representations of the (temporal) structure of the task, and use this information to guide behavior, which seems consistent with model-based reinforcement learning. Future investigations involving these procedures could result in important new insights on the mechanisms that underlie Pavlovian conditioning.
Collapse
Affiliation(s)
- Arthur Prével
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Ruth M Krebs
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| |
Collapse
|
5
|
Prével A, Krebs RM, Kukkonen N, Braem S. Selective reinforcement of conflict processing in the Stroop task. PLoS One 2021; 16:e0255430. [PMID: 34329341 PMCID: PMC8323904 DOI: 10.1371/journal.pone.0255430] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2021] [Accepted: 07/16/2021] [Indexed: 11/18/2022] Open
Abstract
Motivation signals have been shown to influence the engagement of cognitive control processes. However, most studies focus on the invigorating effect of reward prospect, rather than the reinforcing effect of reward feedback. The present study aimed to test whether people strategically adapt conflict processing when confronted with condition-specific congruency-reward contingencies in a manual Stroop task. Results show that the size of the Stroop effect can be affected by selectively rewarding responses following incongruent versus congruent trials. However, our findings also suggest important boundary conditions. Our first two experiments only show a modulation of the Stroop effect in the first half of the experimental blocks, possibly due to our adaptive threshold procedure demotivating adaptive behavior over time. The third experiment showed an overall modulation of the Stroop effect, but did not find evidence for a similar modulation on test items, leaving open whether this effect generalizes to the congruency conditions, or is stimulus-specific. More generally, our results are consistent with computational models of cognitive control and support contemporary learning perspectives on cognitive control. The findings also offer new guidelines and directions for future investigations on the selective reinforcement of cognitive control processes.
Collapse
Affiliation(s)
- Arthur Prével
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
- * E-mail:
| | - Ruth M. Krebs
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Nanne Kukkonen
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Senne Braem
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| |
Collapse
|
6
|
Abstract
In contrast to the large body of work demonstrating second-order conditioning (SOC) in non-human animals, the evidence for SOC in humans is scant. In this review, I examine the existing literature and suggest theoretical and procedural explanations for why SOC has been so elusive in humans. In particular, I discuss potential interactions with conditioned inhibition, whether SOC is rational, and propose critical parameters needed to obtain the effect. I conclude that SOC is a real but difficult phenomenon to obtain in humans, and suggest directions for future research.
Collapse
Affiliation(s)
- Jessica C. Lee
- School of Psychology, University of New South Wales, Sydney, NSW, Australia
| |
Collapse
|
7
|
Goode TD, Ressler RL, Acca GM, Miles OW, Maren S. Bed nucleus of the stria terminalis regulates fear to unpredictable threat signals. eLife 2019; 8:46525. [PMID: 30946011 PMCID: PMC6456295 DOI: 10.7554/elife.46525] [Citation(s) in RCA: 58] [Impact Index Per Article: 11.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2019] [Accepted: 03/28/2019] [Indexed: 12/15/2022] Open
Abstract
The bed nucleus of the stria terminalis (BNST) has been implicated in conditioned fear and anxiety, but the specific factors that engage the BNST in defensive behaviors are unclear. Here we examined whether the BNST mediates freezing to conditioned stimuli (CSs) that poorly predict the onset of aversive unconditioned stimuli (USs) in rats. Reversible inactivation of the BNST selectively reduced freezing to CSs that poorly signaled US onset (e.g., a backward CS that followed the US), but did not eliminate freezing to forward CSs even when they predicted USs of variable intensity. Additionally, backward (but not forward) CSs selectively increased Fos in the ventral BNST and in BNST-projecting neurons in the infralimbic region of the medial prefrontal cortex (mPFC), but not in the hippocampus or amygdala. These data reveal that BNST circuits regulate fear to unpredictable threats, which may be critical to the etiology and expression of anxiety.
Collapse
Affiliation(s)
- Travis D Goode
- Department of Psychological and Brain Sciences, Institute for Neuroscience, Texas A&M University, College Station, United States
| | - Reed L Ressler
- Department of Psychological and Brain Sciences, Institute for Neuroscience, Texas A&M University, College Station, United States
| | - Gillian M Acca
- Department of Psychological and Brain Sciences, Institute for Neuroscience, Texas A&M University, College Station, United States
| | - Olivia W Miles
- Department of Psychological and Brain Sciences, Institute for Neuroscience, Texas A&M University, College Station, United States
| | - Stephen Maren
- Department of Psychological and Brain Sciences, Institute for Neuroscience, Texas A&M University, College Station, United States
| |
Collapse
|