1
|
Thrailkill EA, Daniels CW. The temporal structure of goal-directed and habitual operant behavior. J Exp Anal Behav 2024; 121:38-51. [PMID: 38131488 PMCID: PMC10872308 DOI: 10.1002/jeab.896] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/15/2023] [Accepted: 12/01/2023] [Indexed: 12/23/2023]
Abstract
Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.
Collapse
|
2
|
Daniels CW, Balsam PD. Prior experience modifies acquisition trajectories via response-strategy sampling. Anim Cogn 2023; 26:1217-1239. [PMID: 37036556 PMCID: PMC11034823 DOI: 10.1007/s10071-023-01769-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2020] [Revised: 03/05/2023] [Accepted: 03/22/2023] [Indexed: 04/11/2023]
Abstract
Few studies have considered how signal detection parameters evolve during acquisition periods. We addressed this gap by training mice with differential prior experience in a conditional discrimination, auditory signal detection task. Naïve mice, mice given separate experience with each of the later correct choice options (Correct Choice Response Transfer, CCRT), and mice experienced in conditional discriminations (Conditional Discrimination Transfer, CDT) were trained to detect the presence or absence of a tone in white noise. We analyzed data assuming a two-period model of acquisition: a pre-solution and solution period (Heinemann EG (1983) in The Presolution period and the detection of statistical associations. In: Quantitative analyses of behavior: discrimination processes, vol. 4, pp. 21-36). Ballinger. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.536.1978andrep=rep1andtype=pdf ). The pre-solution period was characterized by a selective sampling of biased response strategies until adoption of a conditional responding strategy in the solution period. Correspondingly, discriminability remained low until the solution period; criterion took excursions reflecting response-strategy sampling. Prior experience affected the length and composition of the pre-solution period. Whereas CCRT and CDT mice had shorter pre-solution periods than naïve mice, CDT and Naïve mice developed substantial criterion biases and acquired asymptotic discriminability faster than CCRT mice. To explain these data, we propose a learning model in which mice selectively sample and test different response-strategies and corresponding task structures until they exit the pre-solution period. Upon exit, mice adopt the conditional responding strategy and task structure, with action values updated via inference and generalization from the other task structures. Simulations of representative mouse data illustrate the viability of this model.
Collapse
Affiliation(s)
- Carter W Daniels
- Department of Psychiatry, Columbia University, New York, USA.
- New York State Psychiatric Institute, New York, USA.
| | - Peter D Balsam
- Department of Psychiatry, Columbia University, New York, USA
- New York State Psychiatric Institute, New York, USA
- Department of Psychology, Barnard College, New York, USA
| |
Collapse
|
3
|
Gupta TA, Daniels CW, Ortiz JB, Stephens M, Overby P, Romero K, Conrad CD, Sanabria F. The differential role of the dorsal hippocampus in initiating and terminating timed responses: A lesion study using the switch-timing task. Behav Brain Res 2019; 376:112184. [PMID: 31473282 DOI: 10.1016/j.bbr.2019.112184] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2019] [Revised: 08/24/2019] [Accepted: 08/28/2019] [Indexed: 11/27/2022]
Abstract
This study investigated the role of the dorsal hippocampus (dHPC) in the temporal entrainment of behavior, while addressing limitations of previous evidence from peak procedure experiments. Rats were first trained on a switch-timing task in which food was obtained from one of two concurrently available levers; one lever was effective after 8 s and the other after 16 s. After performance stabilized, rats underwent either bilateral NMDA lesions of the dHPC or sham lesions. After recovery, switch-timing training resumed. In a subsequent condition, the switch-timing task was modified such that food was available after either 8 or 32 s. Although dHPC lesions had subtle and complex effects on when rats stopped seeking for food at the 8-s lever (departures), it more systematically reduced the time when rats started seeking for food at the 16-s and 32-s lever (switches). No systematic effect of dHPC lesions were observed on the coefficient of quartile variation (normalized dispersion) of latencies to switch. Within the context of the pacemaker-accumulator framework of interval timing, these findings suggest that partially or wholly independent mechanisms control the initiation and termination of timed responses, and that the dHPC is primarily involved in encoding the time to start responding.
Collapse
Affiliation(s)
- Tanya A Gupta
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| | - Carter W Daniels
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; Columbia University, Department of Psychiatry, 1051 Riverside Drive, New York, NY, 10032, USA.
| | - J Bryce Ortiz
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; The University of Arizona, College of Medicine - Phoenix, 475 N. 5th Street, Phoenix, AZ, 85004, USA.
| | - McAllister Stephens
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; The University of Kentucky, Department of Psychology, 106-B Kastle Hall, Lexington, KY 40506-0044.
| | - Paula Overby
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| | - Korinna Romero
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA; Arizona State University, College of Health Solutions, 550 N. 3rd Street, Phoenix, AZ, 85004-0698, USA.
| | - Cheryl D Conrad
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| | - Federico Sanabria
- Arizona State University, Department of Psychology, P.O. Box 871104, Tempe, AZ, 85287-1104, USA.
| |
Collapse
|
4
|
Sanabria F, Daniels CW, Gupta T, Santos C. A computational formulation of the behavior systems account of the temporal organization of motivated behavior. Behav Processes 2019; 169:103952. [PMID: 31543283 PMCID: PMC6907728 DOI: 10.1016/j.beproc.2019.103952] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2019] [Revised: 08/30/2019] [Accepted: 08/31/2019] [Indexed: 02/02/2023]
Abstract
The behavior systems framework suggests that motivated behavior-e.g., seeking food and mates, avoiding predators-consists of sequences of actions organized within nested behavioral states. This framework has bridged behavioral ecology and experimental psychology, providing key insights into critical behavioral processes. In particular, the behavior systems framework entails a particular organization of behavior over time. The present paper examines whether such organization emerges from a generic Markov process, where the current behavioral state determines the probability distribution of subsequent behavioral states. This proposition is developed as a systematic examination of increasingly complex Markov models, seeking a computational formulation that balances adherence to the behavior systems approach, parsimony, and conformity to data. As a result of this exercise, a nonstationary partially hidden Markov model is selected as a computational formulation of the predatory subsystem. It is noted that the temporal distribution of discrete responses may further unveil the structure and parameters of the model but, without proper mathematical modeling, these discrete responses may be misleading. Opportunities for further elaboration of the proposed computational formulation are identified, including developments in its architecture, extensions to defensive and reproductive subsystems, and methodological refinements.
Collapse
Affiliation(s)
| | - Carter W Daniels
- Arizona State University, United States; Columbia University, United States
| | | | | |
Collapse
|