Defining stimulus representation in stimulus-response associations formed on the basis of task execution and verbal codes

Transfer of cognitive control adjustments within and between speakers

Congruency effects in conflict tasks are typically larger after congruent compared to incongruent trials. This congruency sequence effect (CSE) indicates that top-down adjustments of cognitive control transfer between processing episodes, at least when controlling for bottom-up memory processes by alternating between stimulus-response (S-R) sets in confound-minimised designs. According to the control-retrieval account, cognitive control is bound to task-irrelevant context features (e.g., stimulus position or modality) and retrieved upon subsequent context feature repetitions. A confound-minimised CSE should therefore be larger when context features repeat rather than change between two trials. This study tested this prediction for a more abstract contextual stimulus feature, speaker gender. In two preregistered auditory prime-probe task experiments, participants classified colour words spoken by a female or male voice. Across both experiments, we found confound-minimised CSEs that were not reliably affected by whether the speaker gender repeated or changed. This indicates that speaker transitions have virtually no influence on the transfer of control adjustments in the absence of S-R repetitions. By contrast, when allowing for bottom-up memory processes by repeating the S-R set, CSEs were consistently larger when the speaker gender repeated compared to changed. This suggests that speaker transitions can in principle influence transfer between processing episodes. The discrepancy also held true when considering learning and test episodes separated by an intervening episode. Thus, the present findings call for a refinement of the control-retrieval account to accommodate the role of more abstract contextual stimulus features for the maintenance of memory traces in auditory conflict processing.

One-shot stimulus-control associations generalize over different stimulus viewpoints and exemplars

Cognitive control processes are central to adaptive behavior, but how control is applied in a context-appropriate manner is not fully understood. One way to produce context-sensitive control is by mnemonically linking particular control settings to specific stimuli that demanded those settings in a prior encounter. In support of this episodic reinstatement of control hypothesis, recent studies have produced evidence for the formation of stimulus-control associations in one-shot, prime-probe learning paradigms. However, since those studies employed perceptually identical stimuli across prime and probe presentations, it is not yet known how generalizable one-shot stimulus-control associations are. In the current study, we therefore probed whether associations formed between a prime object and the control process of task-switching would generalize to probe objects seen from a different viewpoint (Experiment 1), to different exemplars of the same object type (Experiment 2), and to different members of the object category (Experiment 3). We replicated prior findings of one-shot control associations for identical prime/probe stimuli. Importantly, we additionally found that these episodic control effects are expressed regardless of changes in viewpoint and exemplar, but do not seem to generalize to other category members. These findings elucidate the scope of generalization of the episodic reinstatement of cognitive control.

Evidence for a Selective Influence of Short-Term Experiences on the Retrieval of Item-Specific Long-Term Bindings

Human behavior is guided by prior experience such as bindings between stimuli and responses. Experimentally, this is evident in performance changes when features of the stimulus-response episode reoccur either in the short-term or in the long-term. So far, effects of short-term and long-term bindings are assumed to be independent from one another. In a large-scale re-analysis of eight item-specific stimulus-response priming experiments that orthogonally varied task-specific classifications and actions in the short-term (trial N-1 to trial N) and, item-specifically, in the long-term (lag of several trials), we tested this independence assumption. In detail, we tested whether short-term experiences (repetitions of classification and action features in two consecutive trials) affected the retrieval of item-specific long-term stimulus-classification (S-C) and stimulus-action (S-A) bindings as well as potential long-term C-A bindings. The retrieval of item-specific long-term S-C bindings (i.e., the size of item-specific S-C priming effects) was affected by the persisting activation of classifications from trial N-1 (short-term priming). There were no further interactions between short-term experiences and long-term bindings. These results suggest a feature-specific, selective influence of short-term priming on long-term binding retrieval (e.g., based on shared feature representations). In contrast, however, we found evidence against an influence of short-term C-A bindings on long-term binding retrieval. This finding suggests that the processes contributing to short-term priming and long-term binding retrieval are dissociable from short-term binding and retrieval processes. Our results thus inform current theories on how short-term and long-term bindings are bound and retrieved (e.g., the BRAC framework).

Binding and Retrieval of Response Durations: Subtle Evidence for Episodic Processing of Continuous Movement Features

Re-encountering a stimulus retrieves nominally relevant, categorical response features related to previous action decisions in response to this stimulus. Whether binding and retrieval extend to nominally irrelevant, metric features relating to an actual body movement is unknown, however. In two experiments, we thus tested whether repeating target or distractor stimuli across trials retrieves the irrelevant duration of spatial responses to these stimuli. We found subtle indication of such retrieval by task-relevant target stimuli, suggesting that binding and retrieval also operate on metric features of a motor response. In contrast, there was no sign of binding and retrieval of metric features for distractor stimuli. We discuss these observations regarding the representation of action episodes during action-related decision making and during actual movement initiation and control.

Instructing item-specific switch probability: expectations modulate stimulus-action priming

Both active response execution and passive listening to verbal codes (a form of instruction) in single prime trials lead to item-specific repetition priming effects when stimuli re-occur in single probe trials. This holds for task-specific classification (stimulus-classification, SC priming, e.g., apple-small) and action (stimulus-action, SA priming, e.g., apple-right key press). To address the influence of expectation on item-specific SC and SA associations, we tested if item-specific SC and SA priming effects were modulated by the instructed probability of re-encountering individual SC or SA mappings (25% vs. 75% instructed switch probability). Importantly, the experienced item-specific switch probability was always 50%. In Experiment 1 (N = 78), item-specific SA/SC switch  expectations affected SA, but not SC priming effects exclusively following active response execution. Experiment 2 (N = 40) was designed to emphasize SA priming by only including item-specific SC repetitions. This yielded stronger SA priming for 25% vs. 75% expected switch probability, both following response execution as in Experiment 1 and also following verbally coded SA associations. Together, these results suggest that SA priming effects, that is, the encoding and retrieval of SA associations, is modulated by item-specific switch expectation. Importantly, this expectation effect cannot be explained by item-specific associative learning mechanisms, as stimuli were primed and probed only once and participants experienced item-specific repetitions/switches equally often across stimuli independent of instructed switch probabilities. This corroborates and extends previous results by showing that SA priming effects are modulated by  expectation not only based on experienced item-specific switch probabilities, but also on mere instruction.

Mind wandering at encoding, but not at retrieval, disrupts one-shot stimulus-control learning

The one-shot pairing of a stimulus with a specific cognitive control process, such as task switching, can bind the two together in memory. The episodic control-binding hypothesis posits that the formation of temporary stimulus-control bindings, which are held in event-files supported by episodic memory, can guide the contextually appropriate application of cognitive control. Across two experiments, we sought to examine the role of task-focused attention in the encoding and implementation of stimulus-control bindings in episodic event-files. In Experiment 1, we obtained self-reports of mind wandering during encoding and implementation of stimulus-control bindings. Results indicated that, whereas mind wandering during the implementation of stimulus-control bindings does not decrease their efficacy, mind wandering during the encoding of these control-state associations interferes with their successful deployment at a later point. In Experiment 2, we complemented these results by using trial-by-trial pupillometry to measure attention, again demonstrating that attention levels at encoding predict the subsequent implementation of stimulus-control bindings better than attention levels at implementation. These results suggest that, although encoding stimulus-control bindings in episodic memory requires active attention and engagement, once encoded, these bindings are automatically deployed to guide behavior when the stimulus recurs. These findings expand our understanding of how cognitive control processes are integrated into episodic event files.

Study-test congruence of response levels in item stimulus-response priming

We investigated stimulus-response (S-R) memory links during object priming using a binary associative size judgement paradigm. At study, participants decided which of two objects was bigger in real life and, at test, made the same or the reverse judgement. We examined the effects of response congruence on item S-R priming in the associative paradigm. In Experiment 1, a task reversal manipulation had minimal impact on RT priming when classifications were congruent for both recombined objects between study and test. Experiment 2 found that RT priming was more disrupted by classification incongruence of the selected than of the nonselected item alone, with incongruence of the nonselected object having no effect on RTs. Experiment 3, however, found that classification incongruence of both items eliminated RT priming, indicating that a significant effect of classification incongruence for the nonselected item is only evident if both items are classification-incongruent. Finally, across all experiments, we found that accuracy was more sensitive than RTs to decision/action incongruence. We interpret these findings in light of a two-stream account of S-R priming, and suggest a few extensions to account for interactions between S-R links of recombined items.

Static internal representation of dynamic situations reveals time compaction in human cognition

Introduction

The human brain has evolved under the constraint of survival in complex dynamic situations. It makes fast and reliable decisions based on internal representations of the environment. Whereas neural mechanisms involved in the internal representation of space are becoming known, entire spatiotemporal cognition remains a challenge. Growing experimental evidence suggests that brain mechanisms devoted to spatial cognition may also participate in spatiotemporal information processing.

Objectives

The time compaction hypothesis postulates that the brain represents both static and dynamic situations as purely static maps. Such an internal reduction of the external complexity allows humans to process time-changing situations in real-time efficiently. According to time compaction, there may be a deep inner similarity between the representation of conventional static and dynamic visual stimuli. Here, we test the hypothesis and report the first experimental evidence of time compaction in humans.

Methods

We engaged human subjects in a discrimination-learning task consisting in the classification of static and dynamic visual stimuli. When there was a hidden correspondence between static and dynamic stimuli due to time compaction, the learning performance was expected to be modulated. We studied such a modulation experimentally and by a computational model.

Results

The collected data validated the predicted learning modulation and confirmed that time compaction is a salient cognitive strategy adopted by the human brain to process time-changing situations. Mathematical modelling supported the finding. We also revealed that men are more prone to exploit time compaction in accordance with the context of the hypothesis as a cognitive basis for survival.

Conclusions

The static internal representation of dynamic situations is a human cognitive mechanism involved in decision-making and strategy planning to cope with time-changing environments. The finding opens a new venue to understand how humans efficiently interact with our dynamic world and thrive in nature.

Item-Specificity and Intention in Episodic Memory

Schmidt et al.'s (2020) PEP model accurately reflects the complexity of task switching based on bottom-up assumptions and episodic memory, re-evaluating the contribution of commonly presumed top-down processes. Extending it to long-term bindings and their item-specific effects could eludicate puzzling findings regarding the independence of long-term bindings between stimuli, responses, and task-specific categorizations as well as the relation between short-term and long-term bindings. Moreover, ideomotor theories of action control provide a bottom-up basis of incorporating volition and intentional action into the PEP model which is currently restricted to stimulus-based action.

Memories of control: One-shot episodic learning of item-specific stimulus-control associations

The repeated pairing of a particular stimulus with a specific cognitive control process, such as task switching, can bind the two together in memory, resulting in the formation of stimulus-control associations. These bindings are thought to guide the context-sensitive application of cognitive control, but it is not presently known whether such stimulus-control associations are only acquired through slow, incremental learning or could also be mediated by episodic memories of a single experience, so-called one-shot learning. Here, we tested this episodic control-binding hypothesis by probing whether a single co-occurrence of a stimulus and the control process of task switching would lead to significant performance benefits (reduced task switch cost) when that stimulus later re-occurred under the same as opposed to different control demands. Across three experiments, we demonstrate that item-specific stimulus-control associations can be formed based on a single exposure, providing the first strong evidence for episodic memory guidance of cognitive control.

On the Assimilation of Instructions: Stimulus-response Associations are Implemented but not Stimulus-task Associations

The assimilation of instructions consists of two stages. First, a task model is formed on the basis of instructions. Second, this model is implemented, resulting in highly accessible representations, which enable reflexive behavior that guides the application of instructions. Research frequently demonstrated that instructions can lead to automatic response activation, which indicates that stimulus-response associations can be implemented on the basis of a task model. However, instructions not only indicate how to respond (stimulus-response mappings) but also when (i.e., the conditions under which mappings apply). Accordingly, we tested whether instruction implementation leads both to the activation of stimulus-response associations and of associations between stimuli and the context or task in which the instructed stimulus-response mappings are relevant (i.e., stimulus-task associations). In four experiments, we measured if implementing newly instructed stimulus-response mappings also leads to bivalence costs (i.e., shorter latencies when a stimulus can only occur in one task compared to when it can occur in two tasks), which indicate the presence of stimulus-task associations. We consistently observed automatic response activation on the basis of instructions, but no bivalence costs. A discrepancy thus exists between information conveyed in an instructed task model and the elements of that task model that are implemented. We propose that future research on automatic effects of instructions should broaden its scope and focus both on the formation of an instructed task model and its subsequent implementation.

Execution-based and verbal code-based stimulus-response associations: proportion manipulations reveal conflict adaptation processes in item-specific priming

Stimulus-response (S-R) associations consist of two independent components: Stimulus-classification (S-C) and stimulus-action (S-A) associations. Here, we examined whether these S-C and S-A associations were modulated by cognitive control operations. In two item-specific priming experiments, we systematically manipulated the proportion of trials in which item-specific S-C and/or S-A mappings repeated or switched between the single encoding (prime) and single retrieval (probe) instance of each stimulus (i.e., each stimulus appeared only twice). Thus, we assessed the influence of a list-level proportion switch manipulation on the strength of item-specific S-C and S-A associations. Participants responded slower and committed more errors when item-specific S-C or S-A mappings switched rather than repeated between prime and probe (i.e., S-C/S-A switch effects). S-C switch effects were larger when S-C repetitions rather than switches were frequent on the list-level. Similarly, S-A switch effects were modulated by S-A switch proportion. Most importantly, our findings rule out contingency learning and temporal learning as explanations of the observed results and point towards a conflict adaptation mechanism that selectively adapts the encoding and/or retrieval for each S-R component. Finally, we outline how cognitive control over S-R associations operates in the context of item-specific priming.

Instructed and Acquired Contingencies in Response-Inhibition Tasks

Inhibitory control can be triggered directly via the retrieval of previously acquired stimulus-stop associations from memory. However, a recent study suggests that this item-specific stop learning may be mediated via expectancies of the contingencies in play (Best, Lawrence, Logan, McLaren, & Verbruggen, 2016). This could indicate that stimulus-stop learning also induces strategic proactive changes in performance. We further tested this hypothesis in the present study. In addition to measuring expectancies following task completion, we introduced a between-subjects expectancy manipulation in which one group of participants were informed about the stimulus-stop contingencies and another group did not receive any information about the stimulus-stop contingencies. Moreover, we combined this instruction manipulation with a distractor manipulation that was previously used to examine strategic proactive adjustments. We found that the stop-associated items slowed responding in both conditions. Furthermore, participants in both conditions generated expectancies following task completion that were consistent with the stimulus-stop contingencies. The distractor manipulation was ineffective. However, we found differences in the relationship between the expectancy ratings and task performance: in the instructed condition, the expectancies reliably correlated with the response slowing for the stop-associated items, whereas in the uninstructed condition we found no reliable correlation. These differences between the correlations were reliable, and our conclusions were further supported by Bayesian analyses. We conclude that stimulus-stop associations that are acquired either via task instructions or via task practice have similar effects on behaviour but could differ in how they elicit response slowing.

Learning in the absence of overt practice: a novel (previously unseen) stimulus can trigger retrieval of an unpracticed response

Skilled performance is traditionally thought to develop via overt practice. Recent research has demonstrated that merely instructed stimulus-response (S-R) bindings can influence later performance and readily transfer across response modalities. In the present study, we extended this to include instructed category-response (C-R) associations. That is, we investigated whether merely instructed C-R bindings can trigger an unpracticed response (in a different modality) on perception of a novel (previously unseen) stimulus. In a learning-test design, participants had to classify stimuli by comparing them to perceptual category templates (Experiment 1) or semantic category descriptions (Experiment 2) presented prior to each block. During learning blocks, participants had to respond manually, respond vocally, or listen passively to the correct response being spoken. A manual response was always required at test. In test blocks, the categories could either be novel or repeated from the learning block, whereas half of the stimuli were always novel and half were always repeated from the learning block. Because stimulus and category repetitions were manipulated orthogonally, it was possible to directly compare the relative contribution of S-R and C-R associations to performance. In Experiment 1, test performance was enhanced by repeating the C-R bindings independently of the stimulus. In Experiment 2, there was also evidence of an S-R repetition benefit independent of the classification. Critically, instructed associations formed in one response modality were robust to changes in the required response, even when no overt response was required during training, indicating the need to update the traditional view of associative learning.