Doing Without Schema Hierarchies: A Recurrent Connectionist Approach to Normal and Impaired Routine Sequential Action

Using network science to provide insights into the structure of event knowledge

The structure of event knowledge plays a critical role in prediction, reconstruction of memory for personal events, construction of possible future events, action, language usage, and social interactions. Despite numerous theoretical proposals such as scripts, schemas, and stories, the highly variable and rich nature of events and event knowledge have been formidable barriers to characterizing the structure of event knowledge in memory. We used network science to provide insights into the temporal structure of common events. Based on participants' production and ordering of the activities that make up events, we established empirical profiles for 80 common events to characterize the temporal structure of activities. We used the event networks to investigate multiple issues regarding the variability in the richness and complexity of people's knowledge of common events, including: the temporal structure of events; event prototypes that might emerge from learning across many experiential instances and be expressed by people; the degree to which scenes (communities) are present in various events; the degree to which people believe certain activities are central to an event; how centrality might be distributed across an event's activities; and similarities among events in terms of their content and their temporal structure. Thus, we provide novel insights into human event knowledge, and describe 18 predictions for future human studies.

Task-dependent coarticulation of movement sequences

Combining individual actions into sequences is a hallmark of everyday activities. Classical theories propose that the motor system forms a single specification of the sequence as a whole, leading to the coarticulation of the different elements. In contrast, recent neural recordings challenge this idea and suggest independent execution of each element specified separately. Here, we show that separate or coarticulated sequences can result from the same task-dependent controller, without implying different representations in the brain. Simulations show that planning for multiple reaches simultaneously allows separate or coarticulated sequences depending on instructions about intermediate goals. Human experiments in a two-reach sequence task validated this model. Furthermore, in co-articulated sequences, the second goal influenced long-latency stretch responses to external loads applied during the first reach, demonstrating the involvement of the sensorimotor network supporting fast feedback control. Overall, our study establishes a computational framework for sequence production that highlights the importance of feedback control in this essential motor skill.

Updating predictions in a complex repertoire of actions and its neural representation

Even though actions we observe in everyday life seem to unfold in a continuous manner, they are automatically divided into meaningful chunks, that are single actions or segments, which provide information for the formation and updating of internal predictive models. Specifically, boundaries between actions constitute a hub for predictive processing since the prediction of the current action comes to an end and calls for updating of predictions for the next action. In the current study, we investigated neural processes which characterize such boundaries using a repertoire of complex action sequences with a predefined probabilistic structure. Action sequences consisted of actions that started with the hand touching an object (T) and ended with the hand releasing the object (U). These action boundaries were determined using an automatic computer vision algorithm. Participants trained all action sequences by imitating demo videos. Subsequently, they returned for an fMRI session during which the original action sequences were presented in addition to slightly modified versions thereof. Participants completed a post-fMRI memory test to assess the retention of original action sequences. The exchange of individual actions, and thus a violation of action prediction, resulted in increased activation of the action observation network and the anterior insula. At U events, marking the end of an action, increased brain activation in supplementary motor area, striatum, and lingual gyrus was indicative of the retrieval of the previously encoded action repertoire. As expected, brain activation at U events also reflected the predefined probabilistic branching structure of the action repertoire. At T events, marking the beginning of the next action, midline and hippocampal regions were recruited, reflecting the selected prediction of the unfolding action segment. In conclusion, our findings contribute to a better understanding of the various cerebral processes characterizing prediction during the observation of complex action repertoires.

Causal computations of supplementary motor area on spatial impulsivity

Spatial proximity to important stimuli often induces impulsive behaviour. How we overcome impulsive tendencies is what determines behaviour to be adaptive. Here, we used virtual reality to investigate whether the spatial proximity of stimuli is causally related to the supplementary motor area (SMA) functions. In two experiments, we set out to investigate these processes using a virtual environment that recreates close and distant spaces to test the causal contributions of the SMA in spatial impulsivity. In an online first experiment (N = 93) we validated and measured the influence of distant stimuli using a go/no-go task with close (21 cm) or distant stimuli (360 cm). In experiment 2 (N = 28), we applied transcranial static magnetic stimulation (tSMS) over the SMA (double-blind, crossover, sham-controlled design) to test its computations in controlling impulsive tendencies towards close vs distant stimuli. Reaction times and error rates (omission and commission) were analysed. In addition, the EZ Model parameters (a, v, Ter and MDT) were computed. Close stimuli elicited faster responses compared to distant stimuli but also exhibited higher error rates, specifically in commission errors (experiment 1). Real stimulation over SMA slowed response latencies (experiment 2), an effect mediated by an increase in decision thresholds (a). Current findings suggest that impulsivity might be modulated by spatial proximity, resulting in accelerated actions that may lead to an increase of inaccurate responses to nearby objects. Our study also provides a first starting point on the role of the SMA in regulating spatial impulsivity.

Cognitive neuropsychological and neuroanatomic predictors of naturalistic action performance in left hemisphere stroke: a retrospective analysis

Individuals who have experienced a left hemisphere cerebrovascular accident (LCVA) have been shown to make errors in naturalistic action tasks designed to assess the ability to perform everyday activities such as preparing a cup of coffee. Naturalistic action errors in this population are often attributed to limb apraxia, a common deficit in the representation and performance of object-related actions. However, naturalistic action impairments are also observed in right hemisphere stroke and traumatic brain injury, populations infrequently associated with apraxia, and errors across all these populations are influenced by overall severity. Based on these and other data, an alternative (though not mutually exclusive) account is that naturalistic action errors in LCVA are also a consequence of deficits in general attentional resource availability or allocation. In this study, we conducted a retrospective analysis of data from a large group of 51 individuals with LCVA who had completed a test of naturalistic action, along with a battery of tests assessing praxis, attention allocation and control, reasoning, and language abilities to determine which of these capacities contribute uniquely to naturalistic action impairments. Using a regularized regression method, we found that naturalistic action impairments are predicted by both praxis deficits (hand posture sequencing and gesture recognition), as well as attention allocation and control deficits (orienting and dividing attention), along with language comprehension ability and age. Using support vector regression-lesion symptom mapping (SVR-LSM), we also demonstrated that naturalistic action impairments are associated with lesions to posterior middle temporal gyrus and anterior inferior parietal lobule - regions known to be implicated in praxis; as well the middle frontal gyrus that has been implicated in both praxis and attention allocation and control. Taken together, these findings support the hypothesis that naturalistic action impairments in LCVA are a consequence of apraxia as well as deficits in attention allocation and control.

When knowing the activity is not enough to predict gaze

It is reasonable to assume that where people look in the world is largely determined by what they are doing. The reasoning is that the activity determines where it is useful to look at each moment in time. Assuming that it is vital to accurately judge the positions of the steps when navigating a staircase, it is surprising that people differ a lot in the extent to which they look at the steps. Apparently, some people consider the accuracy of peripheral vision, predictability of the step size, and feeling the edges of the steps with their feet to be good enough. If so, occluding part of the view of the staircase and making it more important to place one's feet gently might make it more beneficial to look directly at the steps before stepping onto them, so that people will more consistently look at many steps. We tested this idea by asking people to walk on staircases, either with or without a tray with two cups of water on it. When carrying the tray, people walked more slowly, but they shifted their gaze across steps in much the same way as they did when walking without the tray. They did not look at more steps. There was a clear positive correlation between the fraction of steps that people looked at when walking with and without the tray. Thus, the variability in the extent to which people look at the steps persists when one makes walking on the staircase more challenging.

Toward the Emergence of Intelligent Control: Episodic Generalization and Optimization

Human cognition is unique in its ability to perform a wide range of tasks and to learn new tasks quickly. Both abilities have long been associated with the acquisition of knowledge that can generalize across tasks and the flexible use of that knowledge to execute goal-directed behavior. We investigate how this emerges in a neural network by describing and testing the Episodic Generalization and Optimization (EGO) framework. The framework consists of an episodic memory module, which rapidly learns relationships between stimuli; a semantic pathway, which more slowly learns how stimuli map to responses; and a recurrent context module, which maintains a representation of task-relevant context information, integrates this over time, and uses it both to recall context-relevant memories (in episodic memory) and to bias processing in favor of context-relevant features and responses (in the semantic pathway). We use the framework to address empirical phenomena across reinforcement learning, event segmentation, and category learning, showing in simulations that the same set of underlying mechanisms accounts for human performance in all three domains. The results demonstrate how the components of the EGO framework can efficiently learn knowledge that can be flexibly generalized across tasks, furthering our understanding of how humans can quickly learn how to perform a wide range of tasks-a capability that is fundamental to human intelligence.

Action Selection in Everyday Activities: The Opportunistic Planning Model

While action selection strategies in well-defined domains have received considerable attention, little is yet known about how people choose what to do next in ill-defined tasks. In this contribution, we shed light on this issue by considering everyday tasks, which in many cases have a multitude of possible solutions (e.g., it does not matter in which order the items are brought to the table when setting a table) and are thus categorized as ill-defined problems. Even if there are no hard constraints on the ordering of subtasks in everyday activities, our research shows that people exhibit specific preferences. We propose that these preferences arise from bounded rationality, that is, people only have limited knowledge and processing power available, which results in a preference to minimize the overall physical and cognitive effort. In the context of everyday activities, this can be achieved by (a) taking properties of the spatial environment into account to use them to one's advantage, and (b) employing a stepwise-optimal action selection strategy. We present the Opportunistic Planning Model as an explanatory cognitive model, which instantiates these assumptions, and show that the model is able to generalize to new everyday tasks, outperforming machine learning models such as neural networks during generalization.

Hierarchical organization of human physical activity

Human physical activity (HPA), a fundamental physiological signal characteristic of bodily motion is of rapidly growing interest in multidisciplinary research. Here we report the existence of hitherto unidentified hierarchical levels in the temporal organization of HPA on the ultradian scale: on the minute's scale, passive periods are followed by activity bursts of similar intensity ('quanta') that are organized into superstructures on the hours- and on the daily scale. The time course of HPA can be considered a stochastic, quasi-binary process, where quanta, assigned to task-oriented actions are organized into work packages on higher levels of hierarchy. In order to grasp the essence of this complex dynamic behaviour, we established a stochastic mathematical model which could reproduce the main statistical features of real activity time series. The results are expected to provide important data for developing novel behavioural models and advancing the diagnostics of neurological or psychiatric diseases.

The right face at the wrong place: How motor intentions can override outcome monitoring

The concept of intentions is often taken for granted in the cognitive and neural sciences, and comparing outcomes with internal goals is seen as critical for our sense of agency. We created an experiment where participants decided which face they preferred, and we either created outcome errors by covertly switching the position of the chosen face or induced motor errors by deviating the mouse cursor, or we did both at the same time. In the final case, participants experienced a motor error, but the outcome ended up correct. The result showed that when they received the right face, but at the wrong place, participants rejected the outcome they actually wanted in a majority of the trials. Thus, contrary to common belief, higher-order outcomes do not always regulate our actions. Instead, motor "wrongness" might sometimes override goal "rightness" and lead us to reject the outcome we actually want.

Rapid learning of temporal dependencies at multiple timescales

Our environment contains temporal information unfolding simultaneously at multiple timescales. How do we learn and represent these dynamic and overlapping information streams? We investigated these processes in a statistical learning paradigm with simultaneous short and long timescale contingencies. Human participants (N=96) played a game where they learned to quickly click on a target image when it appeared in one of 9 locations, in 8 different contexts. Across contexts, we manipulated the order of target locations: at a short timescale, the order of pairs of sequential locations in which the target appeared; at a longer timescale, the set of locations that appeared in the first vs. second half of the game. Participants periodically predicted the upcoming target location, and later performed similarity judgements comparing the games based on their order properties. Participants showed context dependent sensitivity to order information at both short and long timescales, with evidence of stronger learning for short timescales. We modeled the learning paradigm using a gated recurrent network trained to make immediate predictions, which demonstrated multilevel learning timecourses and patterns of sensitivity to the similarity structure of the games that mirrored human participants. The model grouped games with matching rule structure and dissociated games based on low-level order information more so than high-level order information. The work shows how humans and models can rapidly and concurrently acquire order information at different timescales.

Measuring context-response associations that drive habits

People achieve important life outcomes of health, financial security, and productivity by repeating operant behavior. To identify whether such operants reflect goal pursuit or habit, the present research introduces a new paradigm that yields objective measures of learning and controls for the motivations of goal pursuit. In two experiments, participants practiced a sequential task of making sushi and then completed a test of the strength of cue-response (habit) associations in memory. Finally, they repeated the sushi task without instructions while under cognitive load (designed to impede deliberation about goals). As predicted, greater task practice yielded stronger cue-response associations, which in turn promoted task success. Practice did not improve performance by enhancing goal intentions or other task motivations. We conclude that repetition facilitates performance by creating mental associations that automatically activate practiced, habitual responses upon perception of recurring context cues.

The effect of abstract inter-chunk relationships on serial-order control

Hierarchical control is often thought to dissect a complex task space into isolated subspaces in order to eliminate interference. Yet, there is also evidence from serial-order control tasks that our cognitive system can make use of abstract relationships between different parts (chunks) of a sequence. Past evidence in this regard was limited to situations with ordered stimuli (e.g., numbers or positions) that may have aided the detection of relationships and allowed gradual learning and hypothesis testing. Therefore, we used a modified task-span paradigm (with no ordered elements between tasks) in which participants performed memorized sequences of tasks that were encoded in terms of separate chunks of three tasks each. To allow examination of learning effects, each sequence was "cycled" through repeatedly. Importantly, we compared sequences whose chunks were governed by a common, abstract grammar with sequences whose chunks were governed by different grammars. Experiment 1 examined the effect of relationships between shared-element chunks (e.g., ABB-BAA vs. ABB-BAB), Experiment 2 and 3 between different-element chunks (e.g., ABA-CDC vs. ABA-CCD), and Experiment 4 examined second-order relationships (e.g., ABA-ABB--CDC-CDD vs. ABA-ABB--CDC-CCD). Robust evidence in favor of beneficial effects of abstract inter-chunk relationships was obtained across all four experiments. Importantly, these effects were at least as strong in initial cycles of performing a given sequence as during later cycles, suggesting that the cognitive system operates with an "expectation of abstract relationships," rather than benefitting from them through gradual learning. We discuss the implications of these results for models of hierarchical control.

Differentiated, rather than shared, strategies for time-coordinated action in social and non-social domains in autistic individuals

Autism spectrum disorder (ASD) is a neurodevelopmental condition with a highly heterogeneous adult phenotype that includes social and non-social behavioral characteristics. The link between the characteristics assignable to the different domains remains unresolved. One possibility is that social and non-social behaviors in autism are modulated by a common underlying deficit. However, here we report evidence supporting an alternative concept that is individual-centered rather than deficit-centered. Individuals are assumed to have a distinctive style in the strategies they adopt to perform social and non-social tasks with these styles presumably being structured differently between autistic individuals and typically-developed (TD) individuals. We tested this hypothesis for the execution of time-coordinated (synchronized) actions. Participants performed (i) a social task that required synchronized gaze and pointing actions to interact with another person, and (ii) a non-social task that required finger-tapping actions synchronized to periodic stimuli at different time-scales and sensory modalities. In both tasks, synchronization behavior differed between ASD and TD groups. However, a principal component analysis of individual behaviors across tasks revealed associations between social and non-social features for the TD persons but such cross-domain associations were strikingly absent for autistic individuals. The highly differentiated strategies between domains in ASD are inconsistent with a general synchronization deficit and instead highlight the individualized developmental heterogeneity in the acquisition of domain-specific behaviors. We propose a cognitive model to help disentangle individual-centered from deficit-centered effects in other domains. Our findings reinforce the importance to identify individually differentiated phenotypes to personalize autism therapies.

Tool mastering today - an interdisciplinary perspective

Tools have coined human life, living conditions, and culture. Recognizing the cognitive architecture underlying tool use would allow us to comprehend its evolution, development, and physiological basis. However, the cognitive underpinnings of tool mastering remain little understood in spite of long-time research in neuroscientific, psychological, behavioral and technological fields. Moreover, the recent transition of tool use to the digital domain poses new challenges for explaining the underlying processes. In this interdisciplinary review, we propose three building blocks of tool mastering: (A) perceptual and motor abilities integrate to tool manipulation knowledge, (B) perceptual and cognitive abilities to functional tool knowledge, and (C) motor and cognitive abilities to means-end knowledge about tool use. This framework allows for integrating and structuring research findings and theoretical assumptions regarding the functional architecture of tool mastering via behavior in humans and non-human primates, brain networks, as well as computational and robotic models. An interdisciplinary perspective also helps to identify open questions and to inspire innovative research approaches. The framework can be applied to studies on the transition from classical to modern, non-mechanical tools and from analogue to digital user-tool interactions in virtual reality, which come with increased functional opacity and sensorimotor decoupling between tool user, tool, and target. By working towards an integrative theory on the cognitive architecture of the use of tools and technological assistants, this review aims at stimulating future interdisciplinary research avenues.

Hysteresis in motor and language production

Hysteresis in motor planning and syntactic priming in language planning refer to the influence of prior production history on current production behaviour. Computational efficiency accounts of action hysteresis and theoretical accounts of syntactic priming both argue that reusing an existing plan is less costly than generating a novel plan. Despite these similarities across motor and language frameworks, research on planning in these domains has largely been conducted independently. The current study adapted an existing language paradigm to mirror the incremental nature of a manual motor task to investigate the presence of parallel hysteresis effects across domains. We observed asymmetries in production choice for both the motor and language tasks that resulted from the influence of prior history. Furthermore, these hysteresis effects were more exaggerated for subordinate production forms implicating an inverse preference effect that spanned domain. Consistent with computational efficiency accounts, across both task participants exhibited reaction time savings on trials in which they reused a recent production choice. Together, these findings lend support to the broader notion that there are common production biases that span both motor and language domains.

Dual-axes of functional organisation across lateral parietal cortex: the angular gyrus forms part of a multi-modal buffering system

Decades of neuropsychological and neuroimaging evidence have implicated the lateral parietal cortex (LPC) in a myriad of cognitive domains, generating numerous influential theoretical models. However, these theories fail to explain why distinct cognitive activities appear to implicate common neural regions. Here we discuss a unifying model in which the angular gyrus forms part of a wider LPC system with a core underlying neurocomputational function; the multi-sensory buffering of spatio-temporally extended representations. We review the principles derived from computational modelling with neuroimaging task data and functional and structural connectivity measures that underpin the unified neurocomputational framework. We propose that although a variety of cognitive activities might draw on shared underlying machinery, variations in task preference across angular gyrus, and wider LPC, arise from graded changes in the underlying structural connectivity of the region to different input/output information sources. More specifically, we propose two primary axes of organisation: a dorsal-ventral axis and an anterior-posterior axis, with variations in task preference arising from underlying connectivity to different core cognitive networks (e.g. the executive, language, visual, or episodic memory networks).

The convergence and divergence of episodic and semantic functions across lateral parietal cortex

Decades of research have highlighted the importance of lateral parietal cortex (LPC) across a myriad of cognitive domains. Yet, the underlying function of LPC remains unclear. Two domains that have emphasized LPC involvement are semantic memory and episodic memory retrieval. From each domain, sophisticated functional models have been proposed, as well as the more domain-general assumption that LPC is engaged by any form of internally directed cognition (episodic/semantic retrieval being examples). Here we used a combination of functional magnetic resonance imaging, functional connectivity, and diffusion tensor imaging white-matter connectivity to show that (i) ventral LPC (angular gyrus [AG]) was positively engaged during episodic retrieval but disengaged during semantic memory retrieval and (ii) activity negatively varied with task difficulty in the semantic task whereas episodic activation was independent of difficulty. In contrast, dorsal LPC (intraparietal sulcus) showed domain general activation that was positively correlated with task difficulty. Finally, (iii) a dorsal-ventral and anterior-posterior gradient of functional and structural connectivity was found across the AG (e.g. mid-AG connected with episodic retrieval). We propose a unifying model in which LPC as a whole might share a common underlying neurocomputation (multimodal buffering) with variations in the emergent cognitive functions across subregions arising from differences in the underlying connectivity.

Understanding the multidimensional cognitive deficits of logopenic variant primary progressive aphasia

The logopenic variant of primary progressive aphasia is characterized by early deficits in language production and phonological short-term memory, attributed to left-lateralized temporoparietal, inferior parietal and posterior temporal neurodegeneration. Despite patients primarily complaining of language difficulties, emerging evidence points to performance deficits in non-linguistic domains. Temporoparietal cortex, and functional brain networks anchored to this region, are implicated as putative neural substrates of non-linguistic cognitive deficits in logopenic variant primary progressive aphasia, suggesting that degeneration of a shared set of brain regions may result in co-occurring linguistic and non-linguistic dysfunction early in the disease course. Here, we provide a Review aimed at broadening the understanding of logopenic variant primary progressive aphasia beyond the lens of an exclusive language disorder. By considering behavioural and neuroimaging research on non-linguistic dysfunction in logopenic variant primary progressive aphasia, we propose that a significant portion of multidimensional cognitive features can be explained by degeneration of temporal/inferior parietal cortices and connected regions. Drawing on insights from normative cognitive neuroscience, we propose that these regions underpin a combination of domain-general and domain-selective cognitive processes, whose disruption results in multifaceted cognitive deficits including aphasia. This account explains the common emergence of linguistic and non-linguistic cognitive difficulties in logopenic variant primary progressive aphasia, and predicts phenotypic diversification associated with progression of pathology in posterior neocortex.

A weighted constraint satisfaction approach to human goal-directed decision making

When we plan for long-range goals, proximal information cannot be exploited in a blindly myopic way, as relevant future information must also be considered. But when a subgoal must be resolved first, irrelevant future information should not interfere with the processing of more proximal, subgoal-relevant information. We explore the idea that decision making in both situations relies on the flexible modulation of the degree to which different pieces of information under consideration are weighted, rather than explicitly decomposing a problem into smaller parts and solving each part independently. We asked participants to find the shortest goal-reaching paths in mazes and modeled their initial path choices as a noisy, weighted information integration process. In a base task where choosing the optimal initial path required weighting starting-point and goal-proximal factors equally, participants did take both constraints into account, with participants who made more accurate choices tending to exhibit more balanced weighting. The base task was then embedded as an initial subtask in a larger maze, where the same two factors constrained the optimal path to a subgoal, and the final goal position was irrelevant to the initial path choice. In this more complex task, participants’ choices reflected predominant consideration of the subgoal-relevant constraints, but also some influence of the initially-irrelevant final goal. More accurate participants placed much less weight on the optimality-irrelevant goal and again tended to weight the two initially-relevant constraints more equally. These findings suggest that humans may rely on a graded, task-sensitive weighting of multiple constraints to generate approximately optimal decision outcomes in both hierarchical and non-hierarchical goal-directed tasks.

Different problems require the consideration of different information sources, including often useful long-range, future information that may impact our immediate decisions. However, when future information is irrelevant to a key subgoal, it can be desirable to focus on achieving the subgoal first. We suggest that humans rely on appropriately weighting relevant information over irrelevant information to generate decision outcomes in both types of situations. We conducted behavioral experiments and fitted models of decision processes to understand to what extent people considered various task factors in choosing the initial path in different mazes, both when a simple maze occurred alone or was embedded as an initial part in a larger maze. Our results show that people approximate the optimal decision outcomes in both tasks by modulating the weighting of different factors during planning, and that people who made more accurate initial path choices modulated these weightings more successfully than those who made less accurate choices.

Context Stability in Habit Building Increases Automaticity and Goal Attainment

In this paper, we investigate the effects of context stability on automaticity and goal attainment in intentional habit building. We used hierarchical growth curve modeling and multilevel mediation to test our hypotheses on two datasets. In Study 1, N = 95 university students (N = 2,482 habit repetitions) built new study habits over a period of 6 weeks with manipulated context stability. One group was instructed to constantly vary the context of their habit repetitions by changing rooms and times and the other group was instructed to keep the context of habit performance stable. In Study 2, N = 308 habits (N = 2,368 habit repetitions) from N = 218 users of a published habit building app were analyzed without manipulating but measuring context stability. We found the same pattern in both datasets: Context stability predicted more automaticity and higher habit repetition goal attainment. We also found that the effect of context stability on habit repetition goal attainment was partially mediated by automaticity in both datasets. These results show that context does not only act as a trigger for habit instigation but also has an ongoing effect on habit execution.

A Schema-Based Instructional Design Model for Self-Paced Learning Environments

Although research on schema has been widely investigated for the past decades, little research has addressed the development of a systematic instructional design theory using schema principles and processes. This study proposes a systematic schema-based instructional design model, including general and schema analysis, schema-based design, and development processes and techniques for evaluating a learner’s acquired schema. By synthesizing empirical studies, this study comprehensively reviews literature on schema and foundational principles for learning. The goal of the study is to enrich the knowledge base of schema-based instructional design for different learning environments. Thus, the study is concluded by a discussion on how to utilize a schema-based instructional design for self-paced learning environments with additional implications and further recommendations.

A neural network model of when to retrieve and encode episodic memories

Recent human behavioral and neuroimaging results suggest that people are selective in when they encode and retrieve episodic memories. To explain these findings, we trained a memory-augmented neural network to use its episodic memory to support prediction of upcoming states in an environment where past situations sometimes reoccur. We found that the network learned to retrieve selectively as a function of several factors, including its uncertainty about the upcoming state. Additionally, we found that selectively encoding episodic memories at the end of an event (but not mid-event) led to better subsequent prediction performance. In all of these cases, the benefits of selective retrieval and encoding can be explained in terms of reducing the risk of retrieving irrelevant memories. Overall, these modeling results provide a resource-rational account of why episodic retrieval and encoding should be selective and lead to several testable predictions.

A Critical Period for Robust Curriculum-Based Deep Reinforcement Learning of Sequential Action in a Robot Arm

Many everyday activities are sequential in nature. That is, they can be seen as a sequence of subactions and sometimes subgoals. In the motor execution of sequential action, context effects are observed in which later subactions modulate the execution of earlier subactions (e.g., reaching for an overturned mug, people will optimize their grasp to achieve a comfortable end state). A trajectory (movement) adaptation of an often‐used paradigm in the study of sequential action, the serial response time task, showed several context effects of which centering behavior is of special interest. Centering behavior refers to the tendency (or strategy) of subjects to move their arm or mouse cursor to a position equidistant to all stimuli in the absence of predictive information, thereby reducing movement time to all possible targets. In the current study, we investigated sequential action in a virtual robotic agent trained using proximal policy optimization, a state‐of‐the‐art deep reinforcement learning algorithm. The agent was trained to reach for appearing targets, similar to a serial response time task given to humans. We found that agents were more likely to develop centering behavior similar to human subjects after curricularized learning. In our curriculum, we first rewarded agents for reaching targets before introducing a penalty for energy expenditure. When the penalty was applied with no curriculum, many agents failed to learn the task due to a lack of action space exploration, resulting in high variability of agents' performance. Our findings suggest that in virtual agents, similar to infants, early energetic exploration can promote robust later learning. This may have the same effect as infants' curiosity‐based learning by which they shape their own curriculum. However, introducing new goals cannot wait too long, as there may be critical periods in development after which agents (as humans) cannot flexibly learn to incorporate new objectives. These lessons are making their way into machine learning and offer exciting new avenues for studying both human and machine learning of sequential action.

In a sequential reaching task, a robot arm trained using deep reinforcement learning optimizes its behavior in a similar manner to humans. This so‐called centering behavior, however, only tends to develop when learning is performed in a curriculum in which the reward function becomes increasingly complex over time. Furthermore, the onset of the curriculum seems to have a sensitive period, with too early or too late onset leading to unstable or suboptimal performance.

The Control of Event-File Management

According to the Theory of Event Coding, both perceived and self-produced events are coded by binding codes of the features of these events into event files. Here I argue that distinguishing between the actual binding process and the retrieval of event files is empirically difficult but theoretically important. As a first step towards disentangling these processes, I provide a brief overview of what the available evidence tells us with respect to the control of the binding process and the control of the retrieval process. I argue that there is not much evidence for selectivity of the binding process: Various kinds of stimuli and actions seem to be spontaneously integrated under various conditions, and there is increasing evidence that emotions, task instructions, and task context are coded into event files as well. On the other hand, there is increasing evidence for a high degree of selectivity of the retrieval process, suggesting that most if not all observations of effective impact on event files reflect an impact on retrieval, but not binding proper. I conclude by pointing out open questions and issues.

Towards the next generation of recurrent network models for cognitive neuroscience

Recurrent neural networks (RNNs) trained with machine learning techniques on cognitive tasks have become a widely accepted tool for neuroscientists. In this short opinion piece, we discuss fundamental challenges faced by the early work of this approach and recent steps to overcome such challenges and build next-generation RNN models for cognition. We propose several essential questions that practitioners of this approach should address to continue to build future generations of RNN models.

Reinforcement learning and its connections with neuroscience and psychology

Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision making in the brain. In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature. We then discuss the implications of this observed relationship between RL, neuroscience and psychology and its role in advancing research in both AI and brain science.

Touching events predict human action segmentation in brain and behavior

Recognizing the actions of others depends on segmentation into meaningful events. After decades of research in this area, it remains still unclear how humans do this and which brain areas support underlying processes. Here we show that a computer vision-based model of touching and untouching events can predict human behavior in segmenting object manipulation actions with high accuracy. Using this computational model and functional Magnetic Resonance Imaging (fMRI), we pinpoint the neural networks underlying this segmentation behavior during an implicit action observation task. Segmentation was announced by a strong increase of visual activity at touching events followed by the engagement of frontal, hippocampal and insula regions, signaling updating expectation at subsequent untouching events. Brain activity and behavior show that touching-untouching motifs are critical features for identifying the key elements of actions including object manipulations.

Habits and Goals in Human Behavior: Separate but Interacting Systems

People automatically repeat behaviors that were frequently rewarded in the past in a given context. Such repetition is commonly attributed to habit, or associations in memory between a context and a response. Once habits form, contexts directly activate the response in mind. An opposing view is that habitual behaviors depend on goals. However, we show that this view is challenged by the goal independence of habits across the fields of social and health psychology, behavioral neuroscience, animal learning, and computational modeling. It also is challenged by direct tests revealing that habits do not depend on implicit goals. Furthermore, we show that two features of habit memory-rapid activation of specific responses and resistance to change-explain the different conditions under which people act on habit versus persuing goals. Finally, we tested these features with a novel secondary analysis of action-slip data. We found that habitual responses are activated regardless of goals, but they can be performed in concert with goal pursuit.

The Development of Learning, Performing, and Controlling Repeated Sequential Actions in Young Children

Our daily lives are composed of several sequential actions that we perform routinely, such as making breakfast, taking a train, and changing clothes. Previous research has demonstrated that a routine system plays a role in performing and controlling repeated sequential actions in familiar situations, and a top-down control system involves the control of the routine system in novel situations. Specifically, most developmental studies have focused on the top-down control system (e.g., executive functions) as a factor enabling the control of goal-directed actions in novel situations. Yet, it has not been thoroughly examined how young children learn, perform, and control repeated sequential actions in familiar contexts. In this review, based on recent computational accounts for adults, we highlight two critical aspects of the routine system from a developmental perspective: (1) automatic flexible changes of contextual representations, which enables humans to select context-dependent actions appropriately; and (2) detection of deviant situations, which signals the need for control to avoid errors. In addition, we propose the developmental mechanism underlying the routine system and its potential driving factors such as statistical regularities and executive functions. Finally, we suggest that an investigation into the interplay between routine and executive functions can form foundations for understanding learning, performing, and controlling repeated sequential actions in young children and discuss future directions in this area.

Look-ahead fixations during visuomotor behavior: Evidence from assembling a camping tent

Eye movements can support ongoing manipulative actions, but a class of so-called look ahead fixations (LAFs) are related to future tasks. We examined LAFs in a complex natural task—assembling a camping tent. Tent assembly is a relatively uncommon task and requires the completion of multiple subtasks in sequence over a 5- to 20-minute duration. Participants wore a head-mounted camera and eye tracker. Subtasks and LAFs were annotated. We document four novel aspects of LAFs. First, LAFs were not random and their frequency was biased to certain objects and subtasks. Second, latencies are larger than previously noted, with 35% of LAFs occurring within 10 seconds before motor manipulation and 75% within 100 seconds. Third, LAF behavior extends far into future subtasks, because only 47% of LAFs are made to objects relevant to the current subtask. Seventy-five percent of LAFs are to objects used within five upcoming steps. Last, LAFs are often directed repeatedly to the target before manipulation, suggesting memory volatility. LAFs with short fixation–action latencies have been hypothesized to benefit future visual search and/or motor manipulation. However, the diversity of LAFs suggest they may also reflect scene exploration and task relevance, as well as longer term problem solving and task planning.

Neural Representations of Task Context and Temporal Order During Action Sequence Execution

Routine action sequences can share a great deal of similarity in terms of their stimulus response mappings. As a consequence, their correct execution relies crucially on the ability to preserve contextual and temporal information. However, there are few empirical studies on the neural mechanism and the brain areas maintaining such information. To address this gap in the literature, we recently recorded the blood-oxygen level dependent (BOLD) response in a newly developed coffee-tea making task. The task involves the execution of four action sequences that each comprise six consecutive decision states, which allows for examining the maintenance of contextual and temporal information. Here, we report a reanalysis of this dataset using a data-driven approach, namely multivariate pattern analysis, that examines context-dependent neural activity across several predefined regions of interest. Results highlight involvement of the inferior-temporal gyrus and lateral prefrontal cortex in maintaining temporal and contextual information for the execution of hierarchically organized action sequences. Furthermore, temporal information seems to be more strongly encoded in areas over the left hemisphere.

Individuals with congenital amusia do not show context-dependent perception of tonal categories

Perceptual adaptation is an active cognitive process where listeners re-analyse speech categories based on new contexts/situations/talkers. It involves top-down influences from higher cortical levels on lower-level auditory processes. Individuals with congenital amusia have impaired pitch processing with reduced connectivity between frontal and temporal regions. This study examined whether deficits in amusia would lead to impaired perceptual adaptation in lexical tone perception. Thirteen Mandarin-speaking amusics and 13 controls identified the category of target tones on an 8-step continuum ranging from rising to high-level, either in isolation or in a high-/low-pitched context. For tones with no context, amusics exhibited reduced categorical perception than controls. While controls' lexical tone categorization demonstrated a significant context effect due to perceptual adaptation, amusics showed similar categorization patterns across both contexts. These findings suggest that congenital amusia impacts the extraction of context-dependent tonal categories in speech perception, indicating that perceptual adaptation may depend on listeners' perceptual acuity.

Independent generation of sequence elements by motor cortex

Rapid execution of motor sequences is believed to depend on fusing movement elements into cohesive units that are executed holistically. We sought to determine the contribution of primary motor and dorsal premotor cortex to this ability. Monkeys performed highly practiced two-reach sequences, interleaved with matched reaches performed alone or separated by a delay. We partitioned neural population activity into components pertaining to preparation, initiation and execution. The hypothesis that movement elements fuse makes specific predictions regarding all three forms of activity. We observed none of these predicted effects. Rapid two-reach sequences involved the same set of neural events as individual reaches but with preparation for the second reach occurring as the first was in flight. Thus, at the level of dorsal premotor and primary motor cortex, skillfully executing a rapid sequence depends not on fusing elements, but on the ability to perform two key processes at the same time.

A Unifying Account of Angular Gyrus Contributions to Episodic and Semantic Cognition

The angular gyrus (AG) region of lateral parietal cortex has been implicated in a wide variety of tasks and functions, generating numerous influential theories. However, these theories largely fail to explain why so many apparently distinct cognitive activities implicate common parietal structures. We propose a unifying model, based on a set of central principles, to account for coalescences of cognitive task activations across AG. To illustrate the proposed framework, we show how these principles account for findings from studies of episodic and semantic memory that have independently implicated the same AG regions but thus far been considered from largely domain-specific perspectives. We conclude that AG computations, as part of a wider lateral parietal system, enable the online dynamic buffering of multisensory spatiotemporally extended representations.

The Best Laid Plans: Computational Principles of Anterior Cingulate Cortex

Despite continual debate for the past 30 years about the function of anterior cingulate cortex (ACC), its key contribution to neurocognition remains unknown. However, recent computational modeling work has provided insight into this question. Here we review computational models that illustrate three core principles of ACC function, related to hierarchy, world models, and cost. We also discuss four constraints on the neural implementation of these principles, related to modularity, binding, encoding, and learning and regulation. These observations suggest a role for ACC in hierarchical model-based hierarchical reinforcement learning (HMB-HRL), which instantiates a mechanism motivating the execution of high-level plans.

The influence of context stability on physical activity and sedentary behaviour habit and behaviour: An ecological momentary assessment study

OBJECTIVES

This study used ecological momentary assessment (EMA) to empirically test the theoretical propositions that habit for and level of physical activity (PA) and sedentary behaviour (SB) should be associated with degree of context stability of those behaviours.

DESIGN

Older adults (N = 104) completed a 10-day EMA protocol and continuous accelerometer monitoring.

METHODS

As part of the EMA protocol older adults answered 6 EMA prompts per day to assess current behaviour as well as social and physical contexts of behaviour. Temporal context was determined via time stamps of EMA questionnaires. Context stability was calculated as the reversed entropy scores of the contexts (physical, social, temporal, behavioural [i.e., type]) of PA and SB weighted for total frequency of context prompts. Habit for PA and SB (operationalized as self-reported behavioural automaticity) was assessed via baseline questionnaire. An ActivPAL monitor was worn to assess average daily time spent in moderate-vigorous PA (MVPA), light PA, and SB, and number of sit-to-stand transitions.

RESULTS

More stable physical contexts for physical activity predicted more MVPA (β = 10.22) and more stable social contexts for sitting predicted more SB (β = 1.36). More variety of time people tended to report engaging in SB, the more SB engaged in (β = -13.76). No context stability scores predicted light PA, sit-to-stand transitions, or habit.

CONCLUSIONS

Although context stability was related to behaviour, this did not appear to be explained by habit, as habit did not differ by the degree of context stability surrounding bouts of PA or SB.

Tea With Milk? A Hierarchical Generative Framework of Sequential Event Comprehension

To make sense of the world around us, we must be able to segment a continual stream of sensory inputs into discrete events. In this review, I propose that in order to comprehend events, we engage hierarchical generative models that "reverse engineer" the intentions of other agents as they produce sequential action in real time. By generating probabilistic predictions for upcoming events, generative models ensure that we are able to keep up with the rapid pace at which perceptual inputs unfold. By tracking our certainty about other agents' goals and the magnitude of prediction errors at multiple temporal scales, generative models enable us to detect event boundaries by inferring when a goal has changed. Moreover, by adapting flexibly to the broader dynamics of the environment and our own comprehension goals, generative models allow us to optimally allocate limited resources. Finally, I argue that we use generative models not only to comprehend events but also to produce events (carry out goal-relevant sequential action) and to continually learn about new events from our surroundings. Taken together, this hierarchical generative framework provides new insights into how the human brain processes events so effortlessly while highlighting the fundamental links between event comprehension, production, and learning.

Contribution of Executive Functions to Learning Sequential Actions in Young Children

This study examined whether executive functions impact how flexibly children represent task context in performing repeated sequential actions. Japanese children in Experiments 1 (N = 52; 3-6 years) and 2 (N = 50, 4-6 years) performed sequential actions repeatedly; one group received reminders. Experiment 1 indicated that reminders promote flexible changes in contextual representations. Experiment 2 observed such effects in younger children and showed executive functions were associated with the flexible representation of task context. Reminders did not perfectly compensate for the role of executive functions but wiped out individual differences in executive functions that contribute to children's acquisition of routines. Therefore, setting goals before context-dependent actions is necessary, but not sufficient, to modulate contextual representations in routines.

Toward a Computational Neuropsychology of Cognitive Flexibility

Cognitive inflexibility is a well-documented, yet non-specific corollary of many neurological diseases. Computational modeling of covert cognitive processes supporting cognitive flexibility may provide progress toward nosologically specific aspects of cognitive inflexibility. We review computational models of the Wisconsin Card Sorting Test (WCST), which represents a gold standard for the clinical assessment of cognitive flexibility. A parallel reinforcement-learning (RL) model provides the best conceptualization of individual trial-by-trial WCST responses among all models considered. Clinical applications of the parallel RL model suggest that patients with Parkinson's disease (PD) and patients with amyotrophic lateral sclerosis (ALS) share a non-specific covert cognitive symptom: bradyphrenia. Impaired stimulus-response learning appears to occur specifically in patients with PD, whereas haphazard responding seems to occur specifically in patients with ALS. Computational modeling hence possesses the potential to reveal nosologically specific profiles of covert cognitive symptoms, which remain undetectable by traditionally applied behavioral methods. The present review exemplifies how computational neuropsychology may advance the assessment of cognitive flexibility. We discuss implications for neuropsychological assessment and directions for future research.

Scooping Rice Into a Glass and Putting a Cell Phone in the Refrigerator: Action Slips in an Individual With a Diffuse Axonal Injury

A 39-year-old man with a diffuse axonal injury self-presented to the cognitive function clinic of the Ashikaga Red Cross Hospital complaining of behavioral errors in his daily life, such as scooping hot rice into a glass instead of a bowl or forgetting to turn off the gas stove after cooking. This type of error has been referred to as an action slip-a form of unintentional behavioral error that occurs when an individual attempts to perform an automatic and/or familiar task. In this case, action slips occurred nine times a day on average and had a serious and long-term impact on the man's daily quality of life. To reduce the impact of action slips, we created a one-on-one cognitive intervention that used a combination of mnemonic strategies, such as verbalizing his actions as he carried them out, and external aids, including a waist pouch, a voice-controlled artificial intelligence (AI) speaker, and an AI key finder. After 3 years of intervention, the man reported some improvement in his daily activities and a reduction in the number of action slips. Thus, intervention strategies for individuals with a diffuse axonal injury may benefit from targeting action slips. To our knowledge, this is the first detailed description of action slips in an individual with a diffuse axonal injury.

A Computational Study of Executive Dysfunction in Amyotrophic Lateral Sclerosis

Executive dysfunction is a well-documented, yet nonspecific corollary of various neurological diseases and psychiatric disorders. Here, we applied computational modeling of latent cognition for executive control in amyotrophic lateral sclerosis (ALS) patients. We utilized a parallel reinforcement learning model of trial-by-trial Wisconsin Card Sorting Test (WCST) behavior. Eighteen ALS patients and 21 matched healthy control participants were assessed on a computerized variant of the WCST (cWCST). ALS patients showed latent cognitive symptoms, which can be characterized as bradyphrenia and haphazard responding. A comparison with results from a recent computational Parkinson's disease (PD) study (Steinke et al., 2020, J Clin Med) suggests that bradyphrenia represents a disease-nonspecific latent cognitive symptom of ALS and PD patients alike. Haphazard responding seems to be a disease-specific latent cognitive symptom of ALS, whereas impaired stimulus-response learning seems to be a disease-specific latent cognitive symptom of PD. These data were obtained from the careful modeling of trial-by-trial behavior on the cWCST, and they suggest that computational cognitive neuropsychology provides nosologically specific indicators of latent facets of executive dysfunction in ALS (and PD) patients, which remain undiscoverable for traditional behavioral cognitive neuropsychology. We discuss implications for neuropsychological assessment, and we discuss opportunities for confirmatory computational brain imaging studies.

On Simulating Neural Damage in Connectionist Networks

A key strength of connectionist modelling is its ability to simulate both intact cognition and the behavioural effects of neural damage. We survey the literature, showing that models have been damaged in a variety of ways, e.g. by removing connections, by adding noise to connection weights, by scaling weights, by removing units and by adding noise to unit activations. While these different implementations of damage have often been assumed to be behaviourally equivalent, some theorists have made aetiological claims that rest on nonequivalence. They suggest that related deficits with different aetiologies might be accounted for by different forms of damage within a single model. We present two case studies that explore the effects of different forms of damage in two influential connectionist models, each of which has been applied to explain neuropsychological deficits. Our results indicate that the effect of simulated damage can indeed be sensitive to the way in which damage is implemented, particularly when the environment comprises subsets of items that differ in their statistical properties, but such effects are sensitive to relatively subtle aspects of the model's training environment. We argue that, as a consequence, substantial methodological care is required if aetiological claims about simulated neural damage are to be justified, and conclude more generally that implementation assumptions, including those concerning simulated damage, must be fully explored when evaluating models of neurological deficits, both to avoid over-extending the explanatory power of specific implementations and to ensure that reported results are replicable.

ELECTRONIC SUPPLEMENTARY MATERIAL

The online version of this article (10.1007/s42113-020-00081-z) contains supplementary material, which is available to authorized users.

Overarching Principles and Dimensions of the Functional Organization in the Inferior Parietal Cortex

The parietal cortex (PC) is implicated in a confusing myriad of different cognitive processes/tasks. Consequently, understanding the nature and organization of the core underlying neurocomputations is challenging. According to the Parietal Unified Connectivity-biased Computation model, two properties underpin PC function and organization. Firstly, PC is a multidomain, context-dependent buffer of time- and space-varying input, the function of which, over time, becomes sensitive to the statistical temporal/spatial structure of events. Secondly, over and above this core buffering computation, differences in long-range connectivity will generate graded variations in task engagement across subregions. The current study tested these hypotheses using a group independent component analysis technique with two independent functional magnetic resonance imaging datasets (task and resting state data). Three functional organizational principles were revealed: Factor 1, inferior PC was sensitive to the statistical structure of sequences for all stimulus types (pictures, sentences, numbers); Factor 2, a dorsal–ventral variation in generally task-positive versus task-negative (variable) engagement; and Factor 3, an anterior–posterior dimension in inferior PC reflecting different engagement in verbal versus visual tasks, respectively. Together, the data suggest that the core neurocomputation implemented by PC is common across domains, with graded task engagement across regions reflecting variations in the connectivity of task-specific networks that interact with PC.

Low-frequency neural activity reflects rule-based chunking during speech listening

Chunking is a key mechanism for sequence processing. Studies on speech sequences have suggested low-frequency cortical activity tracks spoken phrases, that is, chunks of words defined by tacit linguistic knowledge. Here, we investigate whether low-frequency cortical activity reflects a general mechanism for sequence chunking and can track chunks defined by temporarily learned artificial rules. The experiment records magnetoencephalographic (MEG) responses to a sequence of spoken words. To dissociate word properties from the chunk structures, two tasks separately require listeners to group pairs of semantically similar or semantically dissimilar words into chunks. In the MEG spectrum, a clear response is observed at the chunk rate. More importantly, the chunk-rate response is task-dependent. It is phase locked to chunk boundaries, instead of the semantic relatedness between words. The results strongly suggest that cortical activity can track chunks constructed based on task-related rules and potentially reflects a general mechanism for chunk-level representations.

From digital personal assistants like Siri and Alexa to customer service chatbots, computers are slowly learning to talk to us. But as anyone who has interacted with them will appreciate, the results are often imperfect.

Each time we speak or write, we use grammatical rules to combine words in a specific order. These rules enable us to produce new sentences that we have never seen or heard before, and to understand the sentences of others. But computer scientists adopt a different strategy when training computers to use language. Instead of grammar, they provide the computers with vast numbers of example sentences and phrases. The computers then use this input to calculate how likely for one word to follow another in a given context. "The sky is blue" is more common than "the sky is green", for example.

But is it possible that the human brain also uses this approach? When we listen to speech, the brain shows patterns of activity that correspond to units such as sentences. But previous research has been unable to tell whether the brain is using grammatical rules to recognise sentences, or whether it relies on a probability-based approach like a computer.

Using a simple artificial language, Jin et al. have now managed to tease apart these alternatives. Healthy volunteers listened to lists of words while lying inside a brain scanner. The volunteers had to group the words into pairs, otherwise known as chunks, by following various rules that simulated the grammatical rules present in natural languages. Crucially, the volunteers’ brain activity tracked the chunks – which differed depending on which rule had been applied – rather than the individual words. This suggests that the brain processes speech using abstract rules instead of word probabilities.

While computers are now much better at processing language, they still perform worse than people. Understanding how the human brain solves this task could ultimately help to improve the performance of personal digital assistants.