Dynamic field theory of movement preparation

Scanpaths in reading and picture viewing: Computer-assisted optimization of display conditions

A review of the literature shows that in reading and picture viewing cognitive skills play a key role along with visual acuity. Optimal processing conditions are reached only with letter and object sizes that match both cognitive skills and visual acuity. Beginning readers with normal vision need larger letters than skilled readers. In reading, eye movements step the fovea, a high-acuity region 2 degrees diameter, at the physiological pace of the visual system about 4 times per second. A simple computer-based procedure is described that determines the best acuity- and skill-matched letter (or object) sizes in the context of an optimal reading eye movement speed of 8 deg/s.

Response selection in schizophrenia

Schizophrenia patients tend to have longer and more variable latencies of response than healthy control subjects. However, the distributions of data from the two groups overlap to a large extent. Therefore, we investigated (1) whether the process of response selection in schizophrenia patients is like that of slow control subjects or has different properties, and (2) whether the intra-individual variability of schizophrenia patients is intrinsically greater than that of control subjects or reflects their longer mean latency. To answer these questions we tested schizophrenia patients and healthy control subjects in a choice reaction time (RT) task with 2-choice and 4-choice conditions. We analyzed how mean RT in the 2-choice condition predicted mean RT in the 4-choice condition and found that the relation was significantly different between the two groups. In contrast, the intra-individual variability of RT was related to mean RT in the same way for schizophrenia patients and control subjects. These results indicate that the response selection process of schizophrenia patients was not simply a slower version of the same process engaged by control subjects, but it was a selection process with different dynamic properties. In contrast, schizophrenia patients did not have a greater intrinsic variability than control subjects. Furthermore, we found that the difference Deltat between RT measured in the 4-choice condition and RT predicted for the control group in the same condition could be used to discriminate effectively patients and control subjects. However, there was no significant association between Deltat and clinical variables. These results suggest that Deltat could reflect a trait impairment of schizophrenia independent from symptom profile. Finally, we suggest that the impairment of the process of selection of the motor response in schizophrenia reflects the alteration of the time-dependent patterns of neural activity that result from anomalies in the connectivity of the brain areas engaged for the selection of the motor response.

The planning and execution of short auditory sequences

Action planning, but not action execution, in speeded tasks is typically faster when responses and their effects are compatible than when they are incompatible. We tested whether response-effect compatibility (REC) affects the execution of music-like sequential actions that require temporal regularity rather than rapidity. Musicians responded to metronomic visual stimuli by producing sequences of three taps at a specific tempo on three vertically aligned keys. Each tap triggered a tone. Key-to-tone mapping was either compatible or incompatible in terms of spatial height and pitch height. The results indicated that tap timing was more accurate with compatible than with incompatible mappings, both for taps produced before (Tap 1) and after (Taps 2 and 3) the onset of auditory feedback. Thus, the observed influence of REC on action execution was not due exclusively to actual auditory feedback. The anticipation of distal action effects may be involved in planning the dynamics of temporally precise movements.

Action planning in young children's tool use

Tool use consists of at least two coupled phases of activities, involving multi-step problem solving. It therefore provides an interesting window on the development of planning in goal-directed behavior. This study investigated 2-year-olds' and 3-year-olds' hand use in picking up and subsequently using a tool for displacing a target-object towards a specified goal location. The children had to use a stick (Experiment 1; N = 41 in total) or a cane (hooked stick) that was lying in varying starting orientations (Experiment 2; N = 32 in total). Age differences were found in the way goal-related information in combination with tool-related information influenced the choice of which hand to use in different phases of the task. A view on action planning is developed as a dynamic action-selection process for which actions to take. This process integrates factors that are internal to the child's action system (e.g. motor preferences and dexterity) with external (i.e. sensory) sources of information.

Reference-related inhibition produces enhanced position discrimination and fast repulsion near axes of symmetry

Models proposed to account for reference frame effects in spatial cognition often account for performance in some tasks well, but fail to generalize to other tasks. Here, we demonstrate that a new process account of spatial working memory--the dynamic field theory (DFT)--can bridge the gap between perceptual and memory processes in position discrimination and spatial recall, highlighting that the processes underlying spatial recall also operate in position discrimination. In six experiments, we tested two novel predictions of the DFT: first, that discrimination is enhanced near symmetry axes, especially when the perceptual salience of the axis is increased; and second, that performance far from a reference axis depends on the direction in which the second stimulus is presented. The DFT also predicts the magnitude of this direction-dependent modulation. These effects arise from reference-related inhibition in the theory. We discuss how the processes captured by the DFT relate to existing psychophysical models and operate across a diverse array of spatial tasks.

Dynamic updating of distributed neural representations using forward models

In this paper, we present a continuous attractor network model that we hypothesize will give some suggestion of the mechanisms underlying several neural processes such as velocity tuning to visual stimulus, sensory discrimination, sensorimotor transformations, motor control, motor imagery, and imitation. All of these processes share the fundamental characteristic of having to deal with the dynamic integration of motor and sensory variables in order to achieve accurate sensory prediction and/or discrimination. Such principles have already been described in the literature by other high-level modeling studies (Decety and Sommerville in Trends Cogn Sci 7:527-533, 2003; Oztop et al. in Neural Netw 19(3):254-271, 2006; Wolpert et al. in Philos Trans R Soc 358:593-602, 2003). With respect to these studies, our work is more concerned with biologically plausible neural dynamics at a population level. Indeed, we show that a relatively simple extension of the classical neural field models can endow these networks with additional dynamic properties for updating their internal representation using external commands. Moreover, an analysis of the interactions between our model and external inputs also shows interesting properties, which we argue are relevant for a better understanding of the neural processes of the brain.

Integrated neural processes for defining potential actions and deciding between them: a computational model

To successfully accomplish a behavioral goal such as reaching for an object, an animal must solve two related problems: to decide which object to reach and to plan the specific parameters of the movement. Traditionally, these two problems have been viewed as separate, and theories of decision making and motor planning have been developed primarily independently. However, neural data suggests that these processes involve the same brain regions and are performed in an integrated manner. Here, a computational model is described that addresses both the question of how different potential actions are specified and how the brain decides between them. In the model, multiple potential actions are simultaneously represented as continuous regions of activity within populations of cells in frontoparietal cortex. These representations engage in a competition for overt execution that is biased by modulatory influences from prefrontal cortex. The model neural populations exhibit activity patterns that correlate with both the spatial metrics of potential actions and their associated decision variables, in a manner similar to activities in parietal, prefrontal, and premotor cortex. The model therefore suggests an explanation for neural data that have been hard to account for in terms of serial theories that propose that decision making occurs before action planning. In addition to simulating the activity of individual neurons during decision tasks, the model also reproduces key aspects of the spatial and temporal statistics of human choices and makes a number of testable predictions.

Stochastic models of decisions about motion direction: behavior and physiology

Roitman and Shadlen [Roitman J. D., & Shadlen M. N. (2002). Response of neurons in the lateral intraparietal area during a combined visual discrimination reaction time task. Journal of Neuroscience, 22, 9475-9489] have published a non-human primate study on visual decision making. They collected both behavioral and neurophysiological data and provided evidence that the data are qualitatively consistent with a mechanism based on accumulating sensory evidence up to a decision threshold. I have previously demonstrated that a time-variant diffusion model can account quite well quantitatively for both the behavioral and the neural data. In this manuscript I discuss how well the data constrains different components and parameters of the computational process. I also discuss the biological plausibility of the model parameters. I will demonstrate that a relatively large class of models, both with and without temporal integration and both stationary and time-variant could account for the behavioral data. Both the single cell recordings from the parietal cortex and previously published data from the extrastriate visual cortex provide additional constraints. Overall, the data favor a diffusion model with time-variant gain and leaky integrators. The integration time constant, however, turns out not to be well-constrained by the data.

The time course of saccadic decision making: dynamic field theory

Making a saccadic eye movement involves two decisions, the decision to initiate the saccade and the selection of the visual target of the saccade. Here we provide a theoretical account for the time-courses of these two processes, whose instabilities are the basis of decision making. We show how the cross-over from spatial averaging for fast saccades to selection for slow saccades arises from the balance between excitatory and inhibitory processes. Initiating a saccade involves overcoming fixation, as can be observed in the countermanding paradigm, which we model accounting both for the temporal evolution of the suppression probability and its dependence on fixation activity. The interaction between the two forms of decision making is demonstrated by predicting how the cross-over from averaging to selection depends on the fixation stimulus in gap-step-overlap paradigms. We discuss how the activation dynamics of our model may be mapped onto neuronal structures including the motor map and the fixation cells in superior colliculus.

Using dynamic field theory to rethink infant habituation

Much of what psychologists know about infant perception and cognition is based on habituation, but the process itself is still poorly understood. Here the authors offer a dynamic field model of infant visual habituation, which simulates the known features of habituation, including familiarity and novelty effects, stimulus intensity effects, and age and individual differences. The model is based on a general class of dynamic (time-based) models that integrate environmental input in varying metric dimensions to reach a single decision. Here the authors provide simulated visual input of varying strengths, distances, and durations to 2 coupled and interacting fields. The 1st represents the activation that drives "looking," and the 2nd, the inhibition that leads to "looking away," or habituation. By varying the parameters of the field, the authors simulate the time course of habituation trials and show how these dynamics can lead to different depths of habituation, which then determine how the system dishabituates. The authors use the model to simulate a set of influential experiments by R. Baillargeon (1986, 1987a, 1987b) using the well-known "drawbridge" paradigm. The dynamic field model provides a coherent explanation without invoking infant object knowledge. The authors show that small changes in model parameters can lead to qualitatively different outcomes. Because in typical infant cognition experiments, critical parameters are unknown, effects attributed to conceptual knowledge may be explained by the dynamics of habituation.

A dynamic model for action understanding and goal-directed imitation

The understanding of other individuals' actions is a fundamental cognitive skill for all species living in social groups. Recent neurophysiological evidence suggests that an observer may achieve the understanding by mapping visual information onto his own motor repertoire to reproduce the action effect. However, due to differences in embodiment, environmental constraints or motor skills, this mapping very often cannot be direct. In this paper, we present a dynamic network model which represents in its layers the functionality of neurons in different interconnected brain areas known to be involved in action observation/execution tasks. The model aims at substantiating the idea that action understanding is a continuous process which combines sensory evidence, prior task knowledge and a goal-directed matching of action observation and action execution. The model is tested in variations of an imitation task in which an observer with dissimilar embodiment tries to reproduce the perceived or inferred end-state of a grasping-placing sequence. We also propose and test a biologically plausible learning scheme which allows establishing during practice a goal-directed organization of the distributed network. The modeling results are discussed with respect to recent experimental findings in action observation/execution studies.

The dynamic neural field approach to cognitive robotics

This tutorial presents an architecture for autonomous robots to generate behavior in joint action tasks. To efficiently interact with another agent in solving a mutual task, a robot should be endowed with cognitive skills such as memory, decision making, action understanding and prediction. The proposed architecture is strongly inspired by our current understanding of the processing principles and the neuronal circuitry underlying these functionalities in the primate brain. As a mathematical framework, we use a coupled system of dynamic neural fields, each representing the basic functionality of neuronal populations in different brain areas. It implements goal-directed behavior in joint action as a continuous process that builds on the interpretation of observed movements in terms of the partner's action goal. We validate the architecture in two experimental paradigms: (1) a joint search task; (2) a reproduction of an observed or inferred end state of a grasping-placing sequence. We also review some of the mathematical results about dynamic neural fields that are important for the implementation work.

Task Switching as a Two-Stage Decision Process

Saccades represent decisions, and the study of their latency has led to a neurally plausible model of the underlying mechanisms, LATER (Linear Approach to Threshold with Ergodic Rate), that can successfully predict reaction time behavior in simple decision tasks, with fixed instructions. However, if the instructions abruptly change, we have a more complex situation, known as task switching. Psychologists' explanations of the phenomena of task switching have so far tended to be qualitative rather than quantitative, and not intended to relate particularly clearly to existing models of decision making or to likely neural implementations. Here, we investigated task switching using a novel saccadic task: we presented the instructions by stimulus elements identical to those of the task itself, allowing us to compare decisions about instructions with decisions in the actual task. Our results support a relatively simple model consisting of two distinct LATER processes in series: the first detects the instruction, the second implements it.

A Sensorimotor Map: Modulating Lateral Interactions for Anticipation and Planning

Experimental studies of reasoning and planned behavior have provided evidence that nervous systems use internal models to perform predictive motor control, imagery, inference, and planning. Classical (model-free) reinforcement learning approaches omit such a model; standard sensorimotor models account for forward and backward functions of sensorimotor dependencies but do not provide a proper neural representation on which to realize planning. We propose a sensorimotor map to represent such an internal model. The map learns a state representation similar to self-organizing maps but is inherently coupled to sensor and motor signals. Motor activations modulate the lateral connection strengths and thereby induce anticipatory shifts of the activity peak on the sensorimotor map. This mechanism encodes a model of the change of stimuli depending on the current motor activities. The activation dynamics on the map are derived from neural field models. An additional dynamic process on the sensorimotor map (derived from dynamic programming) realizes planning and emits corresponding goal-directed motor sequences, for instance, to navigate through a maze.

Neural variability in premotor cortex provides a signature of motor preparation

We present experiments and analyses designed to test the idea that firing rates in premotor cortex become optimized during motor preparation, approaching their ideal values over time. We measured the across-trial variability of neural responses in dorsal premotor cortex of three monkeys performing a delayed-reach task. Such variability was initially high, but declined after target onset, and was maintained at a rough plateau during the delay. An additional decline was observed after the go cue. Between target onset and movement onset, variability declined by an average of 34%. This decline in variability was observed even when mean firing rate changed little. We hypothesize that this effect is related to the progress of motor preparation. In this interpretation, firing rates are initially variable across trials but are brought, over time, to their "appropriate" values, becoming consistent in the process. Consistent with this hypothesis, reaction times were longer if the go cue was presented shortly after target onset, when variability was still high, and were shorter if the go cue was presented well after target onset, when variability had fallen to its plateau. A similar effect was observed for the natural variability in reaction time: longer (shorter) reaction times tended to occur on trials in which firing rates were more (less) variable. These results reveal a remarkable degree of temporal structure in the variability of cortical neurons. The relationship with reaction time argues that the changes in variability approximately track the progress of motor preparation.

Parallel and distributed neural models of the ideomotor principle: An investigation of imitative cortical pathways

Humans' capacity to imitate has been extensively investigated through a wide-range of behavioral and developmental studies. Yet, despite the huge amount of phenomenological evidence gathered, we are still unable to relate this behavioral data to any specific neural substrate. In this paper, we investigate how principles from psychology can be the result of neural computations and therefore attempt to bridge the gap between monkey neurophysiology and human behavioral data, and hence between these two complementary disciplines. Specifically, we address the principle of ideomotor compatibility, by which 'observing the movements of others influences the quality of one's own performance' and develop two neural models which account for a set of related behavioral studies [Brass, M., Bekkering, H., Wohlschläger, A., & Prinz, W. (2000). Compatibility between observed and executed finger movements: comparing symbolic, spatial and imitative cues. Brain and Cognition 44, 124-143]. We show that the ideomotor effect could be the result of two distinct cognitive pathways, which can be modeled by means of biologically plausible neural architectures. Furthermore, we propose a novel behavioral experiment to confirm or refute either of the two model pathways.

SWIFT explorations of age differences in eye movements during reading

Research on eye movements in reading has made significant advances during the past few years, due to both experimental and computational research. Age effects have not been extensively studied, but the overall pattern suggests more quantitative than qualitative differences in fixation durations and fixation probabilities. Here we focus on age-differential effects of word frequency on reading time and on probabilities of skipping a word or regressing to previous ones. We present an overview of SWIFT [Engbert, R., Nuthmann, A., Richter, E.M., Kliegl, R., 2005. SWIFT: a dynamical model of saccade generation during reading. Psychological Review 112, 777-813], a fully implemented computational model of saccade generation and lexical processing during reading, based on spatially distributed processing over several words. Preliminary simulations of age differences recovered most, but not all experimental effects. Age differences in parameter estimates point towards an important role of visual acuity for oculomotor as well as lexical processing.

The effects of response priming on the planning and execution of goal-directed movements in the presence of a distracting stimulus

Two models of selective reaching have been proposed to account for deviations in movement trajectories in cluttered environments. The response vector model predicts movement trajectories should deviate towards or away from the location a distractor of little or large salience, respectively. In contrast, the response activation model predicts that a distractor with large salience should cause movement deviations towards it whereas a distractor with little salience should not influence the movement. The precuing technique was combined with the distractor interference paradigm to test these predictions. Results indicate that when the target was presented at the precued (salient) location, movements were unaffected by a distractor. Conversely, when the distractor was presented at the precued location while the target was presented at an uncued (non-salient) location, participants demonstrated increased reaction times and trajectory deviations towards the location of the distractor. These findings are consistent with the model of response activation.

Neural correlates of reaching decisions in dorsal premotor cortex: specification of multiple direction choices and final selection of action

We show that while a primate chooses between two reaching actions, its motor system first represents both options and later reflects selection between them. When two potential targets appeared, many (43%) task-related, directionally tuned cells in dorsal premotor cortex (PMd) discharged if one of the targets was near their preferred direction. At the population level, this generated two simultaneous sustained directional signals corresponding to the current reach options. After a subsequent nonspatial cue identified the correct target, the corresponding directional signal increased, and the signal for the rejected target was suppressed. The PMd population reliably predicted the monkey's response choice, including errors. This supports a planning model in which multiple reach options are initially specified and then gradually eliminated in a competition for overt execution, as more information accumulates.

SWIFT: A Dynamical Model of Saccade Generation During Reading

Mathematical models have become an important tool for understanding the control of eye movements during reading. Main goals of the development of the SWIFT model (R. Engbert, A. Longtin, & R. Kliegl, 2002) were to investigate the possibility of spatially distributed processing and to implement a general mechanism for all types of eye movements observed in reading experiments. The authors present an advanced version of SWIFT that integrates properties of the oculomotor system and effects of word recognition to explain many of the experimental phenomena faced in reading research. They propose new procedures for the estimation of model parameters and for the test of the model's performance. They also present a mathematical analysis of the dynamics of the SWIFT model. Finally, within this framework, they present an analysis of the transition from parallel to serial processing.

Unifying representations and responses: perseverative biases arise from a single behavioral system

A dominant account of perseverative errors in early development contends that such errors reflect a failure to inhibit a prepotent response. This study investigated whether perseveration might also arise from a failure to inhibit a prepotent representation. Children watched as a toy was hidden at an A location, waited during a delay, and then watched the experimenter find the toy. After six observation-only A trials, the toy was hidden at a B location, and children were allowed to search for the toy. Two- and 4-year-olds' responses on the B trials were significantly biased toward A even though they had never overtly responded to this location. Thus, perseverative biases in early development can arise as a result of prepotent representations, demonstrating that the prepotent-response account is incomplete. We discuss three alternative interpretations of these results, including the possibility that representational and response-based biases reflect the operation of a single, integrated behavioral system.

Developmental continuity in the processes that underlie spatial recall

This study investigated whether children's spatial recall performance shows three separable characteristics: (1) biases away from symmetry axes (geometric effects); (2) systematic drift over delays; and (3) biases toward the exemplar distribution experienced in the task (experience-dependent effects). In Experiment 1, the location of one target within each geometric category was varied. Children's responses showed biases away from a midline axis that increased over delays. In Experiment 2, multiple targets were placed within each category at the same locations used in Experiment 1. After removing geometric effects, 6-year-olds'--but not 11-year-olds'--responses were biased toward the average remembered location over learning. In Experiment 3, children responded to one target more frequently than the others. Both 6- and 11-year-olds showed biases toward the most frequent target over learning. These results provide a bridge between the performance of younger children and adults, demonstrating continuity in the processes that underlie spatial memory abilities across development.

Development as a dynamic system

Development is about creating something more from something less, for example, a walking and talking toddler from a helpless infant. One current theoretical framework views the developmental process as a change within a complex dynamic system. Development is seen as the emergent product of many decentralized and local interactions that occur in real time. We examine how studying the multicausality of real-time processes could be the key to understanding change over developmental time. We specifically consider recent research and theory on perseverative reaching by infants as a case study that demonstrates this approach.

Attention maintains mental extrapolation of target position: irrelevant distractors eliminate forward displacement after implied motion

Observers' judgments of the final position of a moving target are typically shifted in the direction of implied motion ("representational momentum"). The role of attention is unclear: visual attention may be necessary to maintain or halt target displacement. When attention was captured by irrelevant distractors presented during the retention interval, forward displacement after implied target motion disappeared, suggesting that attention may be necessary to maintain mental extrapolation of target motion. In a further corroborative experiment, the deployment of attention was measured after a sequence of implied motion, and faster responses were observed to stimuli appearing in the direction of motion. Thus, attention may guide the mental extrapolation of target motion. Additionally, eye movements were measured during stimulus presentation and retention interval. The results showed that forward displacement with implied motion does not depend on eye movements. Differences between implied and smooth motion are discussed with respect to recent neurophysiological findings.

The effects of probability and relative direction on human somatomotor electroencephalographic rhythms

Classical views on the preparation of voluntary movement involve separate motor and cognitive processes. Recent neurophysiological evidence challenges this dichotomy. Recordings from within the motor cortices have demonstrated that cognitive factors modulate neural activity. This issue was examined in event-related desynchronizations (ERDs) by using a goal-directed pointing paradigm. The ERD, which is typically recorded in response to self-paced finger movements, has been demonstrated to reflect specific movement parameters. Cognitive factors were introduced by manipulating probability and the relative direction of the responses in a choice reaction time task. The results demonstrated that probability influences the 20 Hz ERD.