Hidden cognitive states revealed in choice reaching tasks

Studying the dynamics of visual search behavior using RT hazard and micro-level speed-accuracy tradeoff functions: A role for recurrent object recognition and cognitive control processes

Thanks to the work of Anne Treisman and many others, the visual search paradigm has become one of the most popular paradigms in the study of visual attention. However, statistics like mean correct response time (RT) and percent error do not usually suffice to decide between the different search models that have been developed. Recently, to move beyond mean performance measures in visual search, RT histograms have been plotted, theoretical waiting time distributions have been fitted, and whole RT and error distributions have been simulated. Here we promote and illustrate the general application of discrete-time hazard analysis to response times, and of micro-level speed-accuracy tradeoff analysis to timed response accuracies. An exploratory analysis of published benchmark search data from feature, conjunction, and spatial configuration search tasks reveals new features of visual search behavior, such as a relatively flat hazard function in the right tail of the RT distributions for all tasks, a clear effect of set size on the shape of the RT distribution for the feature search task, and individual differences in the presence of a systematic pattern of early errors. Our findings suggest that the temporal dynamics of visual search behavior results from a decision process that is temporally modulated by concurrently active recurrent object recognition, learning, and cognitive control processes, next to attentional selection processes.

The Psychology of Reaching: Action Selection, Movement Implementation, and Sensorimotor Learning

The study of motor planning and learning in humans has undergone a dramatic transformation in the 20 years since this journal's last review of this topic. The behavioral analysis of movement, the foundational approach for psychology, has been complemented by ideas from control theory, computer science, statistics, and, most notably, neuroscience. The result of this interdisciplinary approach has been a focus on the computational level of analysis, leading to the development of mechanistic models at the psychological level to explain how humans plan, execute, and consolidate skilled reaching movements. This review emphasizes new perspectives on action selection and motor planning, research that stands in contrast to the previously dominant representation-based perspective of motor programming, as well as an emerging literature highlighting the convergent operation of multiple processes in sensorimotor learning.

The attractiveness of salient distractors to reaching movements is task dependent

Previous studies in visual attention and oculomotor research showed that a physically salient distractor does not always capture attention or the eyes. Under certain top-down task sets, a salient distractor can be actively suppressed, avoiding capture. Even though previous studies showed that reaching movements are also influenced by salient distractors, it is unclear if and how a mechanism of active suppression of distractors would affect reaching movements. Active suppression might also explain why some studies find reaching movements to curve towards a distractor, while others find reaching movements to curve away. In this study, we varied the top-down task set in two separate experiments by manipulating the certainty about the target location. Participants had to reach for a diamond present among three circles. In Experiments 1 and 3, participants had to search for the reach targets; hence, the target's location certainty was low. In Experiments 2 and 3, the target's location was cued before the reach; hence, the target's location certainty was high. We found that reaches curved towards the physically salient, color singleton, distractor in the search-to-reach task (Experiments 1 and 3), but not in the cued reach task (Experiments 2 and 3). Thus, the saliency of the distractor only attracted reaching movements when the certainty of the target's location was low. Our findings suggest that the attractiveness of physically salient distractors to reaching movements depends on the top-down task set. The results can be explained by the effect of active attentional suppression on the competition between movement plans.

Perceptual decisions about object shape bias visuomotor coordination during rapid interception movements

Visual processing for perception and for action is thought to be mediated by two specialized neural pathways. Using a visuomotor decision-making task, we show that participants differentially utilized online perceptual decision-making in reaching and interception and that eye movements necessary for perception influenced motor decision strategies. These results provide evidence that task complexity modulates how pathways processing perception versus action information interact during the visual control of movement.

Attention for Speaking: Prestimulus Motor-cortical Alpha Power Predicts Picture Naming Latencies

There is a range of variability in the speed with which a single speaker will produce the same word from one instance to another. Individual differences studies have shown that the speed of production and the ability to maintain attention are related. This study investigated whether fluctuations in production latencies can be explained by spontaneous fluctuations in speakers' attention just prior to initiating speech planning. A relationship between individuals' incidental attentional state and response performance is well attested in visual perception, with lower prestimulus alpha power associated with faster manual responses. Alpha is thought to have an inhibitory function: Low alpha power suggests less inhibition of a specific brain region, whereas high alpha power suggests more inhibition. Does the same relationship hold for cognitively demanding tasks such as word production? In this study, participants named pictures while EEG was recorded, with alpha power taken to index an individual's momentary attentional state. Participants' level of alpha power just prior to picture presentation and just prior to speech onset predicted subsequent naming latencies. Specifically, higher alpha power in the motor system resulted in faster speech initiation. Our results suggest that one index of a lapse of attention during speaking is reduced inhibition of motor-cortical regions: Decreased motor-cortical alpha power indicates reduced inhibition of this area while early stages of production planning unfold, which leads to increased interference from motor-cortical signals and longer naming latencies. This study shows that the language production system is not impermeable to the influence of attention.

Effects of Visual Scene Complexity on Neural Signatures of Spatial Attention

Spatial selective attention greatly affects our processing of complex visual scenes, yet the way in which the brain selects relevant objects while suppressing irrelevant objects is still unclear. Evidence of these processes has been found using non-invasive electroencephalography (EEG). However, few studies have characterized these measures during attention to dynamic stimuli, and little is known regarding how these measures change with increased scene complexity. Here, we compared attentional modulation of the EEG N1 and alpha power (oscillations between 8–14 Hz) across three visual selective attention tasks. The tasks differed in the number of irrelevant stimuli presented, but all required sustained attention to the orientation trajectory of a lateralized stimulus. In scenes with few irrelevant stimuli, top-down control of spatial attention is associated with strong modulation of both the N1 and alpha power across parietal-occipital channels. In scenes with many irrelevant stimuli in both hemifields, however, top-down control is no longer represented by strong modulation of alpha power, and N1 amplitudes are overall weaker. These results suggest that as a scene becomes more complex, requiring suppression in both hemifields, the neural signatures of top-down control degrade, likely reflecting some limitation in EEG to represent this suppression.

Action information contributes to metacognitive decision-making

Metacognitive abilities allow us to adjust ongoing behavior and modify future decisions in the absence of external feedback. Although metacognition is critical in many daily life settings, it remains unclear what information is actually being monitored and what kind of information is being used for metacognitive decisions. In the present study, we investigated whether response information connected to perceptual events contribute to metacognitive decision-making. Therefore, we recorded EEG signals during a perceptual color discrimination task while participants were asked to provide an estimate about the quality of their decision on each trial. Critically, the moment participants provided their confidence judgments varied across conditions, thereby changing the amount of action information (e.g., response competition or response fluency) available for metacognitive decisions. Results from three experiments demonstrate that metacognitive performance improved when first-order action information was available at the moment metacognitive decisions about the perceptual task had to be provided. This behavioral effect was accompanied by enhanced functional connectivity (beta phase synchrony) between motor areas and prefrontal regions, exclusively observed during metacognitive decision-making. Our findings demonstrate that action information contributes to metacognitive decision-making, thereby painting a picture of metacognition as a process that integrates sensory evidence and information about our interactions with the world.

Reaching decisions during ongoing movements

Neurophysiological studies suggest that when decisions are made between concrete actions, the selection process involves a competition between potential action representations in the same sensorimotor structures involved in executing those actions. However, it is unclear how such models can explain situations, often encountered during natural behavior, in which we make decisions while were are already engaged in performing an action. Does the process of deliberation characterized in classical studies of decision-making proceed the same way when subjects are deciding while already acting? In the present study, human subjects continuously tracked a target moving in the horizontal plane and were occasionally presented with a new target to which they could freely choose to switch at any time, whereupon it became the new tracked target. We found that the probability of choosing to switch increased with decreasing distance to the new target and increasing size of the new target relative to the tracked target, as well as when the direction to the new target was aligned (either toward or opposite) to the current tracking direction. However, contrary to our expectations, subjects did not choose targets that minimized the energetic costs of execution, as calculated by a biomechanical model of the arm. When the constraints of continuous tracking were removed in variants of the task involving point-to-point movements, the expected preference for lower cost choices was seen. These results are discussed in the context of current theories of nested feedback control, internal models of forward dynamics, and high-dimensional neural spaces.NEW & NOTEWORTHY Current theories of decision-making primarily address how subjects make decisions before executing selected actions. However, in our daily lives we often make decisions while already performing some action (e.g., while playing a sport or navigating through a crowd). To gain insight into how current theories can be extended to such "decide-while-acting" scenarios, we examined human decisions during continuous manual tracking and found some intriguing departures from how decisions are made in classical "decide-then-act" paradigms.

A comparison of simple movement behaviors across three different devices

Reaching trajectories have provided a unique tool to observe changes in internal cognitive decisions. Furthermore, technological advances have made devices for measuring reach movements more accessible and researchers have recognized that various populations including children, elderly populations, and non-human primates can easily execute simple movements as responses. As a result, devices such as a three-dimensional (3D) reach tracker, a stylus, or a computer-mouse have been increasingly utilized to study cognitive processes. However, although the specific type of tracking device that a researcher uses may impact behavior due to the constraints it places on movements, most researchers in these fields are unaware of this potential issue. Here, we examined the potential behavioral impact of using each of these three devices. To induce re-directed movements that mimic the movements that often occur following changes in cognitive states, we used a double-step task in which displacement of an initial target location requires participants to quickly re-direct their movement. We found that reach movement parameters were largely comparable across the three devices. However, hand movements measured by a 3D reach tracker showed earlier reach initiation latencies (relative to stylus movements) and more curved movement trajectories (relative to both mouse and stylus movements). Reach movements were also re-directed following target displacement more rapidly. Thus, 3D reach trackers may be ideal for observing fast, subtle changes in internal decision-making processes compared to other devices. Taken together, this study provides a useful reference for comparing and implementing reaching studies to examine human cognition.

A 'no' with a trace of 'yes': A mouse-tracking study of negative sentence processing

There is strong evidence that comprehenders can parse sentences in an incremental fashion. However, when the sentence contains a negation, the evidence is less clear. Previous work has shown that increasing the pragmatic felicity of a negative sentence reduces or eliminates any processing overhead relative to affirmative sentences. However, in previous work felicity has gone hand-in-hand with the predictability of critical material in a sentence. In three experiments reported here, we presented equally felicitous sentences with critical material of varying predictability (operationalised as the number of possible completions) to test whether this might be a critical factor determining the ease with which partial sentences containing a negation are interpreted. Participants completed a truth-value judgement task (Experiment 1) or a sentence completion task (Experiments 2 and 3) after viewing a visual environment that provided the context for a test sentence, which could differ in truth value (in Experiment 1 only), polarity (affirmative or negative), and number of possible completions (one, two, or three). In all three experiments, we recorded response times and accuracy, but also response dynamics via participants' computer mouse trajectories, allowing us to test specific hypotheses about the time course of comprehension. Across all experiments, in conditions with one or two possible targets, we observed consistent detrimental effects of negative polarity, suggesting that the difficulty in processing negation cannot be reduced to effects relating to predictability or pragmatic felicity. We discuss this finding in relation to incremental and two-stage models of processing and outline a new account of the processing difficulty arising from negation in terms of a conflict between what is locally activated on the basis of individual words and phrases and the global meaning of a negative sentence.

Human-machine Interaction: Adapted Safety Assistance in Mentality Using Hidden Markov Chain and Petri Net

This study proposes a cognition-adaptive approach for the administrative control of human-machine safety interaction through Internet of Things (IoT) data. As part of Industry 4.0, a human operator possesses various characteristics, but cannot be consistently understood as well as a machine. Thus, human-machine interaction plays an important role. This study focuses on incumbent challenges on the basis of estimated mental states. Given the operation logs from data recording hardware, a Hidden Markov model on top of a human cognitive model was trained to capture a production line worker’s sequential faults. Our study found that retaining workers’ attention is insufficient and tracking the state of perception is key to accomplishing production tasks. A safe workflow policy requires attention and perception. Accordingly, our proposed Petri Net enhances operation safety and improves production efficiency.

Psychometrics of the continuous mind: Measuring cognitive sub-processes via mouse tracking

The measurement of psychological properties often relies on discrete measures, for example, answers in questionnaires or responses in tasks. This focus on discrete measures neglects information that is present in the process leading to an answer or a response. A method to trace such processes is mouse tracking. Mouse tracking promises to open a continuous window onto the processes leading from a stimulus to a response. However, most mouse-tracking studies fall short of the promise to extract dynamic psychometrically valid markers for the different sub-processes, which are intertwined on the way to the final response. Here we used time-continuous multiple regression (TCMR) to extract dynamic markers for the different sub-processes leading to a response. From these markers, we extracted information about the timing, the duration, and the strength of the influence of the different sub-processes. We evaluated these dynamic measures of sub-processes for their psychometric properties, i.e. reliability, which is a basis for their use in the study of individual differences. Furthermore, we applied these dynamic measures in a group-level study to identify differences in the sub-processes of resolving response conflict between groups performing either a Simon or a flanker task. We found specific temporal patterns that match predictions from a conceptual model of these tasks. We concluded that the extracted information from mouse movements could be used as psychometrically valid dynamic measures of psychological properties and their differences across individuals and situations.A software toolbox to perform the described analyses in Matlab is provided (osf.io/5e3vn).

Lost to translation: How design factors of the mouse-tracking procedure impact the inference from action to cognition

From an embodiment perspective, action and cognition influence each other constantly. This interaction has been utilized in mouse-tracking studies to infer cognitive states from movements, assuming a continuous manifestation of cognitive processing into movement. However, it is mostly unknown how this manifestation is affected by the variety of possible design choices in mouse-tracking paradigms. Here we studied how three design factors impact the manifestation of cognition into movement in a Simon task with mouse tracking. We varied the response selection (i.e., with or without clicking), the ratio between hand and mouse cursor movement, and the location of the response boxes. The results show that all design factors can blur or even prevent the manifestation of cognition into movement, as reflected by a reduction in movement consistency and action dynamics, as well as by the adoption of unsuitable movement strategies. We conclude that deliberate and careful design choices in mouse-tracking experiments are crucial to ensuring a continuous manifestation of cognition in movement. We discuss the importance of developing a standard practice in the design of mouse-tracking experiments.

The hidden repertoire of brain dynamics and dysfunction

The purpose of this paper is to describe a framework for the understanding of rules that govern how neural system dynamics are coordinated to produce behavior. The framework, structured flows on manifolds (SFM), posits that neural processes are flows depicting system interactions that occur on relatively low-dimension manifolds, which constrain possible functional configurations. Although this is a general framework, we focus on the application to brain disorders. We first explain the Epileptor, a phenomenological computational model showing fast and slow dynamics, but also a hidden repertoire whose expression is similar to refractory status epilepticus. We suggest that epilepsy represents an innate brain state whose potential may be realized only under certain circumstances. Conversely, deficits from damage or disease processes, such as stroke or dementia, may reflect both the disease process per se and the adaptation of the brain. SFM uniquely captures both scenarios. Finally, we link neuromodulation effects and switches in functional network configurations to fast and slow dynamics that coordinate the expression of SFM in the context of cognition. The tools to measure and model SFM already exist, giving researchers access to the dynamics of neural processes that support the concomitant dynamics of the cognitive and behavioral processes.

Decision-making in sensorimotor control

Skilled sensorimotor interactions with the world result from a series of decision-making processes that determine, on the basis of information extracted during the unfolding sequence of events, which movements to make and when and how to make them. Despite this inherent link between decision-making and sensorimotor control, research into each of these two areas has largely evolved in isolation, and it is only fairly recently that researchers have begun investigating how they interact and, together, influence behaviour. Here, we review recent behavioural, neurophysiological and computational research that highlights the role of decision-making processes in the selection, planning and control of goal-directed movements in humans and nonhuman primates.

Gaze and Motor Traces of Language Processing: Evidence from Autism Spectrum Disorders in Comparison to Typical Controls

We investigated what strategies underlie figurative language processing in two groups of participants distinguished by the presence of a developmental deficit, highly-verbal participants with autism, and control participants without autism in two age ranges each. Individuals with autism spectrum disorder are characterised by impaired social interaction and communication. Even at the high end of the spectrum, where structural language is adequate, difficulties in comprehending non-literal aspects of language are widely attested. The exact causes of these problems are, however, still open to debate. In an interactive sentence-picture matching task participants selected the most suitable image representation of a non-literal figurative expression that matched the target meaning, while their eye-movements and hand movements were being tracked. Our results suggest that individuals with ASD have different processing patterns than typically developing peers when interpreting figurative language, even when they provide the correct answers. Both children with and without autism, and participants with autism display greater uncertainty and competition between alternatives when providing the answer, often reflected in also considering the literal interpretation of the expression against its target figurative meaning. We provide evidence that expression transparency and decomposability play a central role in figurative language processing across all groups.

Biological motion and animacy belief induce similar effects on involuntary shifts of attention

Biological motion is salient to the human visual and motor systems and may be intrinsic to the perception of animacy. Evidence for the salience of visual stimuli moving with trajectories consistent with biological motion comes from studies showing that such stimuli can trigger shifts of attention in the direction of that motion. The present study was conducted to determine whether or not top-down beliefs about animacy can modify the salience of a nonbiologically moving stimulus to the visuomotor system. A nonpredictive cuing task was used in which a white dot moved from a central location toward a left- or right-sided target placeholder. The target randomly appeared at either location 200, 600, or 1,300 ms after the motion onset. Five groups of participants experienced different stimulus conditions: (1) biological motion, (2) inverted biological motion, (3) nonbiological motion, (4) animacy belief (paired with nonbiological motion), and (5) computer-generated belief (paired with nonbiological motion). Analysis of response times revealed that the motion in the biological motion and animacy belief groups, but not in the inverted and nonbiological motion groups, affected processing of the target information. These findings indicate that biological motion is salient to the visual system and that top-down beliefs regarding the animacy of the stimulus can tune the visual and motor systems to increase the salience of nonbiological motion.

Pragmatic processing: An investigation of the (anti-)presuppositions of determiners using mouse-tracking

A presupposition is a condition that has to be met in order for a linguistic expression to be appropriate. The definite determiner (as in the banana) triggers the uniqueness-presupposition that there is a uniquely identifiable banana in the relevant discourse context. The indefinite determiner (as in a banana) is similarly associated with anti-uniqueness (that there are several bananas). Application of the Maximize Presupposition principle to the indefinite determiner suggests that this latter effect results indirectly as an anti-presupposition from considering the uniqueness-presupposition of the definite determiner, which is then negated. This results in increased processing difficulty. We utilized mouse-tracking to compare processing of definite and indefinite determiners when used felicitously and infelicitously in a particular context. First, processing of the indefinite determiner was associated with more processing difficulties compared with the definite determiner. Second, we also observed evidence for an initial temporary activation and evaluation of the uniqueness-presupposition, just as derived from anti-presupposition theory and the Maximize Presupposition principle.

The role of attention in motor control and learning

Performing and learning motor behaviors are fundamental to everyday life. The relations between perceptual input and motor output have been studied and are well understood for simple experimental settings. Recent findings, however, suggest that motor actions also critically depend on cognitive factors; these influences are most notable in complex environments that place high demands on attention and memory. In this review, the role of various aspects of attention in motor control is discussed, focusing on the following points: (1) recent findings concerning interactions between attentional resources and motor skill acquisition, (2) the consistency of attentional states (divided versus focused) and motor memory retrieval, and (3) the locus of attention (internal versus external) and motor performance. These findings collectively highlight the interplay between attention and motor systems, which in turn has practical implications for developing and improving motor training and rehabilitation programs.

Effect of Aging on Change of Intention

Decision making often requires making arbitrary choices ("picking") between alternatives that make no difference to the agent, that are equally desirable, or when the potential reward is unknown. Using event-related potentials we tested the effect of age on this common type of decision making. We compared two age groups: ages 18-25, and ages 41-67 on a masked-priming paradigm while recording EEG and EMG. Participants pressed a right or left button following either an instructive arrow cue or a neutral free-choice picking cue, both preceded by a masked arrow or neutral prime. The prime affected the behavior on the Instructed and the Free-choice picking conditions both in the younger and older groups. Moreover, electrophysiological "Change of Intention" (ChoI) was observed via lateralized readiness potential (LRP) in both age groups - the polarity of the LRP indicated first preparation to move the primed hand and then preparation to move the other hand. However, the older participants were more conservative in responding to the instructive cue, exhibiting a speed-accuracy trade-off, with slower response times, less errors in incongruent trials, and reduced probability of EMG activity in the non-responding hand. Additionally, "Change of Intention" was observed in both age groups in slow RT trials with a neutral prime as a result of an endogenous early intention to respond in a direction opposite the eventual instructing arrow cue. We conclude that the basic behavioral and electrophysiological signatures of implicit ChoI are common to a wide range of ages. However, older subjects, despite showing a similar dynamic decision trajectory as younger adults, are slower, more prudent and finalize the decision making process before letting the information affect the peripheral motor system. In contrast, the flow of information in younger subjects occurs in parallel to the decision process.

The validity and consistency of continuous joystick response in perceptual decision-making

A computer joystick is an efficient and cost-effective response device for recording continuous movements in psychological experiments. Movement trajectories and other measures from continuous responses have expanded the insights gained from discrete responses (e.g., button presses) by providing unique information about how cognitive processes unfold over time. However, few studies have evaluated the validity of joystick responses with reference to conventional key presses, and how response modality can affect cognitive processes. Here we systematically compared human participants' behavioral performance of perceptual decision-making when they responded with either joystick movements or key presses in a four-alternative motion discrimination task. We found evidence that the response modality did not affect raw behavioral measures, including decision accuracy and mean response time, at the group level. Furthermore, to compare the underlying decision processes between the two response modalities, we fitted a drift-diffusion model of decision-making to individual participants' behavioral data. Bayesian analyses of the model parameters showed no evidence that switching from key presses to continuous joystick movements modulated the decision-making process. These results supported continuous joystick actions as a valid apparatus for continuous movements, although we highlight the need for caution when conducting experiments with continuous movement responses.

Beyond reach: Do symmetric changes in motor costs affect decision making? A registered report

Executing an important decision can be as easy as moving a mouse cursor or reaching towards the preferred option with a hand. But would we decide differently if choosing required walking a few steps towards an option? More generally, is our preference invariant to the means and motor costs of reporting it? Previous research demonstrated that asymmetric motor costs can nudge the decision-maker towards a less costly option. However, virtually all traditional decision-making theories predict that increasing motor costs symmetrically for all options should not affect choice in any way. This prediction is disputed by the theory of embodied cognition, which suggests that motor behavior is an integral part of cognitive processes, and that motor costs can affect our choices. In this registered report, we investigated whether varying motor costs can affect response dynamics and the final choices in an intertemporal choice task: choosing between a readily available small reward and a larger but delayed reward. Our study compared choices reported by moving a computer mouse cursor towards the preferred option with the choices executed via a more motor costly walking procedure. First, we investigated whether relative values of the intertemporal choice options affect walking trajectories in the same way as they affect mouse cursor dynamics. Second, we tested a hypothesis that, in the walking condition, increased motor costs of a preference reversal would decrease the number of changes-of-mind and therefore increase the proportion of impulsive, smaller-but-sooner choices. We confirmed the hypothesis that walking trajectories reflect covert dynamics of decision making, and rejected the hypothesis that increased motor costs of responding affect decisions in an intertemporal choice task. Overall, this study contributes to the empirical basis enabling the decision-making theories to address the complex interplay between cognitive and motor processes.

Memory for retinotopic locations is more accurate than memory for spatiotopic locations, even for visually guided reaching

To interact successfully with objects, we must maintain stable representations of their locations in the world. However, their images on the retina may be displaced several times per second by large, rapid eye movements. A number of studies have demonstrated that visual processing is heavily influenced by gaze-centered (retinotopic) information, including a recent finding that memory for an object's location is more accurate and precise in gaze-centered (retinotopic) than world-centered (spatiotopic) coordinates (Golomb & Kanwisher, 2012b). This effect is somewhat surprising, given our intuition that behavior is successfully guided by spatiotopic representations. In the present experiment, we asked whether the visual system may rely on a more spatiotopic memory store depending on the mode of responding. Specifically, we tested whether reaching toward and tapping directly on an object's location could improve memory for its spatiotopic location. Participants performed a spatial working memory task under four conditions: retinotopic vs. spatiotopic task, and computer mouse click vs. touchscreen reaching response. When participants responded by clicking with a mouse on the screen, we replicated Golomb & Kanwisher's original results, finding that memory was more accurate in retinotopic than spatiotopic coordinates and that the accuracy of spatiotopic memory deteriorated substantially more than retinotopic memory with additional eye movements during the memory delay. Critically, we found the same pattern of results when participants responded by using their finger to reach and tap the remembered location on the monitor. These results further support the hypothesis that spatial memory is natively retinotopic; we found no evidence that engaging the motor system improves spatiotopic memory across saccades.

How Not to Fall for the White Bear: Combined Frequency and Recency Manipulations Diminish Negation Effects on Overt Behavior

Processing negated mental representations comes with a price: Not only are negations harder to resolve than affirmative statements, but they may even invoke ironic effects, producing the exact opposite of the intended outcome. Negation effects also behave ironically when subjected to high-frequency training; when they are confronted often, the difficulty to process negations strangely increases. Here we show that negation effects can be mitigated under certain circumstances. Based on models of cognitive control and conflict adaptation, we hypothesized that negation effects diminish when two criteria are met: negations have to be resolved not only frequently, but also just recently. We confirmed this prediction in two experiments by using an innovative, two-dimensional finger tracking design, in which we measured the influence of the original semantic content during negation processing via temporal and spatial measures. Negation effects were present throughout the experiment, but were reduced after recent negations, particularly during or after a high-frequency negation context. The combined influence of frequency and recency thus seems to be the most successful and promising attempt to mitigate ironic negation effects on overt behavior.

Sketching the Power of Machine Learning to Decrypt a Neural Systems Model of Behavior

Uncovering brain-behavior mechanisms is the ultimate goal of neuroscience. A formidable amount of discoveries has been made in the past 50 years, but the very essence of brain-behavior mechanisms still escapes us. The recent exploitation of machine learning (ML) tools in neuroscience opens new avenues for illuminating these mechanisms. A key advantage of ML is to enable the treatment of large data, combing highly complex processes. This essay provides a glimpse of how ML tools could test a heuristic neural systems model of motivated behavior, the triadic neural systems model, which was designed to understand behavioral transitions in adolescence. This essay previews analytic strategies, using fictitious examples, to demonstrate the potential power of ML to decrypt the neural networks of motivated behavior, generically and across development. Of note, our intent is not to provide a tutorial for these analyses nor a pipeline. The ultimate objective is to relate, as simply as possible, how complex neuroscience constructs can benefit from ML methods for validation and further discovery. By extension, the present work provides a guide that can serve to query the mechanisms underlying the contributions of prefrontal circuits to emotion regulation. The target audience concerns mainly clinical neuroscientists. As a caveat, this broad approach leaves gaps, for which references to comprehensive publications are provided.

Stuck at the starting line: How the starting procedure influences mouse-tracking data

Mouse-tracking is an increasingly popular method to trace cognitive processes. As is common for a novel method, the exact methodological procedures employed in an individual study are still relatively idiosyncratic and the effects of different methodological setups on mouse-tracking measures have not been explored so far. Here, we study the impact of one commonly occurring methodological variation, namely whether participants have to initiate their mouse movements to trigger stimulus presentation (dynamic starting condition) or whether the stimulus is presented automatically after a fixed delay and participants can freely decide when to initiate their movements (static starting condition). We compared data from a previous study in which participants performed a mouse-tracking version of a Simon task with a dynamic starting condition to data from a new study that employed a static starting condition in an otherwise identical setup. Results showed reliable Simon effects and Congruency Sequence effects on response time (RT) and discrete trial-level mouse-tracking measures (i.e., average deviation) in both starting conditions. In contrast, within-trial continuous measures (i.e., extracted temporal segments) were weaker and occurred in a more temporally compressed way in the static compared to the dynamic starting condition. This was in line with generally less consistent movements within and across participants in the static compared to the dynamic condition. Our results suggest that studies that use within-trial continuous measures to assess dynamic aspects of mouse movements should apply dynamic starting procedures to enhance the leakage of cognitive processing into the mouse movements.

Mouse tracking as a window into decision making

Mouse tracking promises to be an efficient method to investigate the dynamics of cognitive processes: It is easier to deploy than eyetracking, yet in principle it is much more fine-grained than looking at response times. We investigated these claimed benefits directly, asking how the features of decision processes-notably, decision changes-might be captured in mouse movements. We ran two experiments, one in which we explicitly manipulated whether our stimuli triggered a flip in decision, and one in which we replicated more ecological, classical mouse-tracking results on linguistic negation (Dale & Duran, Cognitive Science, 35, 983-996, 2011). We concluded, first, that spatial information (mouse path) is more important than temporal information (speed and acceleration) for detecting decision changes, and we offer a comparison of the sensitivities of various typical measures used in analyses of mouse tracking (area under the trajectory curve, direction flips, etc.). We do so using an "optimal" analysis of our data (a linear discriminant analysis explicitly trained to classify trajectories) and see what type of data (position, speed, or acceleration) it capitalizes on. We also quantify how its results compare with those based on more standard measures.

Target Uncertainty During Motor Decision-Making: The Time Course of Movement Variability Reveals the Effect of Different Sources of Uncertainty on the Control of Reaching Movements

The processes underlying motor decision-making have recently caught considerable amount of scientific attention, focusing on the integration of empirical evidence from sensorimotor control research with psychological theories and computational models on decision-making. Empirical studies on motor decision-making suggest that the kinematics of goal-directed reaching movements are sensitive to the level of target uncertainty during movement planning. However, the source of uncertainty as a relevant factor influencing the process of motor decision-making has not been sufficiently considered, yet. In this study, we test the assumption that the source of target uncertainty has an effect on motor decision-making, which can be proven by analyzing movement variability during the time course of movement execution. Ten healthy young adults performed three blocks with 66 trials of goal-directed reaching movements in each block, across which the source and level of reach target uncertainty at movement onset were manipulated (“no uncertainty”, “extrinsic uncertainty”, and “intrinsic uncertainty”). Fingertip position of the right index finger was recorded using an optical motion tracking system. Standard kinematic measures (i.e., path length and movement duration) as well as variability of fingertip position across the time course of movement execution and at movement end were analyzed. In line with previous studies, we found that a high level of extrinsic target uncertainty leads to increased overall movement duration, which could be attributed to increased path length in this condition, as compared to intrinsic and no target uncertainty (all p < 0.001). Movement duration and path length did not show any differences between the latter two conditions. However, the time course analysis of movement variability revealed significant differences between these two conditions, with increased variability of fingertip position in the presence of intrinsic target uncertainty (Condition × Sampling point: p = 0.01), though considerably less than under high extrinsic target uncertainty (p ≤ 0.001). These findings suggest that both the level and source of uncertainty have a significant effect on the processing of potential action plans during motor decision-making, which can be revealed through the analysis of the time course of movement variability at the end-effector level.

The time for action is at hand

The science of mental life and behavior has paid scant attention to the means by which mental life is translated into physical behavior. Why this is so was the topic of a 2005 American Psychologist article whose main title was "The Cinderella of Psychology." In the present article, we briefly review some of the reasons why motor control was relegated to the sidelines of psychology. Then we point to work showing that experimental psychologists have much to contribute to research on action generation. We focus on studies showing that actions are generated in a way that, at least by default, minimize changes between successive actions. The method is computationally as well as physically economical but also requires consideration of costs, including costs of different kinds. How such costs are compared is discussed in the next section. The final section offers comments about the future of psychologically focused action research. Two additional themes of the review concern methods for studying action generation. First, much can be learned through naturalistic observation. Second, subsequent experiments, designed to check naturalistic observations, can use very simple equipment and procedures. This can make the study of action generation easy to pursue in the psychology laboratory.

Decision and navigation in mouse parietal cortex

Posterior parietal cortex (PPC) has been implicated in navigation, in the control of movement, and in visually-guided decisions. To relate these views, we measured activity in PPC while mice performed a virtual navigation task driven by visual decisions. PPC neurons were selective for specific combinations of the animal's spatial position and heading angle. This selectivity closely predicted both the activity of individual PPC neurons, and the arrangement of their collective firing patterns in choice-selective sequences. These sequences reflected PPC encoding of the animal's navigation trajectory. Using decision as a predictor instead of heading yielded worse fits, and using it in addition to heading only slightly improved the fits. Alternative models based on visual or motor variables were inferior. We conclude that when mice use vision to choose their trajectories, a large fraction of parietal cortex activity can be predicted from simple attributes such as spatial position and heading.

Modeling Mindsets with Kalman Filter

Mathematical models have played an essential role in interface design. This study focused on “mindsets”—people’s tacit beliefs about attributes—and investigated the extent to which: (1) mindsets can be extracted in a motion trajectory in target selection, and (2) a dynamic state-space model, such as the Kalman filter, helps quantify mindsets. Participants were experimentally manipulated to hold fixed or growth mindsets in a “mock” memory test, and later performed a concept-learning task in which the movement of the computer cursor was recorded in every trial. By inspecting motion trajectories of the cursor, we observed clear disparities in the impact of mindsets; participants who were induced with a fixed mindset moved the cursor faster as compared to those who were induced with a growth mindset. To examine further the mechanism of this influence, we fitted a Kalman filter model to the trajectory data; we found that system-level error-covariance in the Kalman filter model could effectively separate motion trajectories gleaned from the two mindset conditions. Taken together, results from the experiment suggest that people’s mindsets can be captured in motor trajectories in target selection and the Kalman filter helps quantify mindsets. It is argued that people’s personality, attitude, and mindset are embodied in motor behavior underlying target selection and these psychological variables can be studied mathematically with a feedback control system.

Human VMPFC encodes early signatures of confidence in perceptual decisions

Choice confidence, an individual’s internal estimate of judgment accuracy, plays a critical role in adaptive behaviour, yet its neural representations during decision formation remain underexplored. Here, we recorded simultaneous EEG-fMRI while participants performed a direction discrimination task and rated their confidence on each trial. Using multivariate single-trial discriminant analysis of the EEG, we identified a stimulus-independent component encoding confidence, which appeared prior to subjects’ explicit choice and confidence report, and was consistent with a confidence measure predicted by an accumulation-to-bound model of decision-making. Importantly, trial-to-trial variability in this electrophysiologically-derived confidence signal was uniquely associated with fMRI responses in the ventromedial prefrontal cortex (VMPFC), a region not typically associated with confidence for perceptual decisions. Furthermore, activity in the VMPFC was functionally coupled with regions of the frontal cortex linked to perceptual decision-making and metacognition. Our results suggest that the VMPFC holds an early confidence representation arising from decision dynamics, preceding and potentially informing metacognitive evaluation.

While waiting to cross the road on a foggy morning, you see a shape in the distance that appears to be an approaching car. How do you decide if it is safe to cross? We often have to make important decisions about the world based on imperfect information. What guides our subsequent actions in these situations is a sense of accuracy, or confidence, that we associate with our initial judgments. You would not step off the kerb if you were only 10% confident the car was a safe distance away. But how, when, and where in the brain does such confidence emerge?

Gherman and Philiastides examined how brain activity relates to confidence during the early stages of decision-making, that is, before people have explicitly committed to a particular choice. Healthy volunteers were asked to judge the direction in which dots were moving across a screen. They then had to rate how confident they were in their decision. Two techniques – EEG and fMRI – tracked their brain activity during the task. EEG uses scalp electrodes to reveal when and how electrical activity is changing inside the brain, while fMRI, a type of brain scan, shows where these changes in brain activity occur. Used together, the two techniques provide a greater understanding of brain activity than either used alone.

Activity in multiple regions of the brain correlated with confidence at different stages of the task. Certain brain networks showed confidence-related activity while the volunteers tried to judge the direction of movement, and others were engaged when volunteers made their confidence ratings. However, activity in only one area reliably indicated how confident the volunteers felt before they had made their choice. This area, the ventromedial prefrontal cortex, also helps process rewards. This suggests that feelings of confidence early in the decision-making process could guide our behaviour by virtue of being rewarding.

Many brain disorders – including depression, schizophrenia and Parkinson's disease – compromise decision-making. Patients show changes in accuracy, response times, and in their ability to accurately evaluate their decisions. The methods used in the current study could help reveal the neural changes that cause these impairments. This could lead to new methods to diagnose and predict cognitive deficits, and new ways to treat them at an earlier stage.

Doing psychological science by hand

Over the past decade, mouse-tracking in choice tasks has become a popular method across psychological science. This method exploits hand movements as a measure of multiple response activations that can be tracked continuously over hundreds of milliseconds. Whereas early mouse-tracking research focused on specific debates, researchers have realized the methodology has far broader theoretical value. This more recent work demonstrates that mouse-tracking is a widely applicable measure across the field, capable of exposing the micro-structure of real-time decisions including their component processes and millisecond-resolution time-course in ways that inform theory. In the article, recent advances in the mouse-tracking approach are described, and comparisons with the gold standard measure of reaction time and other temporally-sensitive methodologies are provided. Future directions, including mapping to neural representations with brain-imaging and ways to improve our theoretical understanding of mouse-tracking methodology, are discussed.

Weighted integration of short-term memory and sensory signals in the oculomotor system

Oculomotor behaviors integrate sensory and prior information to overcome sensory-motor delays and noise. After much debate about this process, reliability-based integration has recently been proposed and several models of smooth pursuit now include recurrent Bayesian integration or Kalman filtering. However, there is a lack of behavioral evidence in humans supporting these theoretical predictions. Here, we independently manipulated the reliability of visual and prior information in a smooth pursuit task. Our results show that both smooth pursuit eye velocity and catch-up saccade amplitude were modulated by visual and prior information reliability. We interpret these findings as the continuous reliability-based integration of a short-term memory of target motion with visual information, which support modeling work. Furthermore, we suggest that saccadic and pursuit systems share this short-term memory. We propose that this short-term memory of target motion is quickly built and continuously updated, and constitutes a general building block present in all sensorimotor systems.

Tracking the Within-Trial, Cross-Trial, and Developmental Dynamics of Cognitive Control: Evidence From the Simon Task

Six- to 8-year-olds, 10- to 12-year-olds, and adults (N = 108) performed the Simon task by reaching to targets on a digital display. The spatial and temporal characteristics of their movements were used to assess how two key processes underlying cognitive control-a threshold adjustment process and a controlled selection process-unfold over the course of a response (within-trial dynamics), are modulated by recent experience (cross-trial dynamics), and contribute to age-related gains in control (developmental dynamics). The results indicate that the controlled selection process undergoes a more protracted development than the threshold adjustment process. The results also shed light on a prominent debate concerning the cross-trial dynamics of control by supporting the feature integration account over the conflict adaptation account.

Deconstructing the Gratton effect: Targeting dissociable trial sequence effects in children, pre-adolescents, and adults

The Gratton effect refers to the observation that performance on congruency tasks is often enhanced when the congruency of the current trial matches that of the previous trial. This effect has been at the center of recent debates in the literature on cognitive control as researchers have sought to identify the cognitive and neural underpinnings of the effect. Here, we use a technique known as reach tracking to demonstrate that the Gratton effect originally observed in the flanker task is not a singular effect but the result of two separate trial sequence effects that impact dissociable processes underlying cognitive control. Further, our results indicate that these dissociable processes follow divergent developmental trajectories across childhood, pre-adolescence, and adulthood. Taken together, these findings suggest that manual dynamics can be used to disentangle how key processes underlying cognitive control contribute to the response time effects observed across a wide range of cognitive tasks and age groups.

Distinct Patterns of Cognitive Conflict Dynamics in Promise Keepers and Promise Breakers

On a daily basis, we see how different people can be in keeping or breaking a given promise. However, we know very little about the cognitive conflict dynamics that underlie the decision to keep or break a promise and whether this is shaped by inter-individual variability. In order to fill this gap, we applied an ecologically valid promise decision task with real monetary consequences for all involved interaction partners and used mouse tracking to identify the dynamic, on-line cognitive processes that underlie the decision to keep or break a promise. Our findings revealed that on average, the process of breaking a promise is associated with largely curved mouse trajectories, while the process of keeping a promise was not, indicating that breaking a promise is associated with a larger conflict. Interestingly, however, this conflict pattern was strongly shaped by individual differences. Individuals who always kept their promises did not show any signs of conflict (i.e., straight mouse trajectories), indicating that they were not tempted by the monetary benefits associated with breaking the promise. In contrast, individuals who did not always keep their promise exhibited a large conflict (i.e., curved mouse trajectories), irrespective of whether they broke or kept their promise. A possible interpretation of these findings is that these individuals were always tempted by the unchosen decision option – the desire to act in a fair manner when breaking the promise and the monetary benefits when keeping the promise. This study provides the first piece of evidence that there are substantial inter-individual differences in cognitive conflict dynamics that underlie the decision to keep or break promises and that mouse tracking is able to illuminate important insights into individual differences in complex human’s decision processes.

The "Motor" in Implicit Motor Sequence Learning: A Foot-stepping Serial Reaction Time Task

This protocol describes a modified serial reaction time (SRT) task used to study implicit motor sequence learning. Unlike the classic SRT task that involves finger-pressing movements while sitting, the modified SRT task requires participants to step with both feet while maintaining a standing posture. This stepping task necessitates whole body actions that impose postural challenges. The foot-stepping task complements the classic SRT task in several ways. The foot-stepping SRT task is a better proxy for the daily activities that require ongoing postural control, and thus may help us better understand sequence learning in real-life situations. In addition, response time serves as an indicator of sequence learning in the classic SRT task, but it is unclear whether response time, reaction time (RT) representing mental process, or movement time (MT) reflecting the movement itself, is a key player in motor sequence learning. The foot-stepping SRT task allows researchers to disentangle response time into RT and MT, which may clarify how motor planning and movement execution are involved in sequence learning. Lastly, postural control and cognition are interactively related, but little is known about how postural control interacts with learning motor sequences. With a motion capture system, the movement of the whole body (e.g., the center of mass (COM)) can be recorded. Such measures allow us to reveal the dynamic processes underlying discrete responses measured by RT and MT, and may aid in elucidating the relationship between postural control and the explicit and implicit processes involved in sequence learning. Details of the experimental set-up, procedure, and data processing are described. The representative data are adopted from one of our previous studies. Results are related to response time, RT, and MT, as well as the relationship between the anticipatory postural response and the explicit processes involved in implicit motor sequence learning.