Track It to Crack It: Dissecting Processing Stages with Finger Tracking

Embodied sequential sampling models and dynamic neural fields for decision-making: Why hesitate between two when a continuum is the answer

As two alternative options in a forced choice task are separated by design, two classes of computational models of decision-making have thrived independently in the literature for nearly five decades. While sequential sampling models (SSM) focus on response times and keypresses in binary decisions in experimental paradigms, dynamic neural fields (DNF) focus on continuous sensorimotor dimensions and tasks found in perception and robotics. Recent attempts have been made to address limitations in their application to other domains, but strong similarities and compatibility between prominent models from both classes were hardly considered. This article is an attempt at bridging the gap between these classes of models, and simultaneously between disciplines and paradigms relying on binary or continuous responses. A unifying formulation of representative SSM and DNF equations is proposed, varying the number of units which interact and compete to reach a decision. The embodiment of decisions is also considered by coupling cognitive and sensorimotor processes, enabling the model to generate decision trajectories at trial level. The resulting mechanistic model is therefore able to target different paradigms (forced choices or continuous response scales) and measures (final responses or dynamics). The validity of the model is assessed statistically by fitting empirical distributions obtained from human participants in moral decision-making mouse-tracking tasks, for which both dichotomous and nuanced responses are meaningful. Comparing equations at the theoretical level, and model parametrizations at the empirical level, the implications for psychological decision-making processes, as well as the fundamental assumptions and limitations of models and paradigms are discussed.

Conflict in moral and nonmoral decision making: an empirical study coupled with a computational model

While moral psychology research has extensively studied decision making using moral dilemmas, such high-conflict situations may not fully represent all moral decisions. Moreover, most studies on the effect of conflict have focused on nonmoral decisions, and it is unclear how it applies to the moral realm. The present mixed-method research investigates how conflict impacts moral compared to nonmoral decision making. In a preregistered empirical study ( N = 42 ), participants made moral and nonmoral decisions with varying levels of conflict while their mouse trajectories were recorded. Results indicate that moral decisions were more stable in the presence of conflict, while still seeking compromise. In addition, decisions were more affected when conflict got higher. Mouse-tracking data further indicate that some factors are impacting the decision process earlier than others, supporting the relevance of tracing methods to dig into finer-grained decision dynamics. We also present a computational model that aims to capture decision mechanisms and how conflict and morality influence decision making. The model uses dynamic neural fields coupled with sensorimotor control to map a continuous decision space. Two model versions were compared: one with greater perceptual weight for moral information, and another with earlier processing of moral versus nonmoral information. The simulated data more successfully reproduced empirical patterns for the second version, thus providing insights into the underlying decision processes for both moral and nonmoral decisions, in the presence of conflict or not.

Focusing on Future Consequences Enhances Self-Controlled Dietary Choices

Self-controlled dietary decisions, i.e., choosing a healthier food over a tastier one, are a major challenge for many people. Despite the potential profound consequences of frequent poor choices, maintaining a healthy diet proves challenging. This raises the question of how to facilitate self-controlled food decisions to promote healthier choices. The present study compared the influence of implicit and explicit information on food choices and their underlying decision processes. Participants watched two video clips as an implicit manipulation to induce different mindsets. Instructions to focus on either the short-term or long-term consequences of choices served as an explicit manipulation. Participants performed a binary food choice task, including foods with different health and taste values. The choice was made using a computer mouse, whose trajectories we used to calculate the influence of the food properties. Instruction to focus on long-term consequences compared to short-term consequences increased the number of healthy choices, reduced response times for healthy decisions, and increased the influence of health aspects during the decision-making process. The effect of video manipulation showed greater variability. While focusing on long-term consequences facilitated healthy food choices and reduced the underlying decision conflict, the current mindset appeared to have a minor influence.

Cognitive Foundations of Early Mathematics: Investigating the Unique Contributions of Numerical, Executive Function, and Spatial Skills

There is an emerging consensus that numerical, executive function (EF), and spatial skills are foundational to children's mathematical learning and development. Moreover, each skill has been theorized to relate to mathematics for different reasons. Thus, it is possible that each cognitive construct is related to mathematics through distinct pathways. The present study tests this hypothesis. One-hundred and eighty 4- to 9-year-olds (Mage = 6.21) completed a battery of numerical, EF, spatial, and mathematics measures. Factor analyses revealed strong, but separable, relations between children's numerical, EF, and spatial skills. Moreover, the three-factor model (i.e., modelling numerical, EF, and spatial skills as separate latent variables) fit the data better than a general intelligence (g-factor) model. While EF skills were the only unique predictor of number line performance, spatial skills were the only unique predictor of arithmetic (addition) performance. Additionally, spatial skills were related to the use of more advanced addition strategies (e.g., composition/decomposition and retrieval), which in turn were related to children's overall arithmetic performance. That is, children's strategy use fully mediated the relation between spatial skills and arithmetic performance. Taken together, these findings provide new insights into the cognitive foundations of early mathematics, with implications for assessment and instruction moving forward.

Cursor movements to targets labelled "stop": a kinematic analysis

The label 'Stop' potentially generates conflict-signifying important corrective action, or a warning not to touch. To examine potential conflict between an incongruent label (i.e. STOP) and an imperative command (i.e. MOVE!), 18 participants used a computer mouse to move a crosshair cursor to targets with superimposed labels. Trials systematically varied Imperative (blank or MOVE!), Label (+GO + or STOP) and movement Distance. Kinematic analyses examined response latency, movement duration and accuracy. Incongruent labels had little impact upon response latencies, but they affected cursor deceleration and the variability of cursor placement. Although reading is assumed to be obligatory, the impact of written labels is not immediate, instead affecting cursor deceleration. Indeed, responses to controls labelled STOP were less accurate than those labelled + GO+. As labelled interfaces can create error versus command confusions, enhancing the discriminability of controls to afford more obvious visible cues as to method of use is recommended. Practitioner summary: Emergency stop and shutdown controls can cause response conflict as their labels signify both urgent corrective actions and 'don't touch'. Response conflict caused by confusing superimposed labels is resolved as cursors near the target control and may result in reduced movement accuracy. Prior warnings may influence resolution of response conflict. Abbreviations: Hz: Hertz; M: Mean; ms: millisecond; mm: millimetre; S: second; SD: Standard Deviation; SE: Standard Error; USB: Universal Serial Bus.

Hand Motions Reveal Attentional Status and Subliminal Semantic Processing: A Mouse-Tracking Technique

Theories of embodied cognition suggest that hand motions and cognition are closely interconnected. An emerging technique of tracking how participants move a computer mouse (i.e., the mouse-tracking technique) has shown advantages over the traditional response time measurement to detect implicit cognitive conflicts. Previous research suggests that attention is essential for subliminal processing to take place at a semantic level. However, this assumption is challenged by evidence showing the presence of subliminal semantic processing in the near-absence of attention. The inconsistency of evidence could stem from the insufficient sensitivity in the response time measurement. Therefore, we examined the role of attention in subliminal semantic processing by analyzing participants' hand motions using the mouse-tracking technique. The results suggest that subliminal semantic processing is not only enhanced by attention but also occurs when attention is disrupted, challenging the necessity of facilitated top-down attention for subliminal semantic processing, as claimed by a number of studies. In addition, by manipulating the color of attentional cues, our experiment shows that the cue color per se could influence participants' response patterns. Overall, the current study suggests that attentional status and subliminal semantic processing can be reliably revealed by temporal-spatial features extracted from cursor motion trajectories.

Rethinking Vision and Action

Action is an important arbitrator as to whether an individual or a species will survive. Yet, action has not been well integrated into the study of psychology. Action or motor behavior is a field apart. This is traditional science with its need for specialization. The sequence in a typical laboratory experiment of see → decide → act provides the rationale for broad disciplinary categorizations. With renewed interest in action itself, surprising and exciting anomalous findings at odds with this simplified caricature have emerged. They reveal a much more intimate coupling of vision and action, which we describe. In turn, this prompts us to identify and dwell on three pertinent theories deserving of greater notice.

Opening Questions in Visual Working Memory

In this reflective piece on visual working memory, I depart from the laboriously honed skills of writing a review. Instead of integrating approaches, synthesizing evidence, and building a cohesive perspective, I scratch my head and share niggles and puzzlements. I expose where my scholarship and understanding are stumped by findings and standard views in the literature.

Moving beyond response times with accessible measures of manual dynamics

Button-press measures of response time (RT) and accuracy have long served a central role in psychological research. However, RT and accuracy provide limited insight into how cognitive processes unfold over time. To address this limitation, researchers have used hand-tracking techniques to investigate how cognitive processes unfold over the course of a response, are modulated by recent experience, and function across the lifespan. Despite the efficacy of these techniques for investigating a wide range of psychological phenomena, widespread adoption of hand-tracking techniques within the field is hindered by a range of factors, including equipment costs and the use of specialized software. Here, we demonstrate that the behavioral dynamics previously observed with specialized motion-tracking equipment in an Eriksen flanker task can be captured with an affordable, portable, and easy-to-assemble response box. Six-to-eight-year-olds and adults (N = 90) completed a computerized version of the flanker task by pressing and holding a central button until a stimulus array appeared. Participants then responded by releasing the central button and reaching to press one of two response buttons. This method allowed RT to be separated into initiation time (when the central button was released) and movement time (time elapsed between initiation and completion of the response). Consistent with previous research using motion-tracking techniques, initiation times and movement times revealed distinct patterns of effects across trials and between age groups, indicating that the method used in the current study presents a simple solution for researchers from across the psychological and brain sciences looking to move beyond RTs.

Understanding neural signals of post-decisional performance monitoring: An integrative review

Performance monitoring is a key cognitive function, allowing to detect mistakes and adapt future behavior. Post-decisional neural signals have been identified that are sensitive to decision accuracy, decision confidence and subsequent adaptation. Here, we review recent work that supports an understanding of late error/confidence signals in terms of the computational process of post-decisional evidence accumulation. We argue that the error positivity, a positive-going centro-parietal potential measured through scalp electrophysiology, reflects the post-decisional evidence accumulation process itself, which follows a boundary crossing event corresponding to initial decision commitment. This proposal provides a powerful explanation for both the morphological characteristics of the signal and its relation to various expressions of performance monitoring. Moreover, it suggests that the error positivity -a signal with thus far unique properties in cognitive neuroscience - can be leveraged to furnish key new insights into the inputs to, adaptation, and consequences of the post-decisional accumulation process.

Modulation of visually guided action by the image and familiar sizes of real-world objects

In daily life, two aspects of real-world object size perception—the image size of an object and its familiar size in the real world—are highly correlated. Thus, whether these two aspects of object size differently affect goal-directed action (e.g., manual pointing) and how have scarcely been examined. Here, participants reached to touch one of two simultaneously presented objects based on either their image or familiar size, which could be congruent or incongruent (e.g., a rubber duck presented as smaller and larger than a boat, respectively). We observed that when pointing to target objects in the incongruent conditions, participants’ movements were slower and were more curved toward the incorrect object compared with the movements in the congruent conditions. By comparing performance in the congruent and incongruent conditions, we concluded that both image size and familiar size influenced action even when task irrelevant, indicating that both are processed automatically (Konkle & Oliva, 2012a). Image size, however, showed influence earlier in the course of movements and more robustly overall than familiar size. We additionally found that greater relative familiar size differences mitigated the impact of image size processing and increased the impact of familiar size processing on pointing movements. Overall, our data suggest that image size and familiar size perception interact both with each other and with visually guided action, but that the relative contributions of each are unequal and vary based on task demands.

Revealing the effects of temporal orienting of attention on response conflict using continuous movements

Orienting attention in time enables us to prepare for forthcoming perception and action (e.g., estimating the duration of a yellow traffic light when driving). While temporal orienting can facilitate performance on simple tasks, its influence on complex tasks involving response conflict is unclear. Here, we adapted the flanker paradigm to a choice-reaching task where participants used a computer mouse to reach to the left or right side of the screen, as indicated by the central arrow presented with either the congruent or incongruent flankers. We assessed the effects of temporal orienting by manipulating goal-driven temporal expectation (using probabilistic variations in target timing) and stimulus-driven temporal priming (using sequential repetitions versus switches in target timing). We tested how temporal orienting influenced the dynamics of response conflict resolution. Recent choice-reaching studies have indicated that under response conflict, delayed movement initiation captures the response threshold adjustment process, whereas increased curvature toward the incorrect response captures the degree of coactivation of the response alternatives during the controlled response selection process. Both temporal expectation and priming reduced the initiation latency regardless of response conflict, suggesting that both lowered response thresholds independently of response conflict. Notably, temporal expectation, but not temporal priming, increased the curvature toward the incorrect response on incongruent trials. These results suggest that temporal orienting generally increases motor preparedness, but goal-driven temporal orienting particularly interferes with response conflict resolution, likely through its influence on response thresholds. Overall, our study highlights the interplay between temporal orienting and cognitive control in goal-directed action.

The cone method: Inferring decision times from single-trial 3D movement trajectories in choice behavior

Ongoing goal-directed movements can be rapidly adjusted following new environmental information, e.g., when chasing pray or foraging. This makes movement trajectories in go-before-you-know decision-making a suitable behavioral readout of the ongoing decision process. Yet, existing methods of movement analysis are often based on statistically comparing two groups of trial-averaged trajectories and are not easily applied to three-dimensional data, preventing them from being applicable to natural free behavior. We developed and tested the cone method to estimate the point of overt commitment (POC) along a single two- or three-dimensional trajectory, i.e., the position where the movement is adjusted towards a newly selected spatial target. In Experiment 1, we established a “ground truth” data set in which the cone method successfully identified the experimentally constrained POCs across a wide range of all but the shallowest adjustment angles. In Experiment 2, we demonstrate the power of the method in a typical decision-making task with expected decision time differences known from previous findings. The POCs identified by cone method matched these expected effects. In both experiments, we compared the cone method’s single trial performance with a trial-averaging method and obtained comparable results. We discuss the advantages of the single-trajectory cone method over trial-averaging methods and possible applications beyond the examples presented in this study. The cone method provides a distinct addition to existing tools used to study decisions during ongoing movement behavior, which we consider particularly promising towards studies of non-repetitive free behavior.

A Hierarchical Attractor Network Model of perceptual versus intentional decision updates

Changes of Mind are a striking example of our ability to flexibly reverse decisions and change our own actions. Previous studies largely focused on Changes of Mind in decisions about perceptual information. Here we report reversals of decisions that require integrating multiple classes of information: 1) Perceptual evidence, 2) higher-order, voluntary intentions, and 3) motor costs. In an adapted version of the random-dot motion task, participants moved to a target that matched both the external (exogenous) evidence about dot-motion direction and a preceding internally-generated (endogenous) intention about which colour to paint the dots. Movement trajectories revealed whether and when participants changed their mind about the dot-motion direction, or additionally changed their mind about which colour to choose. Our results show that decision reversals about colour intentions are less frequent in participants with stronger intentions (Exp. 1) and when motor costs of intention pursuit are lower (Exp. 2). We further show that these findings can be explained by a hierarchical, multimodal Attractor Network Model that continuously integrates higher-order voluntary intentions with perceptual evidence and motor costs. Our model thus provides a unifying framework in which voluntary actions emerge from a dynamic combination of internal action tendencies and external environmental factors, each of which can be subject to Change of Mind.

Reducing variance or helping the poorest? A mouse tracking approach to investigate cognitive bases of inequality aversion in resource allocation

Humans dislike unequal allocations. Although often conflated, such 'inequality-averse' preferences are separable into two elements: egalitarian concern about the variance and maximin concern about the poorest (maximizing the minimum). Recent research has shown that the maximin concern operates more robustly in allocation decisions than the egalitarian concern. However, the real-time cognitive dynamics of allocation decisions are still unknown. Here, we examined participants' choice behaviour with high temporal resolution using a mouse-tracking technique. Participants made a series of allocation choices for others between two options: a 'non-Utilitarian option' with both smaller variance and higher minimum pay-off (but a smaller total) compared with the other 'Utilitarian option'. Choice data confirmed that participants had strong inequality-averse preferences, and when choosing non-utilitarian allocations, participants' mouse movements prior to choices were more strongly determined by the minimum elements of the non-Utilitarian options than the variance elements. Furthermore, a time-series analysis revealed that this dominance emerged at a very early stage of decision making (around 500 ms after the stimulus onset), suggesting that the maximin concern operated as a strong cognitive anchor almost instantaneously. Our results provide the first temporally fine-scale evidence that people weigh the maximin concern over the egalitarian concern in distributive judgements.

Confidence in visual motion discrimination is preserved in individuals with schizophrenia

BACKGROUND

Metacognition is the set of reflexive processes that allows humans to evaluate the accuracy of their mental operations. Metacognitive deficits have been described in people with schizophrenia using mostly narrative assessment, and they have been linked to several key symptoms.

METHODS

We assessed metacognitive performance objectively by asking people with schizophrenia or schizoaffective disorder (n = 20) and matched healthy participants (n = 21) to perform a visual discrimination task and report their confidence in their performance. Metacognitive performance was defined as the adequacy between visual discrimination performance and confidence.

RESULTS

Bayesian analyses revealed equivalent metacognitive performance in the 2 groups, despite a weaker association between confidence and trajectory tracking during task execution among people with schizophrenia. We reproduced these results using an evidence accumulation model, which showed similar decisional processes in the 2 groups.

LIMITATIONS

These results from a relatively small study sample cannot be generalized to other perceptual and nonperceptual tasks. To meet this purpose, ecological tasks are needed. As well, the role of antipsychotic medication and design deserves greater attention in the future.

CONCLUSION

We found similar decisional and metacognitive capabilities between people with schizophrenia and healthy controls in a visual discrimination task.