How Mouse-tracking Can Advance Social Cognitive Theory

Humans can infer social preferences from decision speed alone

Humans are known to be capable of inferring hidden preferences and beliefs of their conspecifics when observing their decisions. While observational learning based on choices has been explored extensively, the question of how response times (RT) impact our learning of others' social preferences has received little attention. Yet, while observing choices alone can inform us about the direction of preference, they reveal little about the strength of this preference. In contrast, RT provides a continuous measure of strength of preference with faster responses indicating stronger preferences and slower responses signaling hesitation or uncertainty. Here, we outline a preregistered orthogonal design to investigate the involvement of both choices and RT in learning and inferring other's social preferences. Participants observed other people's behavior in a social preferences task (Dictator Game), seeing either their choices, RT, both, or no information. By coupling behavioral analyses with computational modeling, we show that RT is predictive of social preferences and that observers were able to infer those preferences even when receiving only RT information. Based on these findings, we propose a novel observational reinforcement learning model that closely matches participants' inferences in all relevant conditions. In contrast to previous literature suggesting that, from a Bayesian perspective, people should be able to learn equally well from choices and RT, we show that observers' behavior substantially deviates from this prediction. Our study elucidates a hitherto unknown sophistication in human observational learning but also identifies important limitations to this ability.

Single point motion kinematics convey emotional signals in children and adults

This study investigates whether humans recognize different emotions conveyed only by the kinematics of a single moving geometrical shape and how this competence unfolds during development, from childhood to adulthood. To this aim, animations in which a shape moved according to happy, fearful, or neutral cartoons were shown, in a forced-choice paradigm, to 7- and 10-year-old children and adults. Accuracy and response times were recorded, and the movement of the mouse while the participants selected a response was tracked. Results showed that 10-year-old children and adults recognize happiness and fear when conveyed solely by different kinematics, with an advantage for fearful stimuli. Fearful stimuli were also accurately identified at 7-year-olds, together with neutral stimuli, while, at this age, the accuracy for happiness was not significantly different than chance. Overall, results demonstrates that emotions can be identified by a single point motion alone during both childhood and adulthood. Moreover, motion contributes in various measures to the comprehension of emotions, with fear recognized earlier in development and more readily even later on, when all emotions are accurately labeled.

Attribute latencies causally shape intertemporal decisions

Intertemporal choices - decisions that play out over time - pervade our life. Thus, how people make intertemporal choices is a fundamental question. Here, we investigate the role of attribute latency (the time between when people start to process different attributes) in shaping intertemporal preferences using five experiments with choices between smaller-sooner and larger-later rewards. In the first experiment, we identify attribute latencies using mouse-trajectories and find that they predict individual differences in choices, response times, and changes across time constraints. In the other four experiments we test the causal link from attribute latencies to choice, staggering the display of the attributes. This changes attribute latencies and intertemporal preferences. Displaying the amount information first makes people more patient, while displaying time information first does the opposite. These findings highlight the importance of intra-choice dynamics in shaping intertemporal choices and suggest that manipulating attribute latency may be a useful technique for nudging.

Temporal dynamics of costly avoidance in naturalistic fears: Evidence for sequential-sampling of fear and reward information

Excessive avoidance is characteristic for anxiety disorders, even when approach would lead to positive outcomes. The process of how such approach-avoidance conflicts are resolved is not sufficiently understood. We examined the temporal dynamics of approach-avoidance in intense fear of spiders. Highly fearful and non-fearful participants chose repeatedly between a fixed no spider/low reward and a spider/high reward option with varying fear (probability of spider presentation) and reward information (reward magnitude). By sequentially presenting fear and reward information, we distinguished whether decisions are dynamically driven by both information (sequential-sampling) or whether the impact of fear information is inhibited (cognitive control). Mouse movements were recorded to assess temporal decision dynamics (i.e., how strongly which information impacts decision preference at which timepoint). Highly fearful participants showed stronger avoidance despite lower gains (i.e., costly avoidance). Time-continuous multiple regression of their mouse movements yielded a stronger impact of fear compared to reward information. Importantly, presenting either information first (fear or reward) enhanced its impact during the early decision process. These findings support sequential sampling of fear and reward information, but not inhibitory control. Hence, pathological avoidance may be characterized by biased evidence accumulation rather than altered cognitive control.

Evidence for Shifting Cognitive Strategies when Icons Appear in Unexpected Locations

OBJECTIVE

The present study examines the cognitive effects of placing icons in unexpected spatial locations within websites.

BACKGROUND

Prior research has revealed evidence for cognitive conflict when web icons occur in unexpected locations (e.g., cart, top left), generally consistent with a dynamical systems models. Here, we compare the relative strength of evidence for both dual and dynamical systems models.

METHODS

Participants clicked on icons located in either expected (e.g., cart, top right) or unexpected (e.g., cart, top left) locations while mouse trajectories were continuously recorded. Trajectories were classified according to prototypes associated with each cognitive model. The dynamical systems model predicts curved trajectories, while the dual-systems model predicts straight and change of mind trajectories.

RESULTS

Trajectory classification revealed that curved trajectories increased (+11%), while straight and change of mind trajectories decreased (-12%) when target icons occurred in unexpected locations (p < .001).

CONCLUSION

Rather than employing a single cognitive strategy, users shift from a primarily dual-systems to dynamical systems strategy when icons occur in unexpected locations.

APPLICATION

Potential applications of this work include the assessment of cognitive impacts such as mental workload and cognitive conflict during real-time interaction with websites and other screen-based interfaces, personalization and adaptive interfaces based on an individual's cognitive strategy, and data-driven A/B testing of alternative interface designs.

Earlier social information has a stronger influence on judgments

People's decisions are often informed by the choices of others. Evidence accumulation models provide a mechanistic account of how such social information enters the choice process. Previous research taking this approach has suggested two fundamentally different cognitive mechanisms by which people incorporate social information. On the one hand, individuals may update their evidence level instantaneously when observing social information. On the other hand, they may gradually integrate social information over time. These accounts make different predictions on how the timing of social information impacts its influence. The former predicts that timing has no impact on social information uptake. The latter predicts that social information which arrives earlier has a stronger impact because its impact increases over time. We tested both predictions in two studies in which participants first observed a perceptual stimulus. They then entered a deliberation phase in which social information arrived either early or late before reporting their judgment. In Experiment 1, early social information remained visible until the end and was thus displayed for longer than late social information. In Experiment 2, which was preregistered, early and late social information were displayed for an equal duration. In both studies, early social information had a larger impact on individuals' judgments. Further, an evidence accumulation analysis found that social information integration was best explained by both an immediate update of evidence and continuous integration over time. Because in social systems, timing plays a key role (e.g., propagation of information in social networks), our findings inform theories explaining the temporal evolution of social impact and the emergent social dynamics.

Predicting Hand Movements With Distributional Semantics: Evidence From Mouse-Tracking

Although mouse-tracking has been taken as a real-time window on different aspects of human decision-making processes, whether purely semantic information affects response conflict at the level of motor output as measured through mouse movements is still unknown. Here, across two experiments, we investigated the effects of semantic knowledge by predicting participants' performance in a standard keyboard task and in a mouse-tracking task through distributional semantics, a usage-based modeling approach to meaning. In Experiment 1, participants were shown word pairs and were required to perform a two-alternative forced choice task selecting either the more abstract or the more concrete word, using standard keyboard presses. In Experiment 2, participants performed the same task, yet this time response selection was achieved by moving the computer mouse. Results showed that the involvement of semantic components in the task at hand is observable using both standard reaction times (Experiment 1) as well as using indexes extracted from mouse trajectories (Experiment 2). In particular, mouse trajectories reflected the response conflict and its temporal evolution, with a larger deviation for increasing word semantic relatedness. These findings support the validity of mouse-tracking as a method to detect deep and implicit decision-making features. Additionally, by demonstrating that a usage-based model of meaning can account for the different degrees of cognitive conflict associated with task achievement, these findings testify the impact of the human semantic memory on decision-making processes.

Stereotypes shape response competition when forming impressions

Dynamic models of impression formation posit that bottom-up factors (e.g., a target's facial features) and top-down factors (e.g., perceiver knowledge of stereotypes) continuously interact over time until a stable categorization or impression emerges. Most previous work on the dynamic resolution of judgments over time has focused on either categorization (e.g., "is this person male/female?") or specific trait impressions (e.g., "is this person trustworthy?"). In two mousetracking studies-exploratory (N = 226) and confirmatory (N = 300)-we test a domain-general effect of cultural stereotypes shaping the process underlying impressions of targets. We find that the trajectories of participants' mouse movements gravitate toward impressions congruent with their stereotype knowledge. For example, to the extent that a participant reports knowledge of a "Black men are less [trait]" stereotype, their mouse trajectory initially gravitates toward categorizing individual Black male faces as "less [trait]," regardless of their final judgment of the target.

The competition dynamics of approach and avoidance motivations following interpersonal transgression

Two behavioral motivations coexist in transgressors following an interpersonal transgression-approaching and compensating the victim and avoiding the victim. Little is known about how these motivations arise, compete, and drive transgressors' decisions. The present study adopted a social interaction task to manipulate participants' (i.e., the transgressor) responsibility for another's (i.e., the victim) monetary loss and measure the participants' tradeoff between compensating the victim and avoiding face-to-face interactions with the victim. Following each transgression, participants used a computer mouse to choose between two options differing in the amount of compensation to the victim and the probability of face-to-face contact with the victim. Results showed that as participants' responsibility increased, 1) the decision weights on contact avoidance relative to compensation increased, and 2) the onset of the contact-avoidance attribute was expedited and that of the compensation attribute was delayed. These results demonstrate how competing social motivations following transgression evolve and determine social decision-making and shed light on how social-affective state modulates the dynamics of decision-making in general.

When ethics also matter: Influence of taste, health, and ethical attributes on food decisions traced with a novel mouse-tracking paradigm

Understanding which food attributes influence food decisions is a matter of public health and a lever for interventions promoting healthy diets. Research shows that food decisions are strongly influenced by taste, with health having a weaker and later influence in the food decision process. Yet, the influence of other food attributes and specifically ethical attributes in food decision processes-as traceable in mouse-tracking data-has not been investigated. Furthermore, past research tracing food decision processes with classical mouse-tracking tools has artificially reduced the occurrence of neutral food items, particularly on the taste attribute. This represents an important limitation as neutral items on taste are particularly likely to be influenced by higher-order level attributes, such as health, but also ethics. Extending previous research, two preregistered studies (Study 1, N = 77; Study 2, N = 92) aimed at filling these gaps using a novel one-dimensional mouse-tracking paradigm. Results showed that taste, health, and ethics all influenced food decisions and interacted over time during decision processes. Taste still had the strongest influence, hence replicating previous findings with the present novel mouse-tracking paradigm. Of importance, ethics and health also influenced decisions-and sometimes had an early significant effect-especially for food items rated as neutral on taste. Beyond these effects and taking full advantage of the use of mixed effects models for all analyses, graphical representations of the influence of taste, health, and ethical attributes for all individual food items were provided. Results are discussed considering previous findings and suggested levers for interventions.

Mouse tracking performance: A new approach to analyzing continuous mouse tracking data

Mouse tracking is an important source of data in cognitive science. Most contemporary mouse tracking studies use binary-choice tasks and analyze the curvature or velocity of an individual mouse movement during an experimental trial as participants select from one of the two options. However, there are many types of mouse tracking data available beyond what is produced in a binary-choice task, including naturalistic data from web users. In order to utilize these data, cognitive scientists need tools that are robust to the lack of trial-by-trial structure in most normal computer tasks. We use singular value decomposition (SVD) and detrended fluctuation analysis (DFA) to analyze whole time series of unstructured mouse movement data. We also introduce a new technique for describing two-dimensional mouse traces as complex-valued time series, which allows SVD and DFA to be applied in a straightforward way without losing important spatial information. We find that there is useful information at the level of whole time series, and we use this information to predict performance in an online task. We also discuss how the implications of these results can advance the use of mouse tracking research in cognitive science.

Hand Motions Reveal Attentional Status and Subliminal Semantic Processing: A Mouse-Tracking Technique

Theories of embodied cognition suggest that hand motions and cognition are closely interconnected. An emerging technique of tracking how participants move a computer mouse (i.e., the mouse-tracking technique) has shown advantages over the traditional response time measurement to detect implicit cognitive conflicts. Previous research suggests that attention is essential for subliminal processing to take place at a semantic level. However, this assumption is challenged by evidence showing the presence of subliminal semantic processing in the near-absence of attention. The inconsistency of evidence could stem from the insufficient sensitivity in the response time measurement. Therefore, we examined the role of attention in subliminal semantic processing by analyzing participants' hand motions using the mouse-tracking technique. The results suggest that subliminal semantic processing is not only enhanced by attention but also occurs when attention is disrupted, challenging the necessity of facilitated top-down attention for subliminal semantic processing, as claimed by a number of studies. In addition, by manipulating the color of attentional cues, our experiment shows that the cue color per se could influence participants' response patterns. Overall, the current study suggests that attentional status and subliminal semantic processing can be reliably revealed by temporal-spatial features extracted from cursor motion trajectories.

Privacy Attitudes toward Mouse-Tracking Paradata Collection

Survey participants' mouse movements provide a rich, unobtrusive source of paradata, offering insight into the response process beyond the observed answers. However, the use of mouse tracking may require participants' explicit consent for their movements to be recorded and analyzed. Thus, the question arises of how its presence affects the willingness of participants to take part in a survey at all-if prospective respondents are reluctant to complete a survey if additional measures are recorded, collecting paradata may do more harm than good. Previous research has found that other paradata collection modes reduce the willingness to participate, and that this decrease may be influenced by the specific motivation provided to participants for collecting the data. However, the effects of mouse movement collection on survey consent and participation have not been addressed so far. In a vignette experiment, we show that reported willingness to participate in a survey decreased when mouse tracking was part of the overall consent. However, a larger proportion of the sample indicated willingness to both take part and provide mouse-tracking data when these decisions were combined, compared to an independent opt-in to paradata collection, separated from the decision to complete the study. This suggests that survey practitioners may face a trade-off between maximizing their overall participation rate and maximizing the number of participants who also provide mouse-tracking data. Explaining motivations for paradata collection did not have a positive effect and, in some cases, even reduced participants' reported willingness to take part in the survey.

Action representations in prevention behavior: Evidence from motor execution

Human actions sometimes aim at preventing an event from occurring. How these to-be-prevented events are represented, however, is poorly understood. Recent proposals in the literature point to a possible divide between effect-producing, operant actions, and effect-precluding, prevention actions, suggesting that the control of operant actions relies on codes of environment-related effects whereas prevention actions do not. Here we report two experiments on this issue, showing that spatial features (Experiment 1) as well as temporal features (Experiment 2) of to-be-prevented events influence actions in the same way as corresponding features of to-be-produced effects. This implies that selecting and executing prevention actions relies on anticipated environmental changes, comparable to operant actions.

A neurocognitive approach to studying processes underlying parents' gender socialization

Parental gender socialization refers to ways in which parents teach their children social expectations associated with gender. Relatively little is known about the mechanisms underlying gender socialization. An overview of cognitive and neural processes underlying parental gender socialization is provided. Regarding cognitive processes, evidence exists that parents' implicit and explicit gender stereotypes, attitudes, and gendered attributions are implicated in gender socialization. Other cognitive factors, such as intergroup attitudes, gender essentialism, internal motivation for parenting without gender stereotypes, gender identity, and conflict resolution are theoretically relevant mechanisms underlying gender socialization, but need further investigation. Regarding neural processes, studies demonstrated that attentional processing, conflict monitoring, behavior regulation, and reward processing might underlie stereotypes and biased behavior. However, more research is necessary to test whether these neural processes are also related to parental gender socialization. Based on this overview, a framework is presented of neural and cognitive factors that were theoretically or empirically related to gender socialization.

Replacing vertical actions by mouse movements: a web-suited paradigm for investigating vertical spatial associations

The number of web-based studies in experimental psychology has been growing tremendously throughout the last few years. However, a straightforward web-based implementation does not exist for all types of experimental paradigms. In the current paper, we focus on how vertical response movements-which play a crucial role in spatial cognition and language research-can be translated into a web-based setup. Specifically, we introduce a web-suited counterpart of the vertical Stroop task (e.g., Fox & Shor, in Bull Psychon Soc 7:187-189, 1976; Lachmair et al., in Psychon Bull Rev 18:1180-1188, 2011; Thornton et al., in J Exp Psychol Hum Percept Perform 39:964-973, 2013). We employed nouns referring to entities typically located in lower or upper vertical space (e.g., "worm" and "bird", respectively) in Experiments 1 and 2, and emotional valence words associated with a crouched or an upward bodily posture (e.g., "sadness" and "excitement", respectively) in Experiment 3. Depending on the font color, our participants used their mouse to drag the words to the lower or upper screen location. Across all experiments, we consistently observed congruency effects analogous to those obtained with the lab paradigm using actual vertical arm movements. Consequently, we conclude that our web-suited paradigm establishes a reliable approach to examining vertical spatial associations.

Moving beyond response times with accessible measures of manual dynamics

Button-press measures of response time (RT) and accuracy have long served a central role in psychological research. However, RT and accuracy provide limited insight into how cognitive processes unfold over time. To address this limitation, researchers have used hand-tracking techniques to investigate how cognitive processes unfold over the course of a response, are modulated by recent experience, and function across the lifespan. Despite the efficacy of these techniques for investigating a wide range of psychological phenomena, widespread adoption of hand-tracking techniques within the field is hindered by a range of factors, including equipment costs and the use of specialized software. Here, we demonstrate that the behavioral dynamics previously observed with specialized motion-tracking equipment in an Eriksen flanker task can be captured with an affordable, portable, and easy-to-assemble response box. Six-to-eight-year-olds and adults (N = 90) completed a computerized version of the flanker task by pressing and holding a central button until a stimulus array appeared. Participants then responded by releasing the central button and reaching to press one of two response buttons. This method allowed RT to be separated into initiation time (when the central button was released) and movement time (time elapsed between initiation and completion of the response). Consistent with previous research using motion-tracking techniques, initiation times and movement times revealed distinct patterns of effects across trials and between age groups, indicating that the method used in the current study presents a simple solution for researchers from across the psychological and brain sciences looking to move beyond RTs.

The biased hand. Mouse-tracking metrics to examine the conflict processing in a race-implicit association test

In this study, we adapted a race-Implicit Association Test (race-IAT) to mouse-tracking (MT) technique to identify the more representative target observed MT-metrics and explore the temporal unfolding of the cognitive conflict emerging during the categorisation task. Participants of Western European descent performed a standard keyboard-response race-IAT (RT-race-IAT) and an MT-race-IAT with the same structure. From a behavioural point of view, our sample showed a typical Congruency Effect, thus a pro-White implicit bias, in the RT-race-IAT. In addition, in the MT-race-IAT, the MT-metrics showed a similar Congruency Effect mirroring the higher attraction of the averaged-trajectories towards the incorrect response button in incongruent than congruent trials. Moreover, these MT-metrics were positively associated with RT-race-IAT scores, strengthening the MT approach’s validity in characterising the implicit bias. Furthermore, the distributional analyses showed that mouse trajectories displayed a smooth profile both in congruent and incongruent trials to indicate that the unfolding of the decision process and the raised conflict is guided by dynamical cognitive processing. This latter continuous competition process was studied using a novel phase-based approach which allowed to temporally dissect an Early, a Mid and a Late phase, each of which may differently reflect the decision conflict between automatic and controlled responses in the evolution of the mouse movement towards the target response. Our results show that the MT approach provides an accurate and finer-grained characterisation of the implicit racial attitude than classical RT-IAT. Finally, our novel phase-based approach can be an effective tool to shed light on the implicit conflict processing emerging in a categorisation task with a promising transferable value in different cognitive and neuropsychological fields.

Hands-on false memories: a combined study with distributional semantics and mouse-tracking

Although mouse-tracking has been seen as a real-time window into different aspects of human decision-making processes, currently little is known about how the decision process unfolds in veridical and false memory retrieval. Here, we directly investigated decision-making processes by predicting participants' performance in a mouse-tracking version of a typical Deese-Roediger-McDermott (DRM) task through distributional semantic models, a usage-based approach to meaning. Participants were required to study lists of associated words and then to perform a recognition task with the mouse. Results showed that mouse trajectories were extensively affected by the semantic similarity between the words presented in the recognition phase and the ones previously studied. In particular, the higher the semantic similarity, the larger the conflict driving the choice and the higher the irregularity in the trajectory when correctly rejecting new words (i.e., the false memory items). Conversely, on the temporal evolution of the decision, our results showed that semantic similarity affects more complex temporal measures indexing the online decision processes subserving task performance. Together, these findings demonstrate that semantic similarity can affect human behavior at the level of motor control, testifying its influence on online decision-making processes. More generally, our findings complement previous seminal theories on false memory and provide insights into the impact of the semantic memory structure on different decision-making components.

Costly avoidance of Pavlovian fear stimuli and the temporal dynamics of its decision process

Conflicts between avoiding feared stimuli versus approaching them for competing rewards are essential for functional behavior and anxious psychopathology. Yet, little is known about the underlying decision process. We examined approach-avoidance decisions and their temporal dynamics when avoiding Pavlovian fear stimuli conflicted with gaining rewards. First, a formerly neutral stimulus (CS+) was repeatedly paired with an aversive stimulus (US) to establish Pavlovian fear. Another stimulus (CS−) was never paired with the US. A control group received neutral tones instead of aversive USs. Next, in each of 324 trials, participants chose between a CS−/low reward and a CS+/high reward option. For the latter, probability of CS+ presentation (Pavlovian fear information) and reward magnitude (reward information) varied. Computer mouse movements were tracked to capture the decision dynamics. Although no more USs occurred, pronounced and persistent costly avoidance of the Pavlovian fear CS+ was found. Time-continuous multiple regression of movement trajectories revealed a stronger and faster impact of Pavlovian fear compared to reward information during decision-making. The impact of fear information, but not reward information, modestly decreased across trials. These findings suggest a persistently stronger weighting of fear compared to reward information during approach-avoidance decisions, which may facilitate the development of pathological avoidance.

Cognitive miserliness in argument literacy? Effects of intuitive and analytic thinking on recognizing fallacies

Fallacies are a particular type of informal argument that are psychologically compelling and often used for rhetorical purposes. Fallacies are unreasonable because the reasons they provide for their claims are irrelevant or insufficient. Ability to recognize the weakness of fallacies is part of what we call argument literacy and imporatant in rational thinking. Here we examine classic fallacies of types found in textbooks. In an experiment, participants evaluated the quality of fallacies and reasonable arguments. We instructed participants to think either intuitively, using their first impressions, or analytically, using rational deliberation. We analyzed responses, response times, and cursor trajectories (captured using mouse tracking). The results indicate that instructions to think analytically made people spend more time on the task but did not make them change their minds more often. When participants made errors, they were drawn towards the correct response, while responding correctly was more straightforward. The results are compatible with “smart intuition” accounts of dual-process theories of reasoning, rather than with corrective default-interventionist accounts. The findings are discussed in relation to whether theories developed to account for formal reasoning can help to explain the processing of everyday arguments.

The Specificity and Reliability of Conflict Adaptation: A Mouse-Tracking Study

Researchers have recently begun to question the specificity and reliability of conflict adaptation effects, also known as sequential congruency effects (SCEs), a highly cited effect in cognitive psychology. Some have even used the lack of reliability across tasks (e.g., Flanker, and Stroop) to argue against models of cognitive control that have dominated the field for decades. The present study tested the possibility that domain-general processes across tasks might appear on more sensitive mouse-tracking metrics rather than overall reaction times. The relationship between SCE effects on the Stroop and Flanker tasks were examined for the first time using a mouse-tracking paradigm. Three main findings emerged: (1) Robust SCEs were observed for both the Stroop and Flanker tasks at the group level, (2) Within-task split-half reliabilities for the SCE across dependent variables were weak at best and non-existent in many cases, and (3) SCEs for the Flanker and Stroop tasks did not correlate with each other for overall reaction times, but did show significant correlations between tasks on more dynamic measures that captured processes before response execution. These findings contribute to the literature by highlighting how mouse-tracking may be a fruitful avenue by which future studies can examine the specificity and reliability of conflict adaptation and tease apart different theoretical models producing the effects.

I change my mind to get better: Process tracing-based microanalysis of food choice processes reveals differences between anorexia nervosa and bulimia nervosa during inpatient treatment

Food choice and its underlying processes is understudied in bulimia nervosa (BN) and anorexia nervosa (AN). Thus, we examined cognitive processes during food choice through mouse tracing in AN (n = 36) and BN (n = 27) undergoing inpatient treatment. Both patient groups and matched healthy controls (HC, n = 59) made 153 binary food choices before rating all foods on their liking and calorie density. Choice outcomes and corresponding mouse movements were modelled as a function of inpatient treatment stage in our analyses. Compared to patients with BN and HC, those with AN showed a clear calorie avoidance on most trials. Yet, mouse paths in AN patients early in treatment, revealed a late direction reversal ('change of mind', CoM) on high-calorie choices. AN patients later in treatment, by contrast, showed fewer CoM alongside more choices for - and liking of - high-calorie foods. Patients with BN showed more CoM trials during low-calorie choices and low-calorie choices were more frequent in patients later in treatment. Thus, relative to patients early in treatment, patients who are later in treatment show less of the overall group pattern of consistently choosing low-calorie food (AN) or high-calorie food (BN). Less cognitive regulation (fewer CoM trials) went along with higher liking for high-calorie foods in AN. These cross-sectional differences between AN early and late in treatment might reflect the formation of healthier habits. In addition, clear patient group differences suggest more specific treatment strategies.

(Mouse cursor)-Tracking food decisions in binge eating disorder reveals preference for high-energy foods and a role of BMI

Binge Eating Disorder (BED) has been associated with deficits in cognitive control and decision-making. Yet, no study has yet investigated the characteristics of food choice and the involved choice conflict in this disorder. In the present study individuals with BED (N = 22) and without BED (N = 61), with a body mass index (BMI) between 21 and 44 completed 153 binary food decisions among foods varying in palatability and energy density. To assess conflict during choice we recorded computer mouse paths and reaction times. Subsequently, participants rated all foods on liking and energy content. Finally, participants completed a bogus taste test with the same foods to measure actual consumption. Predictors were modelled continuously using Bayesian mixed-effects modelling. Individuals with BED liked foods with higher energy content more and chose them more often in the choice task. Yet, actual consumption in the taste test did not differ between groups, neither regarding total consumption, nor of foods with higher energy. Mouse cursor-tracking revealed that control participants with higher BMIs showed more choice conflict than those with lower BMIs. This pattern was reversed in those with BED. The high-energy preference in ratings and food choice represent the first evidence in a controlled laboratory context for disorder-congruent food choice in BED. The fact that this was not reflected in actual consumption might have methodological implications for measuring laboratory eating behaviour. Mouse cursor-tracking gave further insights into choice processes and showed a less conflicted food choice in those with BED with higher BMI compared to those with lower BMI.

Discerning Mouse Trajectory Features With the Drift Diffusion Model

Mouse tracking, a new action-based measure of behavior, has advanced theories of decision making with the notion that cognitive and social decision making is fundamentally dynamic. Implicit in this theory is that people's decision strategies, such as discounting delayed rewards, are stable over task design and that mouse trajectory features correspond to specific segments of decision making. By applying the hierarchical drift diffusion model and the Bayesian delay discounting model, we tested these assumptions. Specifically, we investigated the extent to which the "mouse-tracking" design of decision-making tasks (delay discounting task, DDT and stop-signal task, SST) deviate from the standard "keypress" design of decision making tasks. We found remarkable agreement in delay discounting rates (intertemporal impatience) obtained in the keypress and mouse-tracking versions of DDT (ρ = 0.90) even though these tasks were given about 1 week apart. Rates of evidence accumulation converged well in the two versions (DDT, ρ = .86; SST, ρ = .55). Omission/commission error in SST showed high agreement (ρ = .42, ρ = .53). Mouse-motion features such as maximum velocity and AUC (area under the curve) correlated well with nondecision time (ρ = -.42) and boundary separation (ρ = .44)-the amount of information needed to accumulate prior to making a response. These results indicate that the response time (RT) and motion-based decision tasks converge well at a fundamental level, and that mouse-tracking features such as AUC and maximum velocity do indicate the degree of decision conflict and impulsivity.

Enhancing models of social and strategic decision making with process tracing and neural data

Every decision we take is accompanied by a characteristic pattern of response delay, gaze position, pupil dilation, and neural activity. Nevertheless, many models of social decision making neglect the corresponding process tracing data and focus exclusively on the final choice outcome. Here, we argue that this is a mistake, as the use of process data can help to build better models of human behavior, create better experiments, and improve policy interventions. Specifically, such data allow us to unlock the "black box" of the decision process and evaluate the mechanisms underlying our social choices. Using these data, we can directly validate latent model variables, arbitrate between competing personal motives, and capture information processing strategies. These benefits are especially valuable in social science, where models must predict multi-faceted decisions that are taken in varying contexts and are based on many different types of information. This article is categorized under: Economics > Interactive Decision-Making Neuroscience > Cognition Psychology > Reasoning and Decision Making.

Tracking stress via the computer mouse? Promises and challenges of a potential behavioral stress marker

Computer mouse tracking offers a simple and cost-efficient way to gather continuous behavioral data and has mostly been utilized in psychological science to study cognitive processes. The present study extends the potential applicability of computer mouse tracking and investigates the feasibility of using computer mouse tracking for stress measurement. Drawing on first empirical results and theoretical considerations, we hypothesized that stress affects sensorimotor processes involved in mouse usage. To explore the relationship between stress and computer mouse usage, we conducted a between-participant field experiment in which N = 994 participants worked on four mouse tasks in a high-stress or low-stress condition. In the manipulation check, participants reported different stress levels between the two conditions. However, frequentist and machine learning data analysis approaches did not reveal a clear and systematic relationship between mouse usage and stress. These findings challenge the feasibility of using straightforward computer mouse tracking for generalized stress measurement.

Reducing variance or helping the poorest? A mouse tracking approach to investigate cognitive bases of inequality aversion in resource allocation

Humans dislike unequal allocations. Although often conflated, such 'inequality-averse' preferences are separable into two elements: egalitarian concern about the variance and maximin concern about the poorest (maximizing the minimum). Recent research has shown that the maximin concern operates more robustly in allocation decisions than the egalitarian concern. However, the real-time cognitive dynamics of allocation decisions are still unknown. Here, we examined participants' choice behaviour with high temporal resolution using a mouse-tracking technique. Participants made a series of allocation choices for others between two options: a 'non-Utilitarian option' with both smaller variance and higher minimum pay-off (but a smaller total) compared with the other 'Utilitarian option'. Choice data confirmed that participants had strong inequality-averse preferences, and when choosing non-utilitarian allocations, participants' mouse movements prior to choices were more strongly determined by the minimum elements of the non-Utilitarian options than the variance elements. Furthermore, a time-series analysis revealed that this dominance emerged at a very early stage of decision making (around 500 ms after the stimulus onset), suggesting that the maximin concern operated as a strong cognitive anchor almost instantaneously. Our results provide the first temporally fine-scale evidence that people weigh the maximin concern over the egalitarian concern in distributive judgements.

Mouse-tracking reveals cognitive conflict during negative impression formation in women with Borderline Personality Disorder or Social Anxiety Disorder

Individuals with Borderline Personality Disorder (BPD) or Social Anxiety Disorder (SAD) suffer from substantial interpersonal dysfunction and have difficulties establishing social bonds. A tendency to form negative first impressions of others could contribute to this by way of reducing approach behavior. We tested whether women with BPD or SAD would show negative impression formation compared to healthy women (HCs). We employed the Thin Slices paradigm and showed videos of 52 authentic target participants to 32 women with BPD, 29 women with SAD, and 37 HCs. We asked participants to evaluate whether different positive or negative adjectives described targets and expected BPD raters to provide the most negative ratings, followed by SAD and HC. BPD and SAD raters both agreed with negative adjectives more often than HCs (e.g., ‘Yes, the person is greedy’), and BPD raters rejected positive adjectives more often (e.g., ‘No, the person is not humble.’). However, BPD and SAD raters did not differ significantly from each other. Additionally, we used the novel process tracing method mouse-tracking to assess the cognitive conflict (via trajectory deviations) raters experienced during decision-making. We hypothesized that HCs would experience more conflict when making unfavorable (versus favorable) evaluations and that this pattern would flip in BPD and SAD. We quantified cognitive conflict via maximum absolute deviations (MADs) of the mouse-trajectories. As hypothesized, HCs showed more conflict when rejecting versus agreeing with positive adjectives. The pattern did not flip in BPD and SAD but was substantially reduced, such that BPD and SAD showed similar levels of conflict when rejecting and agreeing with positive adjectives. Contrary to the hypothesis for BPD and SAD, all three groups experienced substantial conflict when agreeing with negative adjectives. We discuss therapeutic implications of the combined choice and mouse-tracking results.

Tracking Prejudice: A Mouse-Tracking Measure of Evaluative Conflict Predicts Discriminatory Behavior

Explicit evaluations of racial out-groups often involve conflict between opposing evaluative tendencies. Yet this type of conflict is difficult to capture with standard measures of evaluative processing, which either ignore explicit evaluation or capture only the aspects of explicit evaluation that are consciously accessible and freely reported. A new tool may fill this gap in our ability to measure conflict in racial evaluation. This tool, called the mouse-tracking measure of racial bias (Race-MT), is designed to capture conflict in explicit evaluations of racial groups, even if that conflict is neither consciously accessible nor freely reported. We vetted the Race-MT by exploring whether it predicts discriminatory behavior. Across five studies (four preregistered, N = 1,492), we used the Race-MT to measure conflict in people’s positive, explicit evaluations of racial out-groups versus in-groups. These measures predicted discriminatory behavior in a noisy, naturalistic setting, suggesting that the Race-MT provides theoretically meaningful and predicatively useful insights into racial evaluation.

Knowledge extraction from pointer movements and its application to detect uncertainty

Pointer-tracking methods can capture a real-time trace at high spatio-temporal resolution of users' pointer interactions with a graphical user interface. This trace is potentially valuable for research on human-computer interaction (HCI) and for investigating perceptual, cognitive and affective processes during HCI. However, little research has reported spatio-temporal pointer features for the purpose of tracking pointer movements in on-line surveys. In two studies, we identified a set of pointer features and movement patterns and showed that these can be easily distinguished. In a third study, we explored the feasibility of using patterns of interactive pointer movements, or micro-behaviours, to detect response uncertainty. Using logistic regression and k-fold cross-validation in model training and testing, the uncertainty model achieved an estimated performance accuracy of 81%. These findings suggest that micro-behaviours provide a promising approach toward developing a better understanding of the relationship between the dynamics of pointer movements and underlying perceptual, cognitive and affective psychological mechanisms.

Does Attention Increase the Value of Choice Alternatives?

A growing recognition of the role of attention in decision-making has been driven by both the technology of eye tracking and the development of models that explicitly incorporate attention. One result of this convergence is the arresting claim that attention, by itself, can increase the perceived value of a decision alternative. In this review, we cover the origins of that claim, its empirical foundation, and the reasoning that supports it. The conclusion is that, to date, there is not sufficient evidence to support the claim. Alternative explanations for the extant evidentiary base are discussed, as is the balance between the bottom-up influence of empirical evidence and the top-down commitment to a conceptual framework.

Using mouse cursor tracking to investigate online cognition: Preserving methodological ingenuity while moving toward reproducible science

Mouse cursor tracking has become a prominent method for characterizing cognitive processes, used in a wide variety of domains of psychological science. Researchers have demonstrated considerable ingenuity in the application of the approach, but the methodology has not undergone systematic analysis to facilitate the development of best practices. Furthermore, recent research has demonstrated effects of experimental design features on a number of mousetracking outcomes. We conducted a systematic review of the mouse-tracking literature to survey the reporting and spread of mouse variables (Cursor speed, Sampling rate, Training), physical characteristics of the experiments (Stimulus position, Response box position) and response requirements (Start procedure, Response procedure, Response deadline). This survey reveals that there is room for improvement in reporting practices, especially of subtler design features that researchers may have assumed would not impact research results (e.g., Cursor speed). We provide recommendations for future best practices in mouse-tracking studies and consider how best to standardize the mouse-tracking literature without excessively constraining the methodological flexibility that is essential to the field.

MatMouse: A Mouse Movements Tracking and Analysis Toolbox for Visual Search Experiments

The present study introduces a new MATLAB toolbox, called MatMouse, suitable for the performance of experimental studies based on mouse movements tracking and analysis. MatMouse supports the implementation of task-based visual search experiments. The proposed toolbox provides specific functions which can be utilized for the experimental building and mouse tracking processes, the analysis of the recorded data in specific metrics, the production of related visualizations, as well as for the generation of statistical grayscale heatmaps which could serve as an objective ground truth product. MatMouse can be executed as a standalone package or integrated in existing MATLAB scripts and/or toolboxes. In order to highlight the functionalities of the introduced toolbox, a complete case study example is presented. MatMouse is freely distributed to the scientific community under the third version of GNU General Public License (GPL v3) on GitHub platform.

Using dynamic monitoring of choices to predict and understand risk preferences

Navigating conflict is integral to decision-making, serving a central role both in the subjective experience of choice as well as contemporary theories of how we choose. However, the lack of a sensitive, accessible, and interpretable metric of conflict has led researchers to focus on choice itself rather than how individuals arrive at that choice. Using mouse-tracking-continuously sampling computer mouse location as participants decide-we demonstrate the theoretical and practical uses of dynamic assessments of choice from decision onset through conclusion. Specifically, we use mouse tracking to index conflict, quantified by the relative directness to the chosen option, in a domain for which conflict is integral: decisions involving risk. In deciding whether to accept risk, decision makers must integrate gains, losses, status quos, and outcome probabilities, a process that inevitably involves conflict. Across three preregistered studies, we tracked participants' motor movements while they decided whether to accept or reject gambles. Our results show that 1) mouse-tracking metrics of conflict sensitively detect differences in the subjective value of risky versus certain options; 2) these metrics of conflict strongly predict participants' risk preferences (loss aversion and decreasing marginal utility), even on a single-trial level; 3) these mouse-tracking metrics outperform participants' reaction times in predicting risk preferences; and 4) manipulating risk preferences via a broad versus narrow bracketing manipulation influences conflict as indexed by mouse tracking. Together, these results highlight the importance of measuring conflict during risky choice and demonstrate the usefulness of mouse tracking as a tool to do so.

The dynamics of self-control: within-participant modeling of binary food choices and underlying decision processes as a function of restrained eating

Successful self-control during food choice might require inhibition of impulses to avoid indulging in tempting but calorie-dense foods, and this might particularly apply to individuals restraining their food intake. Adopting a novel within-participant modeling approach, we tested 62 females during a mouse-tracking based binary food choice task. Subsequent ratings of foods on palatability, healthiness, and calorie density were modeled as predictors for both decision outcome (choice) and decision process (measures of self-control conflict) while considering the moderating role of restrained eating. Results revealed that individuals higher on restrained eating were less likely to choose more high-calorie foods and showed less self-control conflict when choosing healthier foods. The latter finding is in contrast with the common assumption of self-control as requiring effortful and conscious inhibition of temptation impulses. Interestingly, restrained eaters rated healthy and low-calorie foods as more palatable than individuals with lower restrained eating scores, both in the main experiment and an independent replication study, hinting at an automatic and rather effortless mechanism of self-control (palatability shift) that obviates effortful inhibition of temptation impulses.

When in Doubt, Follow the Crowd? Responsiveness to Social Proof Nudges in the Absence of Clear Preferences

Nudges have gained popularity as a behavioral change tool that aims to facilitate the selection of the sensible choice option by altering the way choice options are presented. Although nudges are designed to facilitate these choices without interfering with people’s prior preferences, both the relation between individuals’ prior preferences and nudge effectiveness, as well as the notion that nudges ‘facilitate’ decision-making have received little empirical scrutiny. Two studies examine the hypothesis that a social proof nudge is particularly effective when people have no clear prior preference, either because people are indifferent (in a color-categorization task; Study 1, N = 255) or because people experience a choice conflict (making shopping decisions about meat products; Study 2, N = 97). Both studies employed a social proof nudge to steer participants’ choices. The potential facilitating effect of the nudge was tested using a mouse-tracker paradigm that implicitly assessed experienced uncertainty during decision-making. Results showed that the nudge was effective in steering participants’ decisions; the facilitation effect (i.e., reduced uncertainty regarding the decision) was only observed for conflicting preferences, but not under indifference. A better understanding of when and how nudges can influence individuals’ behavior may help in deciding whether nudges are an appropriate policy tool for changing particular undesirable behavior.

Mouse tracking reveals structure knowledge in the absence of model-based choice

Converging evidence has demonstrated that humans exhibit two distinct strategies when learning in complex environments. One is model-free learning, i.e., simple reinforcement of rewarded actions, and the other is model-based learning, which considers the structure of the environment. Recent work has argued that people exhibit little model-based behavior unless it leads to higher rewards. Here we use mouse tracking to study model-based learning in stochastic and deterministic (pattern-based) environments of varying difficulty. In both tasks participants’ mouse movements reveal that they learned the structures of their environments, despite the fact that standard behavior-based estimates suggested no such learning in the stochastic task. Thus, we argue that mouse tracking can reveal whether subjects have structure knowledge, which is necessary but not sufficient for model-based choice.

Mouse tracking can reveal people’s subjective beliefs and whether they understand the structure of a task. These data demonstrate that people often do not use this information to make good choices.

Social Transmission of Leadership Preference: Knowledge of Group Membership and Partisan Media Reporting Moderates Perceptions of Leadership Ability From Facial Cues to Competence and Dominance

While first impressions of dominance and competence can influence leadership preference, social transmission of leadership preference has received little attention. The capacity to transmit, store and compute information has increased greatly over recent history, and the new media environment may encourage partisanship (i.e., "echo chambers"), misinformation and rumor spreading to support political and social causes and be conducive both to emotive writing and emotional contagion, which may shape voting behavior. In our pre-registered experiment, we examined whether implicit associations between facial cues to dominance and competence (intelligence) and leadership ability are strengthened by partisan media and knowledge that leaders support or oppose us on a socio-political issue of personal importance. Social information, in general, reduced well-established implicit associations between facial cues and leadership ability. However, as predicted, social knowledge of group membership reduced preferences for facial cues to high dominance and intelligence in out-group leaders. In the opposite-direction to our original prediction, this "in-group bias" was greater under less partisan versus partisan media, with partisan writing eliciting greater state anxiety across the sample. Partisanship also altered the salience of women's facial appearance (i.e., cues to high dominance and intelligence) in out-group versus in-group leaders. Independent of the media environment, men and women displayed an in-group bias toward facial cues of dominance in same-sex leaders. Our findings reveal effects of minimal social information (facial appearance, group membership, media reporting) on leadership judgments, which may have implications for patterns of voting or socio-political behavior at the local or national level.

Mouse movement measures enhance the stop-signal task in adult ADHD assessment

The accurate detection of attention-deficit/hyperactivity disorder (ADHD) symptoms, such as inattentiveness and behavioral disinhibition, is crucial for delivering timely assistance and treatment. ADHD is commonly diagnosed and studied with specialized questionnaires and behavioral tests such as the stop-signal task. However, in cases of late-onset or mild forms of ADHD, behavioral measures often fail to gauge the deficiencies well-highlighted by questionnaires. To improve the sensitivity of behavioral tests, we propose a novel version of the stop-signal task (SST), which integrates mouse cursor tracking. In two studies, we investigated whether introducing mouse movement measures to the stop-signal task improves associations with questionnaire-based measures, as compared to the traditional (keypress-based) version of SST. We also scrutinized the influence of different parameters of stop-signal tasks, such as the method of stop-signal delay setting or definition of response inhibition failure, on these associations. Our results show that a) SSRT has weak association with impulsivity, while mouse movement measures have strong and significant association with impulsivity; b) machine learning models trained on the mouse movement data from "known" participants using nested cross-validation procedure can accurately predict impulsivity ratings of "unknown" participants; c) mouse movement features such as maximum acceleration and maximum velocity are among the most important predictors for impulsivity; d) using preset stop-signal delays prompts behavior that is more indicative of impulsivity.

Social Judgments of Digitally Manipulated Stuttered Speech: An Evaluation of Self-Disclosure on Cognition

Purpose Persons who stutter (PWS) may be susceptible to discrimination because of negative judgments made by listeners. The current study sought to determine how the cognitive system's explicit (i.e., conscious) and implicit (i.e., nonconscious) biases about PWS are impacted by self-disclosure. Method A computer mouse-tracking paradigm was used to evaluate categorical social judgments about PWS. Computer mouse trajectories, which have been shown to reveal underlying cognitive pull or competition between opposing concepts, were used to measure implicit bias (i.e., nonconscious stereotypes). Participants were asked to explicitly categorize the speaker as either intelligent or unintelligent before and after listening to a speaker self-disclose. Mouse cursor trajectories during the explicit response categorization were used to evaluate implicit bias associated with the decision-making process. Results Results indicated that participants chose "intelligent" for a higher proportion of the trials in the disclosure condition compared to baseline, showing that listeners' explicit biases changed after listening to a self-disclosure that the speaker stutters. Results also indicated listeners exhibited a more negative implicit bias, based on computer mouse trajectories, when rating the PWS relative to the "persons who do not stutter" talker, but this negative implicit bias did seem to reduce over time after the disclosure was made. Conclusions These findings indicate that, even though explicit and implicit biases were evident when listeners heard stuttering, both explicit and implicit biases seemed to extinguish over time after a self-disclosure. Although the bias was not completely extinguished, these results provide promising evidence toward developing methods to reduce negative beliefs and reactions toward PWS. Supplemental Material http://osf.io/mwrp7/.