Semantic priming revealed by mouse movement trajectories

Hand Motions Reveal Attentional Status and Subliminal Semantic Processing: A Mouse-Tracking Technique

Theories of embodied cognition suggest that hand motions and cognition are closely interconnected. An emerging technique of tracking how participants move a computer mouse (i.e., the mouse-tracking technique) has shown advantages over the traditional response time measurement to detect implicit cognitive conflicts. Previous research suggests that attention is essential for subliminal processing to take place at a semantic level. However, this assumption is challenged by evidence showing the presence of subliminal semantic processing in the near-absence of attention. The inconsistency of evidence could stem from the insufficient sensitivity in the response time measurement. Therefore, we examined the role of attention in subliminal semantic processing by analyzing participants' hand motions using the mouse-tracking technique. The results suggest that subliminal semantic processing is not only enhanced by attention but also occurs when attention is disrupted, challenging the necessity of facilitated top-down attention for subliminal semantic processing, as claimed by a number of studies. In addition, by manipulating the color of attentional cues, our experiment shows that the cue color per se could influence participants' response patterns. Overall, the current study suggests that attentional status and subliminal semantic processing can be reliably revealed by temporal-spatial features extracted from cursor motion trajectories.

Discerning Mouse Trajectory Features With the Drift Diffusion Model

Mouse tracking, a new action-based measure of behavior, has advanced theories of decision making with the notion that cognitive and social decision making is fundamentally dynamic. Implicit in this theory is that people's decision strategies, such as discounting delayed rewards, are stable over task design and that mouse trajectory features correspond to specific segments of decision making. By applying the hierarchical drift diffusion model and the Bayesian delay discounting model, we tested these assumptions. Specifically, we investigated the extent to which the "mouse-tracking" design of decision-making tasks (delay discounting task, DDT and stop-signal task, SST) deviate from the standard "keypress" design of decision making tasks. We found remarkable agreement in delay discounting rates (intertemporal impatience) obtained in the keypress and mouse-tracking versions of DDT (ρ = 0.90) even though these tasks were given about 1 week apart. Rates of evidence accumulation converged well in the two versions (DDT, ρ = .86; SST, ρ = .55). Omission/commission error in SST showed high agreement (ρ = .42, ρ = .53). Mouse-motion features such as maximum velocity and AUC (area under the curve) correlated well with nondecision time (ρ = -.42) and boundary separation (ρ = .44)-the amount of information needed to accumulate prior to making a response. These results indicate that the response time (RT) and motion-based decision tasks converge well at a fundamental level, and that mouse-tracking features such as AUC and maximum velocity do indicate the degree of decision conflict and impulsivity.

Path Tortuosity in Virtual Reality: A Novel Approach for Quantifying Behavioral Process in a Food Choice Context

There is a pressing need to better understand how parents make feeding decisions for their children, but extant measures focus primarily on outcomes rather than examining the process of food choice as it unfolds. This exploratory study examined parents' translational movement as they moved throughout a virtual reality-based buffet restaurant to select a lunch for their child. Our aim was to explore whether translational movement would be related to cognitive and affective variables that underlie motivation, effort, and ultimate choices within food decision-making contexts (e.g., guilt, self-efficacy). Movement data were quantified in terms of path tortuosity: the degree of straightness of one's path while traveling through a space. Greater path tortuosity predicted a reduction in parents' guilt about their child feeding, above and beyond actual food chosen. Results suggest path tortuosity serves as an implicit measure of effort put forth by parents throughout the food decision-making process. Future work should continue to explore the utility of novel metrics that can be obtained from unique data sources, such as location tracking, for elucidating complicated behavioral processes such as food choice.

Mouse tracking as a window into decision making

Mouse tracking promises to be an efficient method to investigate the dynamics of cognitive processes: It is easier to deploy than eyetracking, yet in principle it is much more fine-grained than looking at response times. We investigated these claimed benefits directly, asking how the features of decision processes-notably, decision changes-might be captured in mouse movements. We ran two experiments, one in which we explicitly manipulated whether our stimuli triggered a flip in decision, and one in which we replicated more ecological, classical mouse-tracking results on linguistic negation (Dale & Duran, Cognitive Science, 35, 983-996, 2011). We concluded, first, that spatial information (mouse path) is more important than temporal information (speed and acceleration) for detecting decision changes, and we offer a comparison of the sensitivities of various typical measures used in analyses of mouse tracking (area under the trajectory curve, direction flips, etc.). We do so using an "optimal" analysis of our data (a linear discriminant analysis explicitly trained to classify trajectories) and see what type of data (position, speed, or acceleration) it capitalizes on. We also quantify how its results compare with those based on more standard measures.

Modeling Mindsets with Kalman Filter

Mathematical models have played an essential role in interface design. This study focused on “mindsets”—people’s tacit beliefs about attributes—and investigated the extent to which: (1) mindsets can be extracted in a motion trajectory in target selection, and (2) a dynamic state-space model, such as the Kalman filter, helps quantify mindsets. Participants were experimentally manipulated to hold fixed or growth mindsets in a “mock” memory test, and later performed a concept-learning task in which the movement of the computer cursor was recorded in every trial. By inspecting motion trajectories of the cursor, we observed clear disparities in the impact of mindsets; participants who were induced with a fixed mindset moved the cursor faster as compared to those who were induced with a growth mindset. To examine further the mechanism of this influence, we fitted a Kalman filter model to the trajectory data; we found that system-level error-covariance in the Kalman filter model could effectively separate motion trajectories gleaned from the two mindset conditions. Taken together, results from the experiment suggest that people’s mindsets can be captured in motor trajectories in target selection and the Kalman filter helps quantify mindsets. It is argued that people’s personality, attitude, and mindset are embodied in motor behavior underlying target selection and these psychological variables can be studied mathematically with a feedback control system.

Exploring the temporal boundary conditions of the articulatory in-out preference effect

Earlier research has documented a preference for words with consonantal articulation patterns that move from the front to the back of the mouth (e.g., MENIKA) over words with reversely wandering consonantal articulation spots (e.g., KENIMA). The present experiments explored the temporal dynamics of the reading process in this in-out preference effect. In three experiments (total N = 344), we gradually reduced the presentation durations of inward and outward wandering words from 1000 ms down to 25 ms to approximate the minimum length of visual stimulus presentation required to trigger the effect. The in-out effect was reliably observed for exposure timings down to 50 ms, but vanished for 25 ms timings, which is line with previous evidence on phonological encoding. Thus, impressively, 50 ms of word presentation is sufficient to evoke the in-out effect. These findings suggest phonological activation to be a prerequisite and thus a driving mechanism of the in-out effect.

Augmented Go/No-Go Task: Mouse Cursor Motion Measures Improve ADHD Symptom Assessment in Healthy College Students

Attention deficit/hyperactivity disorder (ADHD) is frequently characterized as a disorder of executive function (EF). However, behavioral tests of EF, such as go/No-go tasks, often fail to grasp the deficiency in EF revealed by questionnaire-based measures. This inability is usually attributed to questionnaires and behavioral tasks assessing different constructs of EFs. We propose an additional explanation for this discrepancy. We hypothesize that this problem stems from the lack of dynamic assessment of decision-making (e.g., continuous monitoring of motor behavior such as velocity and acceleration in choice reaching) in classical versions of behavioral tasks. We test this hypothesis by introducing dynamic assessment in the form of mouse motion in a go/No-go task. Our results indicate that, among healthy college students, self-report measures of ADHD symptoms become strongly associated with performance in behavioral tasks when continuous assessment (e.g., acceleration in the mouse-cursor motion) is introduced.

Reading Emotion From Mouse Cursor Motions: Affective Computing Approach

Affective computing research has advanced emotion recognition systems using facial expressions, voices, gaits, and physiological signals, yet these methods are often impractical. This study integrates mouse cursor motion analysis into affective computing and investigates the idea that movements of the computer cursor can provide information about emotion of the computer user. We extracted 16-26 trajectory features during a choice-reaching task and examined the link between emotion and cursor motions. Participants were induced for positive or negative emotions by music, film clips, or emotional pictures, and they indicated their emotions with questionnaires. Our 10-fold cross-validation analysis shows that statistical models formed from "known" participants (training data) could predict nearly 10%-20% of the variance of positive affect and attentiveness ratings of "unknown" participants, suggesting that cursor movement patterns such as the area under curve and direction change help infer emotions of computer users.

The role of attention in subliminal semantic processing: A mouse tracking study

Recent evidence suggests that top-down attention facilitates unconscious semantic processing. To clarify the role of attention in unconscious semantic processing, we traced trajectories of the computer mouse in a semantic priming task and scrutinized the extent to which top-down attention enhances unconscious semantic processing in four different stimulus-onset asynchrony (SOA: 50, 200, 500, or 1000ms) conditions. Participants judged whether a target digit (e.g., “6”) was larger or smaller than five, preceded by a masked priming digit (e.g., “9”). The pre-prime duration changed randomly from trial to trial to disrupt participants’ top-down attention in an uncued condition (in a cued condition, a green square cue was presented to facilitate participants’ top-down attention). The results show that top-down attention modifies the time course of subliminal semantic processing, and the temporal attention window lasts more than 1000ms; attention facilitated by the cue may amplify semantic priming to some extent, yet the amplification effect of attention is relatively minor.

Subliminal or not? Comparing null-hypothesis and Bayesian methods for testing subliminal priming

A difficulty for reports of subliminal priming is demonstrating that participants who actually perceived the prime are not driving the priming effects. There are two conventional methods for testing this. One is to test whether a direct measure of stimulus perception is not significantly above chance on a group level. The other is to use regression to test if an indirect measure of stimulus processing is significantly above zero when the direct measure is at chance. Here we simulated samples in which we assumed that only participants who perceived the primes were primed by it. Conventional analyses applied to these samples had a very large error rate of falsely supporting subliminal priming. Calculating a Bayes factor for the samples very seldom falsely supported subliminal priming. We conclude that conventional tests are not reliable diagnostics of subliminal priming. Instead, we recommend that experimenters calculate a Bayes factor when investigating subliminal priming.

The rapid-chase theory does not extend to movement execution

It is assumed that the processing of a prime followed by a mask occurs sequentially in a feedforward manner when the three (initiation, takeover, and independence) criteria outlined by the rapid-chase theory are met. The purpose of the current study was to determine if the processing of the prime and mask fit the predictions of the rapid-chase theory when the prime and mask are presented during an ongoing movement. In two experiments, participants made rapid pointing movements to a target indicated by the mask. In Experiment 1, the prime was presented at movement onset and the prime-mask stimulus onset asynchrony (SOA) was manipulated. In Experiment 2, the prime-mask SOA was constant but the delay between movement and prime onset was manipulated. Although the results support the initiation and takeover criteria, the data did not support the independence criterion. Consequently, the rapid-chase theory does not appear to extend to movement execution.

Subliminal semantic priming in near absence of attention: A cursor motion study

The role of attention in subliminal semantic priming remains controversial: some researchers argue that attention is necessary for subliminal semantic priming, while others suggest that subliminal semantic processing is free from the influence of attention. The present study employs a cursor motion method to measure priming and evaluate the influence of attention. Specifically, by employing a semantic priming task developed by Naccache, Blandin, and Dehaene (2002), we investigate the extent to which top-down attention influences semantic priming. Results indicate that, consistent with the Naccache et al. (2002) results, attention facilitates priming. However, inconsistent with their theory, significant priming is still observed even in near absence of attention. We suggest that top-down attention helps but is not necessary for subliminal semantic processing.