Unfolding visual lexical decision in time

Predicting Hand Movements With Distributional Semantics: Evidence From Mouse-Tracking

Although mouse-tracking has been taken as a real-time window on different aspects of human decision-making processes, whether purely semantic information affects response conflict at the level of motor output as measured through mouse movements is still unknown. Here, across two experiments, we investigated the effects of semantic knowledge by predicting participants' performance in a standard keyboard task and in a mouse-tracking task through distributional semantics, a usage-based modeling approach to meaning. In Experiment 1, participants were shown word pairs and were required to perform a two-alternative forced choice task selecting either the more abstract or the more concrete word, using standard keyboard presses. In Experiment 2, participants performed the same task, yet this time response selection was achieved by moving the computer mouse. Results showed that the involvement of semantic components in the task at hand is observable using both standard reaction times (Experiment 1) as well as using indexes extracted from mouse trajectories (Experiment 2). In particular, mouse trajectories reflected the response conflict and its temporal evolution, with a larger deviation for increasing word semantic relatedness. These findings support the validity of mouse-tracking as a method to detect deep and implicit decision-making features. Additionally, by demonstrating that a usage-based model of meaning can account for the different degrees of cognitive conflict associated with task achievement, these findings testify the impact of the human semantic memory on decision-making processes.

Risky decisions are influenced by individual attributes as a function of risk preference

It has long been assumed in economic theory that multi-attribute decisions involving several attributes or dimensions - such as probabilities and amounts of money to be earned during risky choices - are resolved by first combining the attributes of each option to form an overall expected value and then comparing the expected values of the alternative options, using a unique evidence accumulation process. A plausible alternative would be performing independent comparisons between the individual attributes and then integrating the results of the comparisons afterwards. Here, we devise a novel method to disambiguate between these types of models, by orthogonally manipulating the expected value of choice options and the relative salience of their attributes. Our results, based on behavioral measures and drift-diffusion models, provide evidence in favor of the framework where information about individual attributes independently impacts deliberation. This suggests that risky decisions are resolved by running in parallel multiple comparisons between the separate attributes - possibly alongside an additional comparison of expected value. This result stands in contrast with the assumption of standard economic theory that choices require a unique comparison of expected values and suggests that at the cognitive level, decision processes might be more distributed than commonly assumed. Beyond our planned analyses, we also discovered that attribute salience affects people of different risk preference type in different ways: risk-averse participants seem to focus more on probability, except when monetary amount is particularly high; risk-neutral/seeking participants, in contrast, seem to focus more on monetary amount, except when probability is particularly low.

Evidence for entropy maximisation in human free choice behaviour

The freedom to choose between options is strongly linked to notions of free will. Accordingly, several studies have shown that individuals demonstrate a preference for choice, or the availability of multiple options, over and above utilitarian value. Yet we lack a decision-making framework that integrates preference for choice with traditional utility maximisation in free choice behaviour. Here we test the predictions of an inference-based model of decision-making in which an agent actively seeks states yielding entropy (availability of options) in addition to utility (economic reward). We designed a study in which participants freely navigated a virtual environment consisting of two consecutive choices leading to reward locations in separate rooms. Critically, the choice of one room always led to two final doors while, in the second room, only one door was permissible to choose. This design allowed us to separately determine the influence of utility and entropy on participants' choice behaviour and their self-evaluation of free will. We found that choice behaviour was better predicted by an inference-based model than by expected utility alone, and that both the availability of options and the value of the context positively influenced participants' perceived freedom of choice. Moreover, this consideration of options was apparent in the ongoing motion dynamics as individuals navigated the environment. In a second study, in which participants selected between rooms that gave access to three or four doors, we observed a similar pattern of results, with participants preferring the room that gave access to more options and feeling freer in it. These results suggest that free choice behaviour is well explained by an inference-based framework in which both utility and entropy are optimised and supports the idea that the feeling of having free will is tightly related to options availability.

Mapping the mental space of emotional concepts through kinematic measures of decision uncertainty

Emotional concepts and their mental representations have been extensively studied. Yet, some ecologically relevant aspects, such as how they are processed in ambiguous contexts (e.g., in relation to other emotional stimuli that share similar characteristics), are incompletely known. We employed a similarity judgement of emotional concepts and manipulated the contextual congruency of the responses along the two main affective dimensions of hedonic valence and physiological activation, respectively. Behavioural and kinematics (mouse-tracking) measures were combined to gather a novel 'similarity index' between emotional concepts, to derive topographical maps of their mental representations. Self-report (interoceptive sensibility, positive-negative affectivity, depression) and physiological measures (heart rate variability, HRV) have been collected to explore their possible association with emotional conceptual representation. Results indicate that emotional concepts typically associated with low arousal profit by contextual congruency, with faster responses and reduced uncertainty when contextual ambiguity decreases. The emotional maps recreate two almost orthogonal axes of valence and arousal, and the similarity measure captures the smooth boundaries between emotions. The emotional map of a subgroup of individuals with low positive affectivity reveals a narrower conceptual distribution, with variations in positive emotions and in individuals with reduced arousal (such as those with reduced HRV). Our work introduces a novel methodology to study emotional conceptual representations, bringing the behavioural dynamics of decision-making processes and choice uncertainty into the affective domain. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.

Examining long-term repetition priming effects in spoken word recognition using computer mouse tracking

Language researchers in a variety of disciplines have used priming as a tool to investigate theoretical questions. In spoken word recognition, long-term repetition priming effects have been obtained across a number of behavioral tasks (e.g., lexical decision, shadowing). Repeated - primed - words are responded to more efficiently than new - unprimed - words. However, to our knowledge, long-term repetition priming effects have not been examined using computer mouse tracking, which would provide data regarding the time course of long-term repetition priming effects. Consequently, we compared participants' lexical decision responses using a computer mouse to primed and unprimed words. We predicted that participants would respond more efficiently to primed words compared to unprimed words. Indeed, across all of the dependent variables investigated (accuracy, reaction time, mouse trajectories) and across environments (in person, online), participants responded more efficiently to primed words than to unprimed words. We also performed additional exploratory analyses examining long-term repetition priming effects for nonwords. Across environments (in person, online), participants had more errors to primed nonwords than to unprimed nonwords, but there were no differences in reaction times and mouse trajectories. The current data demonstrating long-term repetition priming effects in mouse tracking are expected to motivate future investigations examining the time course of various long-term repetition priming effects for both words and nonwords.

Constructing Pseudowords with Constraints on Morphological Features - Application for Polish Pseudonouns and Pseudoverbs

Pseudowords allow researchers to investigate multiple grammatical or syntactic aspects of language processing. In order to serve that purpose, pseudoword stimuli need to preserve certain properties of real language. We provide a Python-based pipeline for the generation of pseudoword stimuli that sound/read naturally in a given language. The pseudowords are designed to resemble real words and clearly indicate their grammatical class for languages that use specific suffixes from parts of speech. We also provide two sets of pseudonouns and pseudoverbs in Polish that are outcomes of the applied pipeline. The sets are equipped with psycholinguistically relevant properties of words, such as orthographic Levenshtein distance 20. We also performed two studies (overall N = 640) to test the validity of the algorithmically constructed stimuli in a human sample. Thus, we present stimuli that were deprived of direct meaning yet are clearly classifiable as grammatical categories while being orthographically and phonologically plausible.

The letter position coding mechanism of second language words during sentence reading: Evidence from eye movements

We conducted three eye movement experiments to investigate the mechanism for coding letter positions in a person's second language during sentence reading; we also examined the role of morphology in this process with a more rigorous manipulation. Given that readers obtain information not only from currently fixated words (i.e., the foveal area) but also from upcoming words (i.e., the parafoveal area) to guide their reading, we examined both when the targets were fixated (Exp. 1) and when the targets were seen parafoveally (Exps. 2 and 3). First, we found the classic transposed letter (TL) effect in Exp. 1, but not in Exp. 2 or 3. This implies that flexible letter position coding exists during sentence reading. However, this was limited to words located in the foveal area, suggesting that L2 readers whose L2 proficiency is not as high as skilled native readers are not able to extract and utilise the parafoveal letter identity and position information of a word, whether the word length is long (Exp. 2) or short (Exp. 3). Second, we found morphological information to influence the magnitude of the TL effect in Exp. 1. These results provide new eye movement evidence for the flexibility of L2 letter position coding during sentence reading, as well as the interactions between the different internal representations of words in this process. Future L2 reading frameworks should integrate word recognition and eye movement control models.

The dynamics of decision-making and action during active sampling

Embodied Cognition Theories (ECTs) of decision-making propose that the decision process pervades the execution of choice actions and manifests itself in these actions. Decision-making scenarios where actions not only express the choice but also help sample information can provide a valuable, ecologically relevant model for this framework. We present a study to address this paradigmatic situation in humans. Subjects categorized (2AFC task) a central object image, blurred to different extents, by moving a cursor toward the left or right of the display. Upward cursor movements reduced the image blur and could be used to sample information. Thus, actions for decision and actions for sampling were orthogonal to each other. We analyzed response trajectories to test whether information-sampling movements co-occurred with the ongoing decision process. Trajectories were bimodally distributed, with one kind being direct towards one response option (non-sampling), and the other kind containing an initial upward component before veering off towards an option (sampling). This implies that there was an initial decision at the early stage of a trial, whether to sample information or not. Importantly, in sampling trials trajectories were not purely upward, but rather had a significant horizontal deviation early on. This result suggests that movements to sample information exhibit an online interaction with the decision process, therefore supporting the prediction of the ECTs under ecologically relevant constrains.

The road towards understanding embodied decisions

Most current decision-making research focuses on classical economic scenarios, where choice offers are prespecified and where action dynamics play no role in the decision. However, our brains evolved to deal with different choice situations: "embodied decisions". As examples of embodied decisions, consider a lion that has to decide which gazelle to chase in the savannah or a person who has to select the next stone to jump on when crossing a river. Embodied decision settings raise novel questions, such as how people select from time-varying choice options and how they track the most relevant choice attributes; but they have long remained challenging to study empirically. Here, we summarize recent progress in the study of embodied decisions in sports analytics and experimental psychology. Furthermore, we introduce a formal methodology to identify the relevant dimensions of embodied choices (present and future affordances) and to map them into the attributes of classical economic decisions (probabilities and utilities), hence aligning them. Studying embodied decisions will greatly expand our understanding of what decision-making is.

The role of valence in word processing: Evidence from lexical decision and emotional Stroop tasks

It is widely accepted that the valence of a word (neutral, positive, or negative) influences lexical processing, yet data from the commonly used lexical decision and emotional Stroop tasks has yielded inconsistent findings regarding the direction of this influence. One critical obstacle to investigating the independent effects of valence is the matching of emotional and neutral stimuli on the lexical, sublexical, and conceptual characteristics known to influence word recognition. The second obstacle is that the cognitive processes which lead to a lexical decision and a colour naming response are unobservable from the response latency measures typically gathered. The present study compiled a set of neutral, positive, and negative words matched triplet-wise on 26 influential characteristics. The novel "mouse tracking" technique was used to analyse the development of responses to these materials in variants of the lexical decision and emotional Stroop task. A conventional key-press emotional Stroop task is also reported. Results revealed a significant processing advantage for positive words over negative and neutral words in the lexical decision task, whereas valence alone did not produce any significant effects in the emotional Stroop task. The discrepancy between the effects of valence across these different tasks is discussed. We also suggest that previous conflicting findings may be confounded by unmatched emotional and neutral stimuli, thus inflating the potential effects of valence.

Bodily Information and Top-Down Affective Priming Jointly Affect the Processing of Fearful Faces

According to embodied theories, the processing of emotions such as happiness or fear is grounded in emotion-specific perceptual, bodily, and physiological processes. Under these views, perceiving an emotional stimulus (e.g., a fearful face) re-enacts interoceptive and bodily states congruent with that emotion (e.g., increases heart rate); and in turn, interoceptive and bodily changes (e.g., increases of heart rate) influence the processing of congruent emotional content. A previous study by Pezzulo et al. (2018) provided evidence for this embodied congruence, reporting that experimentally increasing heart rate with physical exercise facilitated the processing of facial expressions congruent with that interoception (fear), but not those conveying incongruent states (disgust or neutrality). Here, we investigated whether the above (bottom-up) interoceptive manipulation and the (top-down) priming of affective content may jointly influence the processing of happy and fearful faces. The fact that happiness and fear are both associated with high heart rate but have different (positive and negative) valence permits testing the hypothesis that their processing might be facilitated by the same interoceptive manipulation (the increase of heart rate) but two opposite (positive and negative) affective primes. To test this hypothesis, we asked participants to perform a gender-categorization task of happy, fearful, and neutral faces, which were preceded by positive, negative, and neutral primes. Participants performed the same task in two sessions (after rest, with normal heart rate, or exercise, with faster heart rate) and we recorded their response times and mouse movements during the choices. We replicated the finding that when participants were in the exercise condition, they processed fearful faces faster than when they were in the rest condition. However, we did not find the same reduction in response time for happy (or neutral) faces. Furthermore, we found that when participants were in the exercise condition, they processed fearful faces faster in the presence of negative compared to positive or neutral primes; but we found no equivalent facilitation of positive (or neutral) primes during the processing of happy (or neutral) faces. While the asymmetries between the processing of fearful and happy faces require further investigation, our findings promisingly indicate that the processing of fearful faces is jointly influenced by both bottom-up interoceptive states and top-down affective primes that are congruent with the emotion.

Cognitive dynamics of intertemporal choice in gambling disorder

Gambling Disorder (GD) is a behavioral addiction characterized by the persistence of recurrent gambling behaviors despite serious adverse consequences. One of the key features of GD is a marked inability to delay gratification and an overall impairment of decision-making mechanisms. Indeed, in intertemporal choice (ITC) tasks, GD patients usually display a marked tendency to prefer smaller-sooner over larger-later rewards (temporal discounting, TD). However, ITC represents a highly verbal/explicit measure, and as such it might not be sensitive to implicit decision biases. Here we sought to uncover the implicit mechanisms underlying the ITC impairment in GD by employing the process-tracing method of mouse kinematics. To this aim, we collected and analyzed ITCs and kinematics measures from 24 GD patients and 23 matched healthy control participants (HC). In line with the relevant literature, the results showed that GD patients discounted future rewards more steeply compared to HCs. Additionally, the results of kinematics analyses showed that patients were characterized by a strong bias toward the immediate option, which was associated with straight-line trajectories. Conversely, the delayed option was selected with edge-curved trajectories, indicating a bias toward the immediate option which was revised in later stages of processing. Interestingly, kinematics indices were also found to be predictive of individual discounting preferences (i.e., discount rates) across the two groups. Taken together, these results suggest that kinematics indices, by revealing hidden and implicit patterns of attraction toward the unselected choice option, may represent reliable behavioral markers of TD in gambling disorder.

A State Space Approach to Dynamic Modeling of Mouse-Tracking Data

Mouse-tracking recording techniques are becoming very attractive in experimental psychology. They provide an effective means of enhancing the measurement of some real-time cognitive processes involved in categorization, decision-making, and lexical decision tasks. Mouse-tracking data are commonly analyzed using a two-step procedure which first summarizes individuals' hand trajectories with independent measures, and then applies standard statistical models on them. However, this approach can be problematic in many cases. In particular, it does not provide a direct way to capitalize the richness of hand movement variability within a consistent and unified representation. In this article we present a novel, unified framework for mouse-tracking data. Unlike standard approaches to mouse-tracking, our proposal uses stochastic state-space modeling to represent the observed trajectories in terms of both individual movement dynamics and experimental variables. The model is estimated via a Metropolis-Hastings algorithm coupled with a non-linear recursive filter. The characteristics and potentials of the proposed approach are illustrated using a lexical decision case study. The results highlighted how dynamic modeling of mouse-tracking data can considerably improve the analysis of mouse-tracking tasks and the conclusions researchers can draw from them.

Computer mouse tracking reveals motor signatures in a cognitive task of spatial language grounding

In a novel computer mouse tracking paradigm, participants read a spatial phrase such as "The blue item to the left of the red one" and then see a scene composed of 12 visual items. The task is to move the mouse cursor to the target item (here, blue), which requires perceptually grounding the spatial phrase. This entails visually identifying the reference item (here, red) and other relevant items through attentional selection. Response trajectories are attracted toward distractors that share the target color but match the spatial relation less well. Trajectories are also attracted toward items that share the reference color. A competing pair of items that match the specified colors but are in the inverse spatial relation increases attraction over-additively compared to individual items. Trajectories are also influenced by the spatial term itself. While the distractor effect resembles deviation toward potential targets in previous studies, the reference effect suggests that the relevance of the reference item for the relational task, not its role as a potential target, was critical. This account is supported by the strengthened effect of a competing pair. We conclude, therefore, that the attraction effects in the mouse trajectories reflect the neural processes that operate on sensorimotor representations to solve the relational task. The paradigm thus provides an experimental window through motor behavior into higher cognitive function and the evolution of activation in modal substrates, a longstanding topic in the area of embodied cognition.

Gaze and Motor Traces of Language Processing: Evidence from Autism Spectrum Disorders in Comparison to Typical Controls

We investigated what strategies underlie figurative language processing in two groups of participants distinguished by the presence of a developmental deficit, highly-verbal participants with autism, and control participants without autism in two age ranges each. Individuals with autism spectrum disorder are characterised by impaired social interaction and communication. Even at the high end of the spectrum, where structural language is adequate, difficulties in comprehending non-literal aspects of language are widely attested. The exact causes of these problems are, however, still open to debate. In an interactive sentence-picture matching task participants selected the most suitable image representation of a non-literal figurative expression that matched the target meaning, while their eye-movements and hand movements were being tracked. Our results suggest that individuals with ASD have different processing patterns than typically developing peers when interpreting figurative language, even when they provide the correct answers. Both children with and without autism, and participants with autism display greater uncertainty and competition between alternatives when providing the answer, often reflected in also considering the literal interpretation of the expression against its target figurative meaning. We provide evidence that expression transparency and decomposability play a central role in figurative language processing across all groups.

Concreteness and emotional valence of episodic future thinking (EFT) independently affect the dynamics of intertemporal decisions

During intertemporal decisions, the value of future rewards decreases as a function of the delay of its receipt (temporal discounting, TD). Since high discount rates have been associated with a series of problematic behaviours and clinical conditions, current research has focused on possible modulators of TD. Specifically, a reduction of individual discount rates has been shown during episodic future thinking (EFT), wherein time intervals are anchored to personal future events. However, it is not entirely clear whether this effect is mediated by a change in the representation of future events (i.e., from abstract to concrete) or by a positive-emotion modulation. Here, we investigated this issue by manipulating the valence of the EFT (i.e., using negative, neutral and positive episodic tags), and by collecting explicit and implicit measures of behaviour. The results showed a significant reduction of TD in all the three emotional conditions compared to the baseline, with differences among them, thus suggesting the existence of a cumulative effect of the concreteness and affective components of the EFT. The analyses of implicit measures additionally revealed that this effect was mediated by a simultaneous increase/decrease of attraction toward the delayed/immediate alternative. Finally, these effects appeared to be modulated by participants' baseline discounting preferences. These findings provide important insights on clinical applications in reward-related disorders.

Forward versus backward semantic priming: What movement dynamics during lexical decision reveal

Separate processes underlying forward (e.g., crescent MOON) and backward (e.g., office POST) priming have previously been inferred from button-press lexical decision response times, with an automatic prospective mechanism and a strategic retrospective mechanism presumed responsible for forward and backward priming, respectively. We tracked hand/mouse kinematics during lexical decision, and found that forward, backward, and symmetrical (e.g., cat DOG) priming exhibited different movement trajectories, with the effect of forward priming emerging earlier than that of backward priming and with symmetrical priming taking the lead around the time when the backward priming effect came online. The findings provide strong converging evidence for different mechanisms driving forward and backward priming, and demonstrate that continuous kinematic measures can shed light on cognitive processes as they unfold in real time.

Early and late indications of item-specific control in a Stroop mouse tracking study

Previous studies indicated that cognitive conflict continues to bias actions even after a movement has been initiated. The present paper examined whether cognitive control also biases actions after movement initiation. To this end, we had participants perform a Stroop task in which we manipulated the item-specific proportion of (in)congruent trials (80% congruent vs. 20% congruent). Importantly, participants responded via mouse movements, allowing us to evaluate various movement parameters: initiation times, movement times, and movement accuracy. Results showed that mouse movements were faster and more accurate during congruent trials compared to incongruent trials. Moreover, we observed that this congruency effect was larger for 80% congruent compared to 20% congruent items, which reflects item-specific cognitive control. Notably, when responses were initiated very fast – rendering virtually no time for stimulus processing before movement onset – this item-specific control was observed only in movement times. However, for relatively slow initiated responses, item specific control was observed both in initiation and in movement times. These findings demonstrate that item-specific cognitive control biases actions before and after movement initiation.

The influence of communication mode on written language processing and beyond

Empirical evidence suggests a broad impact of communication mode on cognition at large, beyond language processing. Using a sign language since infancy might shape the representation of words and other linguistic stimuli - for example, incorporating in it the movements and signs used to express them. Once integrated into linguistic representations, this visuo-motor content can affect deaf signers' linguistic and cognitive processing.

Increased heart rate after exercise facilitates the processing of fearful but not disgusted faces

Embodied theories of emotion assume that emotional processing is grounded in bodily and affective processes. Accordingly, the perception of an emotion re-enacts congruent sensory and affective states; and conversely, bodily states congruent with a specific emotion facilitate emotional processing. This study tests whether the ability to process facial expressions (faces having a neutral expression, expressing fear, or disgust) can be influenced by making the participants’ body state congruent with the expressed emotion (e.g., high heart rate in the case of faces expressing fear). We designed a task requiring participants to categorize pictures of male and female faces that either had a neutral expression (neutral), or expressed emotions whose linkage with high heart rate is strong (fear) or significantly weaker or absent (disgust). Critically, participants were tested in two conditions: with experimentally induced high heart rate (Exercise) and with normal heart rate (Normal). Participants processed fearful faces (but not disgusted or neutral faces) faster when they were in the Exercise condition than in the Normal condition. These results support the idea that an emotionally congruent body state facilitates the automatic processing of emotionally-charged stimuli and this effect is emotion-specific rather than due to generic factors such as arousal.

The detection of faked identity using unexpected questions and mouse dynamics

The detection of faked identities is a major problem in security. Current memory-detection techniques cannot be used as they require prior knowledge of the respondent’s true identity. Here, we report a novel technique for detecting faked identities based on the use of unexpected questions that may be used to check the respondent identity without any prior autobiographical information. While truth-tellers respond automatically to unexpected questions, liars have to “build” and verify their responses. This lack of automaticity is reflected in the mouse movements used to record the responses as well as in the number of errors. Responses to unexpected questions are compared to responses to expected and control questions (i.e., questions to which a liar also must respond truthfully). Parameters that encode mouse movement were analyzed using machine learning classifiers and the results indicate that the mouse trajectories and errors on unexpected questions efficiently distinguish liars from truth-tellers. Furthermore, we showed that liars may be identified also when they are responding truthfully. Unexpected questions combined with the analysis of mouse movement may efficiently spot participants with faked identities without the need for any prior information on the examinee.

Healthy eating decisions require efficient dietary self-control in children: A mouse-tracking food decision study

Learning how to make healthy eating decisions, (i.e., resisting unhealthy foods and consuming healthy foods), enhances physical development and reduces health risks in children. Although healthy eating decisions are known to be challenging for children, the mechanisms of children's food choice processes are not fully understood. The present study recorded mouse movement trajectories while eighteen children aged 8-13 years were choosing between eating and rejecting foods. Children were inclined to choose to eat rather than to reject foods, and preferred unhealthy foods over healthy foods, implying that rejecting unhealthy foods could be a demanding choice. When children rejected unhealthy foods, mouse trajectories were characterized by large curvature toward an eating choice in the beginning, late decision shifting time toward a rejecting choice, and slowed response times. These results suggested that children exercised greater cognitive efforts with longer decision times to resist unhealthy foods, providing evidence that children require dietary self-control to make healthy eating-decisions by resisting the temptation of unhealthy foods. Developmentally, older children attempted to exercise greater cognitive efforts for consuming healthy foods than younger children, suggesting that development of dietary self-control contributes to healthy eating-decisions. The study also documents that healthy weight children with higher BMIs were more likely to choose to reject healthy foods. Overall, findings have important implications for how children make healthy eating choices and the role of dietary self-control in eating decisions.

Mouse tracking traces the "Camrbidge Unievrsity" effects in monolingual and bilingual minds

Previous monolingual studies have consistently suggested that there was flexibility of letter position encoding in different alphabetic writing systems. This robust letter transposition was named the "Cambridge University" effect. However, to date whether the orthographic neighborhood and cross-language script similarity would modulate the magnitude of the Cambridge University effect during the second-language word recognition in bilingual minds was unknown. We address this question using a mouse-tracking experimental paradigm to trace the internal lexical matching processes underlying the lexical access. Our linear mixed effects models and growth curve analyses revealed that a low orthographic neighborhood can trigger a large magnitude of the Cambridge University effect for monolinguals and bilinguals on their hand trajectories. We also found that different-script bilinguals (Chinese-English bilinguals) exhibited a greater Cambridge University effect than similar-script bilinguals (Spanish-English bilinguals) and English monolinguals. The findings offer compelling evidence that a human lexical match criterion of recognition system can be modified by neighborhood density and cross-language script similarity of readers.

Decisions reduce sensitivity to subsequent information

Behavioural studies over half a century indicate that making categorical choices alters beliefs about the state of the world. People seem biased to confirm previous choices, and to suppress contradicting information. These choice-dependent biases imply a fundamental bound of human rationality. However, it remains unclear whether these effects extend to lower level decisions, and only little is known about the computational mechanisms underlying them. Building on the framework of sequential-sampling models of decision-making, we developed novel psychophysical protocols that enable us to dissect quantitatively how choices affect the way decision-makers accumulate additional noisy evidence. We find robust choice-induced biases in the accumulation of abstract numerical (experiment 1) and low-level perceptual (experiment 2) evidence. These biases deteriorate estimations of the mean value of the numerical sequence (experiment 1) and reduce the likelihood to revise decisions (experiment 2). Computational modelling reveals that choices trigger a reduction of sensitivity to subsequent evidence via multiplicative gain modulation, rather than shifting the decision variable towards the chosen alternative in an additive fashion. Our results thus show that categorical choices alter the evidence accumulation mechanism itself, rather than just its outcome, rendering the decision-maker less sensitive to new information.

Lexical decisions in adults with low and high susceptibility to pattern-related visual stress: a preliminary investigation

Pattern-related visual stress (PRVS) is a form of sensory hypersensitivity that some people experience when viewing high contrast repeating patterns, notably alternating dark and light stripes. Those susceptible to PRVS typically have a strong aversion to such stimuli, and this is often accompanied by experiences of visual discomfort and disturbance. The patterns most likely to elicit symptoms of PRVS have a square-wave grating configuration of spatial frequency ~3 cycles/degree. Such stimuli are characteristic of printed text in which lines of words and the spaces between them present a high contrast grating-like stimulus. Consequently, much printed reading material has the potential to elicit PRVS that may impair reading performance, and this problem appears to be common in individuals with reading difficulties including dyslexia. However, the manner in which PRVS affects reading ability is unknown. One possibility is that the early sensory visual stress may interfere with the later cognitive word recognition stage of the reading process, resulting in reading performance that is slower and/or less accurate. To explore the association of PRVS with word recognition ability, lexical decision performance (speed and accuracy) to words and pronounceable non-words was measured in two groups of adults, having low and high susceptibility to PRVS. Results showed that lexical decisions were generally faster but less accurate in high-PRVS, and also that high-PRVS participants made decisions significantly faster for words than for non-words, revealing a strong lexicality effect that was not present in low-PRVS. These findings are novel and, as yet, unconfirmed by other studies.

Embodied choice: how action influences perceptual decision making

Embodied Choice considers action performance as a proper part of the decision making process rather than merely as a means to report the decision. The central statement of embodied choice is the existence of bidirectional influences between action and decisions. This implies that for a decision expressed by an action, the action dynamics and its constraints (e.g. current trajectory and kinematics) influence the decision making process. Here we use a perceptual decision making task to compare three types of model: a serial decision-then-action model, a parallel decision-and-action model, and an embodied choice model where the action feeds back into the decision making. The embodied model incorporates two key mechanisms that together are lacking in the other models: action preparation and commitment. First, action preparation strategies alleviate delays in enacting a choice but also modify decision termination. Second, action dynamics change the prospects and create a commitment effect to the initially preferred choice. Our results show that these two mechanisms make embodied choice models better suited to combine decision and action appropriately to achieve suitably fast and accurate responses, as usually required in ecologically valid situations. Moreover, embodied choice models with these mechanisms give a better account of trajectory tracking experiments during decision making. In conclusion, the embodied choice framework offers a combined theory of decision and action that gives a clear case that embodied phenomena such as the dynamics of actions can have a causal influence on central cognition.

The modern view of how we make perceptual decisions is of a process of accumulating sensory evidence until reaching a threshold level of certainty. However, this evidence accumulation model neglects the contribution of action and motor processes to the choice that is made. Recent novel studies that track the changing dynamics of actions during perceptual decisions are increasingly revealing the contribution of the actions we make to our perceptual choices. Thus, the action dynamics of our bodies causally influences our central cognition, which is a core assumption of embodied theories of mind. This paper presents an explanation of how actions, encompassing behavioral strategies such as preparation and commitment, can bias decision making processes in ways that optimize the ecological choices of animals behaving in natural environments. It thus combines two disconnected research streams, decision-making and action control in a manner consistent with theoretical and psychological arguments for embodied cognition.

Tracking second thoughts: continuous and discrete revision processes during visual lexical decision

We studied the dynamics of lexical decisions by asking participants to categorize lexical and nonlexical stimuli and recording their mouse movements toward response buttons during the choice. In a previous report we revealed greater trajectory curvature and attraction to competitors for Low Frequency words and Pseudowords. This analysis did not clarify whether the trajectory curvature in the two conditions was due to a continuous dynamic competition between the response alternatives or if a discrete revision process (a "change of mind") took place during the choice from an initially selected response to the opposite one. To disentangle these two possibilities, here we analyse the velocity and acceleration profiles of mouse movements during the choice. Pseudowords' peak movement velocity occurred with 100 ms delay with respect to words and Letters Strings. Acceleration profile for High and Low Frequency words and Letters Strings exhibited a butterfly plot with one acceleration peak at 400 ms and one deceleration peak at 650 ms. Differently, Pseudowords' acceleration profile had double positive peaks (at 400 and 600 ms) followed by movement deceleration, in correspondence with changes in the decision from lexical to nonlexical response buttons. These results speak to different online processes during the categorization of Low Frequency words and Pseudowords, with a continuous competition process for the former and a discrete revision process for the latter.

How do you hold your mouse? Tracking the compatibility effect between hand posture and stimulus size

In keeping with the idea that observing objects activates possible motor responses, several experiments revealed compatibility effects between the hand postures used to report a choice and some characteristics of the stimuli. The real-time dynamics of such compatibility effects are currently unknown. We tracked the time course of a categorization experiment requiring subjects to categorize as natural or artifact figures of big and small objects. Participants reported their choice using either a big mouse (requiring a power grip: a hand posture compatible with the grasping of big objects) or a small mouse (requiring a precision grip: a hand posture compatible with the grasping of small objects). We found a compatibility effect between the grip required by the mouse and the grip elicited by objects, even if it was irrelevant to the task. In a following experiment with the same paradigm, lexical stimuli failed to reproduce the same effect. Nevertheless, a compatibility effect mediated by the target-word category (artificial vs. natural) was observed. We discuss the results in the context of affordance effects literature and grounded theories of cognition.

How active perception and attractor dynamics shape perceptual categorization: a computational model

We propose a computational model of perceptual categorization that fuses elements of grounded and sensorimotor theories of cognition with dynamic models of decision-making. We assume that category information consists in anticipated patterns of agent-environment interactions that can be elicited through overt or covert (simulated) eye movements, object manipulation, etc. This information is firstly encoded when category information is acquired, and then re-enacted during perceptual categorization. The perceptual categorization consists in a dynamic competition between attractors that encode the sensorimotor patterns typical of each category; action prediction success counts as "evidence" for a given category and contributes to falling into the corresponding attractor. The evidence accumulation process is guided by an active perception loop, and the active exploration of objects (e.g., visual exploration) aims at eliciting expected sensorimotor patterns that count as evidence for the object category. We present a computational model incorporating these elements and describing action prediction, active perception, and attractor dynamics as key elements of perceptual categorizations. We test the model in three simulated perceptual categorization tasks, and we discuss its relevance for grounded and sensorimotor theories of cognition.

Topological self-organization and prediction learning support both action and lexical chains in the brain

A growing body of evidence in cognitive psychology and neuroscience suggests a deep interconnection between sensory-motor and language systems in the brain. Based on recent neurophysiological findings on the anatomo-functional organization of the fronto-parietal network, we present a computational model showing that language processing may have reused or co-developed organizing principles, functionality, and learning mechanisms typical of premotor circuit. The proposed model combines principles of Hebbian topological self-organization and prediction learning. Trained on sequences of either motor or linguistic units, the network develops independent neuronal chains, formed by dedicated nodes encoding only context-specific stimuli. Moreover, neurons responding to the same stimulus or class of stimuli tend to cluster together to form topologically connected areas similar to those observed in the brain cortex. Simulations support a unitary explanatory framework reconciling neurophysiological motor data with established behavioral evidence on lexical acquisition, access, and recall.

Local dynamics in decision making: The evolution of preference within and across decisions

Within decisions, perceived alternatives compete until one is preferred. Across decisions, the playing field on which these alternatives compete evolves to favor certain alternatives. Mouse cursor trajectories provide rich continuous information related to such cognitive processes during decision making. In three experiments, participants learned to choose symbols to earn points in a discrimination learning paradigm and the cursor trajectories of their responses were recorded. Decisions between two choices that earned equally high-point rewards exhibited far less competition than decisions between choices that earned equally low-point rewards. Using positional coordinates in the trajectories, it was possible to infer a potential field in which the choice locations occupied areas of minimal potential. These decision spaces evolved through the experiments, as participants learned which options to choose. This visualisation approach provides a potential framework for the analysis of local dynamics in decision-making that could help mitigate both theoretical disputes and disparate empirical results.

How embodied is perceptual decision making? Evidence for separate processing of perceptual and motor decisions

The extent to which different cognitive processes are "embodied" is widely debated. Previous studies have implicated sensorimotor regions such as lateral intraparietal (LIP) area in perceptual decision making. This has led to the view that perceptual decisions are embodied in the same sensorimotor networks that guide body movements. We use event-related fMRI and effective connectivity analysis to investigate whether the human sensorimotor system implements perceptual decisions. We show that when eye and hand motor preparation is disentangled from perceptual decisions, sensorimotor areas are not involved in accumulating sensory evidence toward a perceptual decision. Instead, inferior frontal cortex increases its effective connectivity with sensory regions representing the evidence, is modulated by the amount of evidence, and shows greater task-positive BOLD responses during the perceptual decision stage. Once eye movement planning can begin, however, an intraparietal sulcus (IPS) area, putative LIP, participates in motor decisions. Moreover, sensory evidence levels modulate decision and motor preparation stages differently in different IPS regions, suggesting functional heterogeneity of the IPS. This suggests that different systems implement perceptual versus motor decisions, using different neural signatures.

Visual word recognition in deaf readers: lexicality is modulated by communication mode

Evidence indicates that adequate phonological abilities are necessary to develop proficient reading skills and that later in life phonology also has a role in the covert visual word recognition of expert readers. Impairments of acoustic perception, such as deafness, can lead to atypical phonological representations of written words and letters, which in turn can affect reading proficiency. Here, we report an experiment in which young adults with different levels of acoustic perception (i.e., hearing and deaf individuals) and different modes of communication (i.e., hearing individuals using spoken language, deaf individuals with a preference for sign language, and deaf individuals using the oral modality with less or no competence in sign language) performed a visual lexical decision task, which consisted of categorizing real words and consonant strings. The lexicality effect was restricted to deaf signers who responded faster to real words than consonant strings, showing over-reliance on whole word lexical processing of stimuli. No effect of stimulus type was found in deaf individuals using the oral modality or in hearing individuals. Thus, mode of communication modulates the lexicality effect. This suggests that learning a sign language during development shapes visuo-motor representations of words, which are tuned to the actions used to express them (phono-articulatory movements vs. hand movements) and to associated perceptions. As these visuo-motor representations are elicited during on-line linguistic processing and can overlap with the perceptual-motor processes required to execute the task, they can potentially produce interference or facilitation effects.