The real-time link between person perception and action: brain potential evidence for dynamic continuity

Compensative movement ameliorates reduced efficacy of rapidly-embodied decisions in humans

Dynamic environments, such as sports, often demand rapid decision-making and motor execution. The concept of embodied decision refers to the mutual link between both processes, but little is known about how these processes are balanced under severe time constraints. We address this problem by using a baseball-like hitting paradigm with and without Go/No-go judgment; participants were required to hit (Go) a moving target in the strike area or not to hit (No-go) other targets. We found that Go/No-go judgments were effective with regard to task performance, but efficacy was lost below the time constraint of 0.5 seconds mainly due to a reduction in judgment accuracy rather than movement accuracy. However, either slowing movement initiation in Go trials or canceling the movement in progress in No-go trials improved judgment accuracy. Our findings suggest that embodied decision efficacy is limited in split-second periods, but compensation is possible by changing ongoing movement strategies.

A Functional Coupling of Brain and Behavior During Social Categorization of Faces

Considerable research has focused on how people derive information about others' social category memberships from their faces. Theoretical models posit that early extraction of task-relevant information from a face should determine the efficiency with which that face is categorized, but evidence supporting this idea has been elusive. Here, we used a novel trial-level data analytic approach to examine the relationship between two event-related potential components-the P2, indexing early attention to category-relevant information, and the P3, indexing stimulus evaluation-and the speed of overt categorization judgments. As predicted, a larger face-elicited P2 on a particular trial was associated with faster overt race or gender categorization of that face. Moreover, this association was mediated by P3 latency, indicating that extraction of more category-relevant information early in processing facilitated stimulus evaluation. These findings support continuous flow models of information processing and the long-theorized functional significance of face-elicited neurophysiological responses for social categorization.

Test-Retest Reliability of Event-Related Potentials Across Three Tasks

Abstract. Event-related potentials (ERPs) constitute a useful and cost-effective method to assess the neural underpinnings of multiple cognitive processes. ERPs have been used to track changes in cognitive processes in longitudinal and clinical studies. However, few studies have assessed their test-retest reliability (i.e., their consistency across time). Therefore, in the current study, we aimed to assess the test-retest reliability of ERPs (P100, N100, P200, N200, P3b, lateralized readiness potentials) across three tasks. In two assessments separated by approximately 4 months, ERPs were recorded in 26 healthy participants, during two oddball tasks (motor and counting) and a stimulus-response compatibility paradigm. Pearson’s correlations and intraclass correlations were used to assess the test-retest reliability of ERPs. Correlations between ERPs elicited by the three tasks were assessed with Pearson’s correlations. Our analyses revealed moderate to very strong test-retest reliability for most ERP components across the three tasks. Test-retest reliability did not differ between the motor and counting oddball tasks. Most ERPs were also correlated across paradigms. Therefore, these results confirm that ERPs have the potential to be reliable markers to serve as robust assessment tools in longitudinal or clinical studies.

Dynamic interactive theory as a domain-general account of social perception

The perception of social categories, emotions, and personality traits from others' faces each have been studied extensively but in relative isolation. We synthesize emerging findings suggesting that, in each of these domains of social perception, both a variety of bottom-up facial features and top-down social cognitive processes play a part in driving initial perceptions. Among such top-down processes, social-conceptual knowledge in particular can have a fundamental structuring role in how we perceive others' faces. Extending the Dynamic Interactive framework (Freeman & Ambady, 2011), we outline a perspective whereby the perception of social categories, emotions, and traits from faces can all be conceived as emerging from an integrated system relying on domain-general cognitive properties. Such an account of social perception would envision perceptions to be a rapid, but gradual, process of negotiation between the variety of visual cues inherent to a person and the social cognitive knowledge an individual perceiver brings to the perceptual process. We describe growing evidence in support of this perspective as well as its theoretical implications for social psychology.

The effects of Botulinum toxin on the detection of gradual changes in facial emotion

When we feel sad or depressed, our face invariably “drops”. Conversely, when we try to cheer someone up, we might tell them “keep your smile up”, so presupposing that modifying the configuration of their facial muscles will enhance their mood. A crucial assumption that underpins this hypothesis is that mental states are shaped by information originating from the peripheral neuromotor system — a view operationalised as the Facial Feedback Hypothesis. We used botulinum toxin (BoNT-A) injected over the frown area to temporarily paralyse muscles necessary to express anger. Using a pre-post treatment design, we presented participants with gradually changing videos of a face morphing from neutral to full-blown expressions of either anger or happiness and asked them to press a button as soon as they had detected any change in the display. Results indicate that while all participants (control and BoNT-A) improved their reaction times from pre-test to post-test, the BoNT-A group did not when detecting anger in the post-test. We surmise that frown paralysis disadvantaged participants in their ability to improve the detection of anger. Our finding suggests that facial feedback causally affects perceptual awareness of changes in emotion, as well as people’s ability to use perceptual information to learn.

The spatial dynamics of mouse-tracking reveal that attention capture is stimulus-driven rather than contingent upon top-down goals

There has been a longstanding debate over why stimuli capture attention. Some argue that capture is driven by stimulus salience, while others believe that capture only occurs when the features of a stimulus match what we are searching for. This debate has recently focused on attentional disengagement, with the stimulus-driven camp claiming that all salient stimuli capture attention but attention is quickly disengaged from items dissimilar from our target, producing little cost in terms of response time. We used mouse-tracking to examine the spatial effect of cues that either matched or mismatched an observer's target. Experiment 1 demonstrated that a cue mismatching the feature defining the target initially produced a spatial effect that was rapidly resolved, consistent with quick disengagement. Experiment 2 was a preregistered replication with double the sample size that replicated the results of Experiment 1. Overall, computer mouse-tracking provided a direct observation of attentional disengagement, supporting stimulus-driven capture. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Doing psychological science by hand

Over the past decade, mouse-tracking in choice tasks has become a popular method across psychological science. This method exploits hand movements as a measure of multiple response activations that can be tracked continuously over hundreds of milliseconds. Whereas early mouse-tracking research focused on specific debates, researchers have realized the methodology has far broader theoretical value. This more recent work demonstrates that mouse-tracking is a widely applicable measure across the field, capable of exposing the micro-structure of real-time decisions including their component processes and millisecond-resolution time-course in ways that inform theory. In the article, recent advances in the mouse-tracking approach are described, and comparisons with the gold standard measure of reaction time and other temporally-sensitive methodologies are provided. Future directions, including mapping to neural representations with brain-imaging and ways to improve our theoretical understanding of mouse-tracking methodology, are discussed.

Reading Emotion From Mouse Cursor Motions: Affective Computing Approach

Affective computing research has advanced emotion recognition systems using facial expressions, voices, gaits, and physiological signals, yet these methods are often impractical. This study integrates mouse cursor motion analysis into affective computing and investigates the idea that movements of the computer cursor can provide information about emotion of the computer user. We extracted 16-26 trajectory features during a choice-reaching task and examined the link between emotion and cursor motions. Participants were induced for positive or negative emotions by music, film clips, or emotional pictures, and they indicated their emotions with questionnaires. Our 10-fold cross-validation analysis shows that statistical models formed from "known" participants (training data) could predict nearly 10%-20% of the variance of positive affect and attentiveness ratings of "unknown" participants, suggesting that cursor movement patterns such as the area under curve and direction change help infer emotions of computer users.

Dynamic competition account of men's perceptions of women's sexual interest

This work applies a dynamic competition framework of decision making to the domain of sexual perception, which is linked theoretically and empirically to college men's risk for exhibiting sexual coercion and aggression toward female acquaintances. Within a mouse-tracking paradigm, 152 undergraduate men viewed full-body photographs of women who varied in affect (sexual interest or rejection), clothing style (provocative or conservative), and attractiveness, and decided whether each woman currently felt sexually interested or rejecting. Participants' mouse movements were recorded to capture competition dynamics during online processing (throughout the decisional process), and as an index of the final categorical decision (endpoint of the decisional process). Participants completed a measure of Rape-Supportive Attitudes (RSA), a well-established correlate of male-initiated sexual aggression toward female acquaintances. Mixed-effects analyses revealed greater curvature toward the incorrect response on conceptually incongruent trials (e.g., rejecting and dressed provocatively) than on congruent trials (e.g., rejecting and dressed conservatively). This suggests that the two decision alternatives are simultaneously active and compete continuously over time, consistent with a dynamic competition account. Congruence effects also emerged at the decisional endpoint; accuracy was typically lower when stimulus features were incongruent, rather than congruent. RSA potentiated online congruence effects (intermediate states of behavior) but not offline congruence effects (endpoint states of behavior). In a hierarchical regression analysis, online processing indices accounted for unique variability in RSA above and beyond offline accuracy rates. The process-based account of men's sexual-interest judgments ultimately may point to novel targets for prevention strategies designed to reduce acquaintance-initiated sexual aggression on college campuses.

The iterative nature of person construal: Evidence from event-related potentials

Recently, a dynamic-interactive model of person construal (DI model) has been proposed, whereby the social categories a person represents are determined on the basis of an iterative integration of bottom-up and top-down influences. The current study sought to test this model by leveraging the high temporal resolution of event-related brain potentials (ERPs) as 65 participants viewed male faces that varied by race (White vs Black), fixating either between the eyes or on the forehead. Within face presentations, the effect of fixation, meant to vary bottom-up visual input, initially was large but decreased across early latency neural responses identified by a principal components analysis (PCA). In contrast, the effect of race, reflecting a combination of top-down and bottom-up factors, initially was small but increased across early latency principal components. These patterns support the DI model prediction that bottom-up and top-down processes are iteratively integrated to arrive at a stable construal within 230 ms. Additionally, exploratory multilevel modeling of single trial ERP responses representing a component linked to outgroup categorization (the P2) suggests change in effects of the manipulations over the course of the experiment. Implications of the findings for the DI model are considered.

A Neural Mechanism of Social Categorization

Humans readily sort one another into multiple social categories from mere facial features. However, the facial features used to do so are not always clear-cut because they can be associated with opponent categories (e.g., feminine male face). Recently, computational models and behavioral studies have provided indirect evidence that categorizing such faces is accomplished through dynamic competition between parallel, coactivated social categories that resolve into a stable categorical percept. Using a novel paradigm combining fMRI with real-time hand tracking, the present study examined how the brain translates diverse social cues into categorical percepts. Participants (male and female) categorized faces varying in gender and racial typicality. When categorizing atypical faces, participants' hand movements were simultaneously attracted toward the unselected category response, indexing the degree to which such faces activated the opposite category in parallel. Multivoxel pattern analyses (MVPAs) provided evidence that such social category coactivation manifested in neural patterns of the right fusiform cortex. The extent to which the hand was simultaneously attracted to the opposite gender or race category response option corresponded to increased neural pattern similarity with the average pattern associated with that category, which in turn associated with stronger engagement of the dorsal anterior cingulate cortex. The findings point to a model of social categorization in which occasionally conflicting facial features are resolved through competition between coactivated ventral-temporal cortical representations with the assistance of conflict-monitoring regions. More broadly, the results offer a promising multimodal paradigm to investigate the neural basis of "hidden", temporarily active representations in the service of a broad range of cognitive processes.SIGNIFICANCE STATEMENT Individuals readily sort one another into social categories (e.g., sex, race), which have important consequences for a variety of interpersonal behaviors. However, individuals routinely encounter faces that contain diverse features associated with multiple categories (e.g., feminine male face). Using a novel paradigm combining neuroimaging with hand tracking, the present research sought to address how the brain comes to arrive at stable social categorizations from multiple social cues. The results provide evidence that opponent social categories coactivate in face-processing regions, which compete and may resolve into an eventual stable categorization with the assistance of conflict-monitoring regions. Therefore, the findings provide a neural mechanism through which the brain may translate inherently diverse social cues into coherent categorizations of other people.

What's in a "face file"? Feature binding with facial identity, emotion, and gaze direction

A series of four experiments investigated the binding of facial (i.e., facial identity, emotion, and gaze direction) and non-facial (i.e., spatial location and response location) attributes. Evidence for the creation and retrieval of temporary memory face structures across perception and action has been adduced. These episodic structures-dubbed herein "face files"-consisted of both visuo-visuo and visuo-motor bindings. Feature binding was indicated by partial-repetition costs. That is repeating a combination of facial features or altering them altogether, led to faster responses than repeating or alternating only one of the features. Taken together, the results indicate that: (a) "face files" affect both action and perception mechanisms, (b) binding can take place with facial dimensions and is not restricted to low-level features (Hommel, Visual Cognition 5:183-216, 1998), and (c) the binding of facial and non-facial attributes is facilitated if the dimensions share common spatial or motor codes. The theoretical contributions of these results to "person construal" theories (Freeman, & Ambady, Psychological Science, 20(10), 1183-1188, 2011), as well as to face recognition models (Haxby, Hoffman, & Gobbini, Biological Psychiatry, 51(1), 59-67, 2000) are discussed.

More Than Meets the Eye: Split-Second Social Perception

Recent research suggests that visual perception of social categories is shaped not only by facial features but also by higher-order social cognitive processes (e.g., stereotypes, attitudes, goals). Building on neural computational models of social perception, we outline a perspective of how multiple bottom-up visual cues are flexibly integrated with a range of top-down processes to form perceptions, and we identify a set of key brain regions involved. During this integration, 'hidden' social category activations are often triggered which temporarily impact perception without manifesting in explicit perceptual judgments. Importantly, these hidden impacts and other aspects of the perceptual process predict downstream social consequences - from politicians' electoral success to several evaluative biases - independently of the outcomes of that process.

On social neuroscience methodologies and their applicability to group processes and intergroup relations

Group processes and intergroup relations are one of the most important topics examined by social psychologists. Recent advancements in social neuroscience methodologies provide valuable insight into these processes by allowing researchers to examine different psychological phenomena via neural processes that instantiate them while individuals interact with ingroup and outgroup members. This includes responses that occur outside conscious awareness or are deemed undesirable to overtly express. The purpose of this review is to provide an overview of the different social neuroscience methodologies that afford these possibilities. Specifically, functional magnetic resonance imaging (fMRI), electroencephalography (EEG), functional near infrared spectroscopy (fNIRS), transcranial magnetic stimulation (TMS), and genetic approaches will be discussed. Each section includes a discussion of what the methodology is and how it is used to assess neural function. A secondary goal of the review is to highlight recent studies that have utilized the aforementioned tools to better understand intergroup processes and interactions. Throughout, advantages and limitations of each approach are discussed, particularly with respect to the study of group processes and intergroup relations.

Advanced mouse-tracking analytic techniques for enhancing psychological science

Computer mouse-tracking is a relatively recently developed behavioral methodology that can contribute unique insight into a wide variety of psychological phenomena. By recording mouse movements en route to specific responses on a screen, researchers glean continuous information about tentative commitments to multiple response alternatives over time. This approach yields a richness of data that can be fully explored with a variety of sophisticated analytic techniques, but these approaches are relatively underutilized and can be difficult to adopt. Here we describe several techniques for researchers to examine the onset and timing of evolving decision processes; test the degree of response competition at different time points; assess trajectory complexity with spatial disorder analyses; identify qualitatively distinct psychological processes during response generation; and finally to distill unique and meaningful components from mouse-tracking data for subsequent analysis. With this guide, we hope researchers can address novel hypotheses otherwise inaccessible with more traditional methods.

Influences of semantic and syntactic incongruence on readiness potential in turn-end anticipation

Knowing when it is convenient to take a turn in a conversation is an important task for dialog partners. As it appears that this decision is made before the transition point has been reached, it seems to involve anticipation. There are a variety of studies in the literature that provide possible explanations for turn-end anticipation. This study particularly focuses on how turn-end anticipation relies on syntactic and/or semantic information during utterance processing, as tested with syntactically and semantically violated sentences. With a combination reaction time and EEG experiment, we used the onset latencies of the readiness potential (RP) to uncover possible differences in response preparation. Although the mean anticipation timing accuracy (ATA) values of the behavioral test were all within a similar time range (control sentences: 108 ms, syntactically violated sentences: 93 ms and semantically violated sentences: 116 ms), we found evidence that response preparation is indeed different for syntactically and semantically violated sentences in comparison with control sentences. Our preconscious EEG data, in the form of RP results, indicated a response preparation onset to sentence end interval of 1452 ms in normal sentences, 937 ms in sentences with syntactic violations and 944 ms in sentences with semantic violations. Compared with control sentences, these intervals resulted in a significant RP interruption for both sentence types and indicate an interruption of preconscious response preparation. However, the behavioral response to sentence types occurred at comparable time points.

The response dynamics of preferential choice

The ubiquity of psychological process models requires an increased degree of sophistication in the methods and metrics that we use to evaluate them. We contribute to this venture by capitalizing on recent work in cognitive science analyzing response dynamics, which shows that the bearing information processing dynamics have on intended action is also revealed in the motor system. This decidedly "embodied" view suggests that researchers are missing out on potential dependent variables with which to evaluate their models-those associated with the motor response that produces a choice. The current work develops a method for collecting and analyzing such data in the domain of decision making. We first validate this method using widely normed stimuli from the International Affective Picture System (Experiment 1), and demonstrate that curvature in response trajectories provides a metric of the competition between choice options. We next extend the method to risky decision making (Experiment 2) and develop predictions for three popular classes of process model. The data provided by response dynamics demonstrate that choices contrary to the maxim of risk seeking in losses and risk aversion in gains may be the product of at least one "online" preference reversal, and can thus begin to discriminate amongst the candidate models. Finally, we incorporate attentional data collected via eye-tracking (Experiment 3) to develop a formal computational model of joint information sampling and preference accumulation. In sum, we validate response dynamics for use in preferential choice tasks and demonstrate the unique conclusions afforded by response dynamics over and above traditional methods.

Hand movements reveal the time-course of shape and pigmentation processing in face categorization

Although the roles of shape and pigmentation cues in face categorization have been studied in detail, the time-course of their processing has remained elusive. We measured participants' hand movements via the computer mouse en route to male or female responses (gender task) or young or old responses (age task) on the screen. Participants were presented with male and female faces (gender task) or with young and old faces (age task) that were typical, shape-atypical, or pigmentation-atypical. Before settling into correct categorizations, the processing of atypical cues led hand trajectories to deviate toward the opposite gender or age category. A temporal analysis of these trajectory deviations revealed dissociable dynamics in shape and pigmentation processing. Pigmentation had a privileged, early role during gender categorization, preceding shape effects by approximately 50 ms and preceding pigmentation effects in age categorization by 100 ms. In age categorization, however, pigmentation had a simultaneous onset of influence as shape. It also had a more dominant influence than shape throughout the gender and age categorization process. The results reveal the time-course of shape and pigmentation processing in gender and age categorization.

Hand in motion reveals mind in motion

Recently, researchers have measured hand movements en route to choices on a screen to understand the dynamics of a broad range of psychological processes. We review this growing body of research and explain how manual action exposes the real-time unfolding of underlying cognitive processing. We describe how simple hand motions may be used to continuously index participants’ tentative commitments to different choice alternatives during the evolution of a behavioral response. As such, hand-tracking can provide unusually high-fidelity, real-time motor traces of the mind. These motor traces cast novel theoretical and empirical light onto a wide range of phenomena and serve as a potential bridge between far-reaching areas of psychological science – from language, to high-level cognition and learning, to social cognitive processes.