Active versus passive processing of biological motion

Effects of acute stress on biological motion perception

Biological motion perception is an essential part of the cognitive process. Stress can affect the cognitive process. The present study explored the intrinsic ERP features of the effects of acute psychological stress on biological motion perception. The results contributed scientific evidence for the adaptive behavior changes under acute stress. After a mental arithmetic task was used to induce stress, the paradigm of point-light displays was used to evaluate biological motion perception. Longer reaction time and lower accuracy were found in the inverted walking condition than in the upright walking condition, which was called the "inversion effect". The P2 peak amplitude and the LPP mean amplitude were significantly higher in the local inverted perception than in the local upright walking condition. Compared to the control condition, the stress condition induced lower RT, shorter P1 peak latency of biological motion perception, lower P2 peak amplitude and LPP mean amplitude, and higher N330 peak amplitude. There was an "inversion effect" in biological motion perception. This effect was related to the structural characteristics of biological motion perception but unrelated to the state of acute psychological stress. Acute psychological stress accelerated the reaction time and enhanced attention control of biological motion perception. Attention resources were used earlier, and less attentional investment was made in the early stage of biological motion perception processing. In the late stage, a continuous weakening of inhibition was shown in the parieto-occipital area.

Atypical local and global biological motion perception in children with attention deficit hyperactivity disorder

Perceiving biological motion (BM) is crucial for human survival and social interaction. Many studies have reported impaired BM perception in autism spectrum disorder, which is characterised by deficits in social interaction. Children with attention deficit hyperactivity disorder (ADHD) often exhibit similar difficulties in social interaction. However, few studies have investigated BM perception in children with ADHD. Here, we compared differences in the ability to process local kinematic and global configurational cues, two fundamental abilities of BM perception, between typically developing and ADHD children. We further investigated the relationship between BM perception and social interaction skills measured using the Social Responsiveness Scale and examined the contributions of latent factors (e.g. sex, age, attention, and intelligence) to BM perception. The results revealed that children with ADHD exhibited atypical BM perception. Local and global BM processing showed distinct features. Local BM processing ability was related to social interaction skills, whereas global BM processing ability significantly improved with age. Critically, general BM perception (i.e. both local and global BM processing) may be affected by sustained attentional ability in children with ADHD. This relationship was primarily mediated by reasoning intelligence. These findings elucidate atypical BM perception in ADHD and the latent factors related to BM perception. Moreover, this study provides new evidence that BM perception is a hallmark of social cognition and advances our understanding of the potential roles of local and global processing in BM perception and social cognitive disorders.

What's that on Your Phone? Effects of Mobile Device Task Type on Pedestrian Performance

BACKGROUND

The number of accidents due to distracted pedestrian is on the rise and many governments and institutions are enacting public policies which restrict texting while walking. However, pedestrians do more than just texting when they use their mobile devices on the go.

OBJECTIVE

Exploring pedestrian multitasking, this paper aims to examine the effects of mobile device task type on pedestrian performance outcomes.

METHOD

We performed two studies in lab simulations where 78 participants were asked to perform different tasks on a mobile device (playing a game, reading, writing an email, texting one person, group texting) while performing a pedestrian visual discrimination task while either standing or walking on a treadmill. Behavioral performance as well as neurophysiological data are collected.

RESULTS

Results show that compared to a no-phone control, multitasking with any of the tasks on a mobile device leads to poor performance on a pedestrian visual discrimination task. Playing a game is the most cognitively demanding task and leads to the greatest performance degradation.

CONCLUSION

Our studies show that multitasking with a mobile device has the potential to negatively impact pedestrian safety, regardless of task type. However, the impacts of different mobile device tasks are not all equivalent. More research is needed to tease out the different effects of these various tasks and to design mobile applications which effectively and safely capture pedestrians' attention.

APPLICATION

Public policy, infrastructure, and smart technologies can be used to mitigate the negative effects of mobile multitasking. A more thorough understanding of mobile device task-specific factors at play can help tailor these counter-measures to better aid distracted pedestrians.

Neural processing of bottom-up perception of biological motion under attentional load

Considering its importance for one's survival and social significance, biological motion (BM) perception is assumed to occur automatically. Previous behavioral results showed that task-irrelevant BM in the periphery interfered with task performance at the fovea. Under selective attention, BM perception is supported by a network of regions including the occipito-temporal (OTC), parietal, and premotor cortices. Retinotopy studies that use BM stimulus showed distinct maps for its processing under and away from selective attention. Based on these findings, we investigated how bottom-up perception of BM would be processed in the human brain under attentional load when it was shown away from the focus of attention as a task-irrelevant stimulus. Participants (N = 31) underwent an fMRI study in which they performed an attentionally demanding visual detection task at the fovea while intact or scrambled point light displays of BM were shown at the periphery. Our results showed the main effect of attentional load in fronto-parietal regions and both univariate activity maps and multivariate pattern analysis results support the attentional load modulation on the task-irrelevant peripheral stimuli. However, this effect was not specific to intact BM stimuli and was generalized to motion stimuli as evidenced by the motion-sensitive OTC involvement during the presence of dynamic stimuli in the periphery. These results confirm and extend previous work by showing that task-irrelevant distractors can be processed by stimulus-specific regions when there are enough attentional resources available. We discussed the implications of these results for future studies.

Need for (expected) speed: Exploring the indirect influence of trial type consistency on representational momentum

The biases affecting people's perception of dynamic stimuli are typically robust and strong for specific stimulus configurations. For example, representational momentum describes a systematic perceptual bias in the direction of motion for the final location of a moving stimulus. Under clearly defined stimulus configurations (e.g., specific stimulus identity, size, speed), for example, the frequently used "implied motion" trial sequence, for which a target is subsequently presented in a consistent direction and with a consistent speed, a displacement in motion direction is evidenced. The present study explores the potential influence of expectations regarding directional as well as speed consistencies on representational momentum, elicited by including other, inconsistently moving trial types within the same experimental block. A systematic representational momentum effect was observed when only consistent motion trials were presented. In contrast, when inconsistent target motion trials were mixed within the same block of experimental trials, the representational momentum effect decreased, or was even eliminated (Experiments 1 & 2). Detailed analysis indicated that this reflects a global (proportion of consistent and inconsistent motion trials within a particular experimental block), not local (preceding trial influencing actual trial) effect. Yet, additional follow-up studies (Experiments 3 & 4) support the idea that these changes in perceived location are strongly influenced by the overall stimulus speed statistics in the different experimental blocks. These results are discussed and interpreted in light of recent theoretical developments in the literature on motion perception that highlight the importance of expectations about stimulus speed for motion perception.

Interactionally Embedded Gestalt Principles of Multimodal Human Communication

Natural human interaction requires us to produce and process many different signals, including speech, hand and head gestures, and facial expressions. These communicative signals, which occur in a variety of temporal relations with each other (e.g., parallel or temporally misaligned), must be rapidly processed as a coherent message by the receiver. In this contribution, we introduce the notion of interactionally embedded, affordance-driven gestalt perception as a framework that can explain how this rapid processing of multimodal signals is achieved as efficiently as it is. We discuss empirical evidence showing how basic principles of gestalt perception can explain some aspects of unimodal phenomena such as verbal language processing and visual scene perception but require additional features to explain multimodal human communication. We propose a framework in which high-level gestalt predictions are continuously updated by incoming sensory input, such as unfolding speech and visual signals. We outline the constituent processes that shape high-level gestalt perception and their role in perceiving relevance and prägnanz. Finally, we provide testable predictions that arise from this multimodal interactionally embedded gestalt-perception framework. This review and framework therefore provide a theoretically motivated account of how we may understand the highly complex, multimodal behaviors inherent in natural social interaction.

Stress can lead to an increase in smartphone use in the context of texting while walking

Texting while walking (TWW) is a dangerous behavior that can lead to injury and even death. While several studies have examined the relationship between smartphone use and stress, to our knowledge no studies have yet investigated the relationship between stress and TWW. The objective of the present study was to investigate this relationship by examining the effects of stress on TWW, the effects of TWW on subsequent stress, and the effect of stress on multitasking performance. A total of 80 participants completed two sequential tasks in a laboratory while they walked on a treadmill and responded to a biological motion stimulus imitating the movement of another pedestrian. In the unrestricted task, participants were given the choice to use their personal phones. In the controlled task, they carried a text conversation with a research assistant while they walked and responded to the stimulus. Stress was measured via questionnaire and saliva collection for measure of cortisol (a stress hormone) before and after each task. Results show that greater psychological stress and cortisol variations were associated with a greater number of phone uses during the unrestricted task. Greater phone use during the unrestricted task was associated with lower subsequent psychological stress in women and total time of phone use was correlated with subsequent cortisol levels. Stress measured before the controlled task had no effect on multitasking performance, but participants with moderate performance were those with the highest cortisol levels. Our results suggest that stress could be a precursor to TWW and that it could affect a pedestrian’s ability to stay safe when using their smartphone.

Effects of spatiotemporal (dis)continuity on working memory for human movements

Human movements are dynamic and continuous in nature. However, how the spatiotemporal continuity influences working memory for movements is still unclear. Specifically, spatiotemporal continuity of movements may facilitate integrative processing ("integration") and enhance memory performance by optimizing the encoding process, but it may also diminish memory benefits from distinctive processing ("separation"). In this study, we manipulated the continuity state (continuous/discontinuous) (Experiment 1) and its predictability (Experiment 2) of whole-body movement sequences and tested participants' working memory for observed movements with a single-probe recognition task. We formulated potential influence from spatiotemporal (dis)continuity by two opposite forces - integration vs. separation, and demonstrated a conflict between these two processes across space and time. Moreover, we found that the seemingly stimulus-driven perceptual effects from spatiotemporal (dis)continuity might be supported by a prediction-based mechanism, which guided the selection of an optimal processing strategy. Overall, our finding illustrates an interweaving relationship between spatial and temporal processing during action observation and highlights the importance of considering the dynamic and continuous nature of human movements in visual perception and working memory research.

Attention modulates incidental memory encoding of human movements

Attention has been shown to enhance the processing of task-relevant information while suppressing the processing of task-irrelevant information. However, it is less clear whether this attentional modulation exists when there is an intrinsic dependence between task-relevant and task-irrelevant information, such as the dependence of temporal processing on spatial information. In this study, we used complex whole-body movement sequences to investigate the extent to which the task-irrelevant spatial information (trajectory) is processed when only the temporal information (rhythm) is in focus. Moreover, we examined, if the task-irrelevant spatial information is "co-selected" with the target temporal information as predicted by the intrinsic spatiotemporal dependence, whether task-driven attention that is actively directed to spatial information provides extra benefits. Through a two-phase experiment (an incidental encoding phase followed by a surprise memory test phase), we found that the task-irrelevant spatial information was not only perceived but also encoded in memory, providing further evidence in support of a relatively automatic co-selection of spatial information in temporal processing. Nevertheless, we also found that movements whose trajectories were intentionally attended to during the encoding phase were recognized better in the test phase than those that were not, indicating a further modulation from attention on incidental memory encoding and information processing.

Functional Segregation within the Dorsal Frontoparietal Network: A Multimodal Dynamic Causal Modeling Study

Discrimination and integration of motion direction requires the interplay of multiple brain areas. Theoretical accounts of perception suggest that stimulus-related (i.e., exogenous) and decision-related (i.e., endogenous) factors affect distributed neuronal processing at different levels of the visual hierarchy. To test these predictions, we measured brain activity of healthy participants during a motion discrimination task, using electroencephalography (EEG) and functional magnetic resonance imaging (fMRI). We independently modeled the impact of exogenous factors (task demand) and endogenous factors (perceptual decision-making) on the activity of the motion discrimination network and applied Dynamic Causal Modeling (DCM) to both modalities. DCM for event-related potentials (DCM-ERP) revealed that task demand impacted the reciprocal connections between the primary visual cortex (V1) and medial temporal areas (V5). With practice, higher visual areas were increasingly involved, as revealed by DCM-fMRI. Perceptual decision-making modulated higher levels (e.g., V5-to-Frontal Eye Fields, FEF), in a manner predictive of performance. Our data suggest that lower levels of the visual network support early, feature-based selection of responses, especially when learning strategies have not been implemented. In contrast, perceptual decision-making operates at higher levels of the visual hierarchy by integrating sensory information with the internal state of the subject.

No effect of spatial attention on the processing of a motion ensemble: Evidence from Posner cueing

The formation of ensemble codes is an efficient means through which the visual system represents vast arrays of information. This has led to the claim that ensemble representations are formed with minimal reliance on attentional resources. However, evidence is mixed regarding the effects of attention on ensemble processing, and researchers do not always make it clear how attention is being manipulated by their paradigm of choice. In this study, we examined the effects of Posner cueing - a well-established method of manipulating spatial attention - on the processing of a global motion stimulus, a naturalistic ensemble that requires the pooling of local motion signals. In Experiment 1, using a centrally presented, predictive attentional cue, we found no effect of spatial attention on global motion performance: Accuracy in invalid trials, where attention was misdirected by the cue, did not differ from accuracy in valid trials, where attention was directed to the location of the motion stimulus. In Experiment 2, we maximized the potential for our paradigm to reveal any attentional effects on global motion processing by using a threshold-based measure of performance; however, despite this change, there was again no evidence of an attentional effect on performance. Together, our results show that the processing of a global motion stimulus is unaffected when spatial attention is misdirected, and speak to the efficiency with which such ensemble stimuli are processed.

Spatiotemporal dynamics of responses to biological motion in the human brain

We sought to understand the spatiotemporal characteristics of biological motion perception. We presented observers with biological motion walkers that differed in terms of form coherence or kinematics (i.e., the presence or absence of natural acceleration). Participants were asked to discriminate the facing direction of the stimuli while their magnetoencephalographic responses were concurrently imaged. We found that two univariate response components can be observed around ~200 msec and ~650 msec post-stimulus onset, each engaging lateral-occipital and parietal cortex prior to temporal and frontal cortex. Moreover, while univariate responses show biological motion form-specificity only after 300 msec, multivariate patterns specific to form can be well discriminated from those for local cues as early as 100 msec after stimulus onset. By finally examining the representational similarity of fMRI and MEG patterned responses, we show that early responses to biological motion are most likely sourced to occipital cortex while later responses likely originate from extrastriate body areas.

Comparable search efficiency for human and animal targets in the context of natural scenes

In a previous series of studies, we have shown that search for human targets in the context of natural scenes is more efficient than search for mechanical targets. Here we asked whether this search advantage extends to other categories of biological objects. We used videos of natural scenes to directly contrast search efficiency for animal and human targets among biological or nonbiological distractors. In visual search arrays consisting of two, four, six, or eight videos, observers searched for animal targets among machine distractors, and vice versa (Exp. 1). Another group searched for animal targets among human distractors, and vice versa (Exp. 2). We measured search slope as a proxy for search efficiency, and complemented the slope with eye movement measurements (fixation duration on the target, as well as the proportion of first fixations landing on the target). In both experiments, we observed no differences in search slopes or proportions of first fixations between any of the target-distractor category pairs. With respect to fixation durations, we found shorter on-target fixations only for animal targets as compared to machine targets (Exp. 1). In summary, we did not find that the search advantage for human targets over mechanical targets extends to other biological objects. We also found no search advantage for detecting humans as compared to other biological objects. Overall, our pattern of findings suggests that search efficiency in natural scenes, as elsewhere, depends crucially on the specific target-distractor categories.

Ensemble coding of crowd speed using biological motion

The accurate perception of human crowds is integral to social understanding and interaction. Previous studies have shown that observers are sensitive to several crowd characteristics such as average facial expression, gender, identity, joint attention, and heading direction. In two experiments, we examined ensemble perception of crowd speed using standard point-light walkers (PLW). Participants were asked to estimate the average speed of a crowd consisting of 12 figures moving at different speeds. In Experiment 1, trials of intact PLWs alternated with trials of scrambled PLWs with a viewing duration of 3 seconds. We found that ensemble processing of crowd speed could rely on local motion alone, although a globally intact configuration enhanced performance. In Experiment 2, observers estimated the average speed of intact-PLW crowds that were displayed at reduced viewing durations across five blocks of trials (between 2500 ms and 500 ms). Estimation of fast crowds was precise and accurate regardless of viewing duration, and we estimated that three to four walkers could still be integrated at 500 ms. For slow crowds, we found a systematic deterioration in performance as viewing time reduced, and performance at 500 ms could not be distinguished from a single-walker response strategy. Overall, our results suggest that rapid and accurate ensemble perception of crowd speed is possible, although sensitive to the precise speed range examined.

Using a smartphone while walking: The cost of smartphone-addiction proneness

Distracted walking is an ever-increasing problem. Studies have already shown that using a smartphone while walking impairs attention and increases the risk of accidents. This study seeks to determine if smartphone-addiction proneness magnifies the risks of using a smartphone while walking. In an experimental design, participants, while walking on a treadmill and engaged in a smartphone task, were required to switch tasks by responding to an external stimulus, i.e., determining the direction of movement of a point-light walker. Participants were chosen to cover a range of smartphone-addiction proneness. Four smartphone-use conditions were simulated: a control condition with no smartphone-use, an individual conversation condition, a gaming condition, and a group conversation condition. Our results show that using a smartphone while walking decreases accuracy and increases the number of missed stimuli. Moreover, participants with higher smartphone-addiction proneness scores were also prone to missing more stimuli, and this effect was found regardless of experimental condition. The effect of the smartphone task on accuracy and the number of missed stimuli was mediated by the emotional arousal caused by the smartphone task. Smartphone-addiction proneness was positively correlated with a declared frequency of smartphone use while walking. Furthermore, of all the smartphone tasks, the gaming condition was found to be the most distracting.

Visual attention, biological motion perception, and healthy ageing

Biological motion perception is the ability of the visual system to perceive complex human movement patterns. The previous studies have shown a direct link between attentional abilities and performance on biological motion tasks, both of which have been shown to deteriorate with age. However, it is not known whether there is a direct link between age-related deficits in biological motion processing and attention. Here, we investigated whether age-related changes in biological motion perception are mediated by impaired attentional abilities. To assess basic biological motion performance, we asked 42 younger (M = 21 years) and 39 older adults (M = 69 years) to indicate the facing direction of point-light actions. Performance did not differ between age groups. We assessed visual spatial and selective attentional abilities, using a range of tasks: conjunctive visual search, spatial cueing, and the Stroop task. Across all tasks, older adults were significantly slower to respond and exhibited larger interference/cueing effects, compared to younger adults. To assess attentional demands in relation with biological motion perception, participants performed a biological motion search task for which they had to indicate the presence of a target point-light walker among a varied number of distracters. Older adults were slower, and generally worse than younger adults at discriminating the walkers. Correlations showed that there was no significant relationship between performance in attention tasks and biological motion processing, which indicates that age-related changes in biological motion perception are unlikely to be driven by general attentional decline.

The two-process theory of biological motion processing

Perception, identification, and understanding of others' actions from motion information are vital for our survival in the social world. A breakthrough in the understanding of action perception was the discovery that our visual system is sensitive to human action from the sparse motion input of only a dozen point lights, a phenomenon known as biological motion (BM) processing. Previous psychological and computational models cannot fully explain the emerging evidence for the existence of BM processing during early ontogeny. Here, we propose a two-process model of the mechanisms underlying BM processing. We hypothesize that the first system, the 'Step Detector,' rapidly processes the local foot motion and feet-below-the-body information that is specific to vertebrates, is less dependent on postnatal learning, and involves subcortical networks. The second system, the 'Bodily Action Evaluator,' slowly processes the fine global structure-from-motion, is specific to conspecific, and dependent on gradual learning processed in cortical networks. This proposed model provides new insight into research on the development of BM processing.

Relation Between Working Memory Capacity of Biological Movements and Fluid Intelligence

Studies have revealed that there is an independent buffer for holding biological movements (BM) in working memory (WM), and this BM-WM has a unique link to our social ability. However, it remains unknown as to whether the BM-WM also correlates to our cognitive abilities, such as fluid intelligence (Gf). Since BM processing has been considered as a hallmark of social cognition, which distinguishes from canonical cognitive abilities in many ways, it has been hypothesized that only canonical object-WM (e.g., memorizing color patches), but not BM-WM, emerges to have an intimate relation with Gf. We tested this prediction by measuring the relationship between WM capacity of BM and Gf. With two Gf measurements, we consistently found moderate correlations between BM-WM capacity, the score of both Raven's advanced progressive matrix (RAPM), and the Cattell culture fair intelligence test (CCFIT). This result revealed, for the first time, a close relation between WM and Gf with a social stimulus, and challenged the double-dissociation hypothesis for distinct functions of different WM buffers.

Texting while walking: An expensive switch cost

Texting while walking has been highlighted as a dangerous behavior that leads to impaired judgment and accidents. This impairment could be due to task switching which involves activation of the present task and the inhibition of the previous task. However, the relative contributions of these processes and their brain activity have not yet been studied. We addressed this gap by asking participants to discriminate the orientation of an oncoming human shape in a virtual environment while they were: i) walking on a treadmill, or ii) texting while walking on a treadmill. Participants' performance (i.e., correctly identifying if a walker would pass them to their left or right) and electroencephalography (EEG) data was collected. Unsurprisingly, we found that participants performed better while they were only walking than when texting while walking. However, we also found that the diminished performance is differently related to task set inhibition and task set activation in the two conditions. The alpha oscillations, which can be used as an index of task inhibition, have a significantly different relation to performance in the two conditions, the relation being negative when subjects are texting. This may indicate that the more inhibition is needed, the more the performance is affected by texting. To our knowledge, this is the first study to investigate the brain signature of task switching in texting while walking. This finding is the first step in identifying the source of impaired judgment in texting pedestrians and in finding viable solutions to reduce the risks.

Structural and effective brain connectivity underlying biological motion detection

The perception of actions underwrites a wide range of socio-cognitive functions. Previous neuroimaging and lesion studies identified several components of the brain network for visual biological motion (BM) processing, but interactions among these components and their relationship to behavior remain little understood. Here, using a recently developed integrative analysis of structural and effective connectivity derived from high angular resolution diffusion imaging (HARDI) and functional magnetic resonance imaging (fMRI), we assess the cerebro-cerebellar network for processing of camouflaged point-light BM. Dynamic causal modeling (DCM) informed by probabilistic tractography indicates that the right superior temporal sulcus (STS) serves as an integrator within the temporal module. However, the STS does not appear to be a "gatekeeper" in the functional integration of the occipito-temporal and frontal regions: The fusiform gyrus (FFG) and middle temporal cortex (MTC) are also connected to the right inferior frontal gyrus (IFG) and insula, indicating multiple parallel pathways. BM-specific loops of effective connectivity are seen between the left lateral cerebellar lobule Crus I and right STS, as well as between the left Crus I and right insula. The prevalence of a structural pathway between the FFG and STS is associated with better BM detection. Moreover, a canonical variate analysis shows that the visual sensitivity to BM is best predicted by BM-specific effective connectivity from the FFG to STS and from the IFG, insula, and STS to the early visual cortex. Overall, the study characterizes the architecture of the cerebro-cerebellar network for BM processing and offers prospects for assessing the social brain.

PLAViMoP: How to standardize and simplify the use of point-light displays

The study of biological point-light displays (PLDs) has fascinated researchers for more than 40 years. However, the mechanisms underlying PLD perception remain unclear, partly due to difficulties with precisely controlling and transforming PLD sequences. Furthermore, little agreement exists regarding how transformations are performed. This article introduces a new free-access program called PLAViMoP (Point-Light Display Visualization and Modification Platform) and presents the algorithms for PLD transformations actually included in the software. PLAViMoP fulfills two objectives. First, it standardizes and makes clear many classical spatial and kinematic transformations described in the PLD literature. Furthermore, given its optimized interface, PLAViMOP makes these transformations easy and fast to achieve. Overall, PLAViMoP could directly help scientists avoid technical difficulties and make possible the use of PLDs for nonacademic applications.

Unique Views on Obesity-Related Behaviors and Environments: Research Using Still and Video Images

OBJECTIVES

To document challenges to and benefits from research involving the use of images by capturing examples of such research to assess physical activity- or nutrition-related behaviors and/or environments.

METHODS

Researchers (i.e., key informants) using image capture in their research were identified through knowledge and networks of the authors of this paper and through literature search. Twenty-nine key informants completed a survey covering the type of research, source of images, and challenges and benefits experienced, developed specifically for this study.

RESULTS

Most respondents used still images in their research, with only 26.7% using video. Image sources were categorized as participant generated (n = 13; e.g., participants using smartphones for dietary assessment), researcher generated (n = 10; e.g., wearable cameras with automatic image capture), or curated from third parties (n = 7; e.g., Google Street View). Two of the major challenges that emerged included the need for automated processing of large datasets (58.8%) and participant recruitment/compliance (41.2%). Benefit-related themes included greater perspectives on obesity with increased data coverage (34.6%) and improved accuracy of behavior and environment assessment (34.6%).

CONCLUSIONS

Technological advances will support the increased use of images in the assessment of physical activity, nutrition behaviors, and environments. To advance this area of research, more effective collaborations are needed between health and computer scientists. In particular development of automated data extraction methods for diverse aspects of behavior, environment, and food characteristics are needed. Additionally, progress in standards for addressing ethical issues related to image capture for research purposes is critical.

Heading Through a Crowd

The ability to navigate through crowds of moving people accurately, efficiently, and without causing collisions is essential for our day-to-day lives. Vision provides key information about one's own self-motion as well as the motions of other people in the crowd. These two types of information (optic flow and biological motion) have each been investigated extensively; however, surprisingly little research has been dedicated to investigating how they are processed when presented concurrently. Here, we showed that patterns of biological motion have a negative impact on visual-heading estimation when people within the crowd move their limbs but do not move through the scene. Conversely, limb motion facilitates heading estimation when walkers move independently through the scene. Interestingly, this facilitation occurs for crowds containing both regular and perturbed depictions of humans, suggesting that it is likely caused by low-level motion cues inherent in the biological motion of other people.

Emotional cues and social anxiety resolve ambiguous perception of biological motion

Perceptions of ambiguous biological motion are modulated by different individual cognitive abilities (such as inhibition and empathy) and emotional states (such as anxiety). This study explored facing-the-viewer bias (FTV) in perceiving ambiguous directions of biological motion, and investigated whether task-irrelevant simultaneous face emotional cues in the background and the individual social anxiety traits could affect FTV. We found that facial motion cues as background affect sociobiologically relevant scenarios, including biological motion, but not non-biological situations (conveyed through random dot motion). Individuals with high anxiety traits demonstrated a more dominant FTV bias than individuals with low anxiety traits. Ensemble coding-like processing of task-irrelevant multiple emotional cues could magnify the facing-the-viewer bias than did in the single emotional cue. Overall, those findings suggest a correlation between high-level emotional processing and high-level motion perception (subjective to attentional control) contributes to facing-the-viewer bias.

Heritable aspects of biological motion perception and its covariation with autistic traits

The ability to detect biological motion (BM) and decipher the meaning therein is essential to human survival and social interaction. However, at the individual level, we are not equally equipped with this ability. In particular, impaired BM perception and abnormal neural responses to BM have been observed in autism spectrum disorder (ASD), a highly heritable neurodevelopmental disorder characterized by devastating social deficits. Here, we examined the underlying sources of individual differences in two abilities fundamental to BM perception (i.e., the abilities to process local kinematic and global configurational information of BM) and explored whether BM perception shares a common genetic origin with autistic traits. Using the classical twin method, we found reliable genetic influences on BM perception and revealed a clear dissociation between its two components-whereas genes account for about 50% of the individual variation in local BM processing, global BM processing is largely shaped by environment. Critically, participants' sensitivity to local BM cues was negatively correlated with their autistic traits through the dimension of social communication, with the covariation largely mediated by shared genetic effects. These findings demonstrate that the ability to process BM, especially with regard to its inherent kinetics, is heritable. They also advance our understanding of the sources of the linkage between autistic symptoms and BM perception deficits, opening up the possibility of treating the ability to process local BM information as a distinct hallmark of social cognition.

First, you need a Gestalt: An interaction of bottom-up and top-down streams during the perception of the ambiguously rotating human walker

Our visual system combines sensory evidence with prior knowledge to produce a representation of an outside world. Here, we explored the limits of the feedforward computation using an ambiguously rotating human biological motion. Specifically, we investigated whether an overall rotation, which was added to all the displays used in the study, would be perceived when the point-light walker was presented upside-down, a condition that typically obliterates perception of a human Gestalt. We report that inversion of the point-light walker or the absence of an identifiable Gestalt abolished the perception of an overall rotation. Perception of rotation was restored if the human walker Gestalt could be identified (an upright walker), if observers were informed about the nature of the motion display, or if observers expected to see the rotation of an unknown dynamic object. This implies that a mathematically more complex human motion was accounted for before the remaining motion components could be used to infer an overall rotation. Our results indicate that the perceptual inference does not proceed in a hierarchical manner with the simpler components being identified first. Instead, prior knowledge acts as a starting point for the decomposition of an even relatively simple combination of two motions.

Bistability and Biasing Effects in the Perception of Ambiguous Point-Light Walkers

The perceptually bistable character of point-light walkers has been examined in three experiments. A point-light figure without explicit depth cues constitutes a perfectly ambiguous stimulus: from all viewpoints, multiple interpretations are possible concerning the depth orientation of the figure. In the first experiment, it is shown that non-lateral views of the walker are indeed interpreted in two orientations, either as facing towards the viewer or as facing away from the viewer, but that the interpretation in which the walker is oriented towards the viewer is reported more frequently. In the second experiment the point-light figure was walking backwards, making the global orientation of the point-light figure opposite to the direction of global motion. The interpretation in which the walker was facing the viewer was again reported more frequently. The robustness of these findings was examined in the final experiment, in which the effects of disambiguating the stimulus by introducing a local depth cue (occlusion) or a more global depth cue (applying perspective projection) were explored.

Do People "Pop Out"?

The human body is a highly familiar and socially very important object. Does this mean that the human body has a special status with respect to visual attention? In the current paper we tested whether people in natural scenes attract attention and “pop out” or, alternatively, are at least searched for more efficiently than targets of another category (machines). Observers in our study searched a visual array for dynamic or static scenes containing humans amidst scenes containing machines and vice versa. The arrays consisted of 2, 4, 6 or 8 scenes arranged in a circular array, with targets being present or absent. Search times increased with set size for dynamic and static human and machine targets, arguing against pop out. However, search for human targets was more efficient than for machine targets as indicated by shallower search slopes for human targets. Eye tracking further revealed that observers made more first fixations to human than to machine targets and that their on-target fixation durations were shorter for human compared to machine targets. In summary, our results suggest that searching for people in natural scenes is more efficient than searching for other categories even though people do not pop out.

Self-organizing neural integration of pose-motion features for human action recognition

The visual recognition of complex, articulated human movements is fundamental for a wide range of artificial systems oriented toward human-robot communication, action classification, and action-driven perception. These challenging tasks may generally involve the processing of a huge amount of visual information and learning-based mechanisms for generalizing a set of training actions and classifying new samples. To operate in natural environments, a crucial property is the efficient and robust recognition of actions, also under noisy conditions caused by, for instance, systematic sensor errors and temporarily occluded persons. Studies of the mammalian visual system and its outperforming ability to process biological motion information suggest separate neural pathways for the distinct processing of pose and motion features at multiple levels and the subsequent integration of these visual cues for action perception. We present a neurobiologically-motivated approach to achieve noise-tolerant action recognition in real time. Our model consists of self-organizing Growing When Required (GWR) networks that obtain progressively generalized representations of sensory inputs and learn inherent spatio-temporal dependencies. During the training, the GWR networks dynamically change their topological structure to better match the input space. We first extract pose and motion features from video sequences and then cluster actions in terms of prototypical pose-motion trajectories. Multi-cue trajectories from matching action frames are subsequently combined to provide action dynamics in the joint feature space. Reported experiments show that our approach outperforms previous results on a dataset of full-body actions captured with a depth sensor, and ranks among the best results for a public benchmark of domestic daily actions.

Eye tracking reveals a crucial role for facial motion in recognition of faces by infants

Current knowledge about face processing in infancy comes largely from studies using static face stimuli, but faces that infants see in the real world are mostly moving ones. To bridge this gap, 3-, 6-, and 9-month-old Asian infants (N = 118) were familiarized with either moving or static Asian female faces, and then their face recognition was tested with static face images. Eye-tracking methodology was used to record eye movements during the familiarization and test phases. The results showed a developmental change in eye movement patterns, but only for the moving faces. In addition, the more infants shifted their fixations across facial regions, the better their face recognition was, but only for the moving faces. The results suggest that facial movement influences the way faces are encoded from early in development.

Visual control of an action discrimination in pigeons

Recognizing and categorizing behavior is essential for all animals. The visual and cognitive mechanisms underlying such action discriminations are not well understood, especially in nonhuman animals. To identify the visual bases of action discriminations, four pigeons were tested in a go/no-go procedure to examine the contribution of different visual features in a discrimination of walking and running actions by different digital animal models. Two different tests with point-light displays derived from studies of human biological motion failed to support transfer of the learned action discrimination from fully figured models. Tests with silhouettes, contours, and the selective deletion or occlusion of different parts of the models indicated that information about the global motions of the entire model was critical to the discrimination. This outcome, along with earlier results, suggests that the pigeons’ discrimination of these locomotive actions involved a generalized categorization of the sequence of configural poses. Because the motor systems for locomotion and flying in pigeons share little in common with quadruped motions, the pigeons’ discrimination of these behaviors creates problems for motor theories of action recognition based on mirror neurons or related notions of embodied cognition. It suggests instead that more general motion and shape mechanisms are sufficient for making such discriminations, at least in birds.

Multiple object tracking in autism spectrum disorders

Difficulties in visual attention are often implicated in autism spectrum disorders (ASD) but it remains unclear which aspects of attention are affected. Here, we used a multiple object tracking (MOT) task to quantitatively characterize dynamic attentional function in children with ASD aged 5-12. While the ASD group performed significantly worse overall, the group difference did not increase with increased object speed. This finding suggests that decreased MOT performance is not due to deficits in dynamic attention but instead to a diminished capacity to select and maintain attention on multiple targets. Further, MOT performance improved from 5 to 10 years in both typical and ASD groups with similar developmental trajectories. These results argue against a specific deficit in dynamic attention in ASD.

Biological motion coding in the brain: analysis of visually driven EEG functional networks

Herein, we address the time evolution of brain functional networks computed from electroencephalographic activity driven by visual stimuli. We describe how these functional network signatures change in fast scale when confronted with point-light display stimuli depicting biological motion (BM) as opposed to scrambled motion (SM). Whereas global network measures (average path length, average clustering coefficient, and average betweenness) computed as a function of time did not discriminate between BM and SM, local node properties did. Comparing the network local measures of the BM condition with those of the SM condition, we found higher degree and betweenness values in the left frontal (F7) electrode, as well as a higher clustering coefficient in the right occipital (O2) electrode, for the SM condition. Conversely, for the BM condition, we found higher degree values in central parietal (Pz) electrode and a higher clustering coefficient in the left parietal (P3) electrode. These results are discussed in the context of the brain networks involved in encoding BM versus SM.

Deficient biological motion perception in schizophrenia: results from a motion noise paradigm

Background: Schizophrenia patients exhibit deficient processing of perceptual and cognitive information. However, it is not well-understood how basic perceptual deficits contribute to higher level cognitive problems in this mental disorder. Perception of biological motion, a motion-based cognitive recognition task, relies on both basic visual motion processing and social cognitive processing, thus providing a useful paradigm to evaluate the potentially hierarchical relationship between these two levels of information processing.

Methods: In this study, we designed a biological motion paradigm in which basic visual motion signals were manipulated systematically by incorporating different levels of motion noise. We measured the performances of schizophrenia patients (n = 21) and healthy controls (n = 22) in this biological motion perception task, as well as in coherent motion detection, theory of mind, and a widely used biological motion recognition task.

Results: Schizophrenia patients performed the biological motion perception task with significantly lower accuracy than healthy controls when perceptual signals were moderately degraded by noise. A more substantial degradation of perceptual signals, through using additional noise, impaired biological motion perception in both groups. Performance levels on biological motion recognition, coherent motion detection and theory of mind tasks were also reduced in patients.

Conclusion: The results from the motion-noise biological motion paradigm indicate that in the presence of visual motion noise, the processing of biological motion information in schizophrenia is deficient. Combined with the results of poor basic visual motion perception (coherent motion task) and biological motion recognition, the association between basic motion signals and biological motion perception suggests a need to incorporate the improvement of visual motion perception in social cognitive remediation.

Sensing, assessing, and augmenting threat detection: behavioral, neuroimaging, and brain stimulation evidence for the critical role of attention

Rapidly identifying the potentially threatening movements of other people and objects—biological motion perception and action understanding—is critical to maintaining security in many civilian and military settings. A key approach to improving threat detection in these environments is to sense when less than ideal conditions exist for the human observer, assess that condition relative to an expected standard, and if necessary use tools to augment human performance. Action perception is typically viewed as a relatively “primitive,” automatic function immune to top-down effects. However, recent research shows that attention is a top-down factor that has a critical influence on the identification of threat-related targets. In this paper we show that detection of motion-based threats is attention sensitive when surveillance images are obscured by other movements, when they are visually degraded, when other stimuli or tasks compete for attention, or when low-probability threats must be watched for over long periods of time—all features typical of operational security settings. Neuroimaging studies reveal that action understanding recruits a distributed network of brain regions, including the superior temporal cortex, intraparietal cortex, and inferior frontal cortex. Within this network, attention modulates activation of the superior temporal sulcus (STS) and middle temporal gyrus. The dorsal frontoparietal network may provide the source of attention-modulation signals to action representation areas. Stimulation of this attention network should therefore enhance threat detection. We show that transcranial Direct Current Stimulation (tDCS) at 2 mA accelerates perceptual learning of participants performing a challenging threat-detection task. Together, cognitive, neuroimaging, and brain stimulation studies provide converging evidence for the critical role of attention in the detection and understanding of threat-related intentional actions.

Auditory coding of human movement kinematics

Although visual perception is dominant on motor perception, control and learning, auditory information can enhance and modulate perceptual as well as motor processes in a multifaceted manner. During last decades new methods of auditory augmentation had been developed with movement sonification as one of the most recent approaches expanding auditory movement information also to usually mute phases of movement. Despite general evidence on the effectiveness of movement sonification in different fields of applied research there is nearly no empirical proof on how sonification of gross motor human movement should be configured to achieve information rich sound sequences. Such lack of empirical proof is given for (a) the selection of suitable movement features as well as for (b) effective kinetic-acoustical mapping patterns and for (c) the number of regarded dimensions of sonification. In this study we explore the informational content of artificial acoustical kinematics in terms of a kinematic movement sonification using an intermodal discrimination paradigm. In a repeated measure design we analysed discrimination rates of six everyday upper limb actions to evaluate the effectiveness of seven different kinds of kinematic-acoustical mappings as well as short-term learning effects. The kinematics of the upper limb actions were calculated based on inertial motion sensor data and transformed into seven different sonifications. Sound sequences were randomly presented to participants and discrimination rates as well as confidence of choice were analysed. Data indicate an instantaneous comprehensibility of the artificial movement acoustics as well as short-term learning effects. No differences between different dimensional encodings became evident thus indicating a high efficiency for intermodal pattern discrimination for the acoustically coded velocity distribution of the actions. Taken together movement information related to continuous kinematic parameters can be transformed into the auditory domain. Additionally, pattern based action discrimination is obviously not restricted to the visual modality. Artificial acoustical kinematics might be used to supplement and/or substitute visual motion perception in sports and motor rehabilitation.

Body Configuration Modulates the Usage of Local Cues to Direction in Biological-Motion Perception

The presence of information in a visual display does not guarantee its use by the visual system. Studies of inversion effects in both face recognition and biological-motion perception have shown that the same information may be used by observers when it is presented in an upright display but not used when the display is inverted. In our study, we tested the inversion effect in scrambled biological-motion displays to investigate mechanisms that validate information contained in the local motion of a point-light walker. Using novel biological-motion stimuli that contained no configural cues to the direction in which a walker was facing, we found that manipulating the relative vertical location of the walker’s feet significantly affected observers’ performance on a direction-discrimination task. Our data demonstrate that, by themselves, local cues can almost unambiguously indicate the facing direction of the agent in biological-motion stimuli. Additionally, we document a noteworthy interaction between local and global information and offer a new explanation for the effect of local inversion in biological-motion perception.

Effects of visual context upon functional connectivity during observation of biological motions

The aim of this study was to examine brain responses, in particular functional connectivity, to different visual stimuli depicting familiar biological motions. Ten subjects actively observed familiar biological motions embedded in point-light and video displays. Electroencephalograms were recorded from 64 electrodes. Activity was considered in three frequency bands (4-8 Hz, 8-10 Hz, and 10-13 Hz) using a non-linear measure of functional connectivity. In the 4-8 Hz and 8-10 Hz frequency bands, functional connectivity for the SMA was greater during the observation of biological motions presented in a point-light display compared to the observation of motions presented in a video display. The reverse was observed for the 4-8 Hz frequency band for the left temporal area. Explanations related to: (i) the task demands (i.e., attention and mental effort), (ii) the role(s) of theta and alpha oscillations in cognitive processes, and (iii) the function(s) of cortical areas are discussed. It has been suggested that attention was required to process human biological motions under unfamiliar viewing conditions such as point-light display.

Neural correlates of coherent and biological motion perception in autism

Recent evidence suggests those with autism may be generally impaired in visual motion perception. To examine this, we investigated both coherent and biological motion processing in adolescents with autism employing both psychophysical and fMRI methods. Those with autism performed as well as matched controls during coherent motion perception but had significantly higher thresholds for biological motion perception. The autism group showed reduced posterior Superior Temporal Sulcus (pSTS), parietal and frontal activity during a biological motion task while showing similar levels of activity in MT+/V5 during both coherent and biological motion trials. Activity in MT+/V5 was predictive of individual coherent motion thresholds in both groups. Activity in dorsolateral prefrontal cortex (DLPFC) and pSTS was predictive of biological motion thresholds in control participants but not in those with autism. Notably, however, activity in DLPFC was negatively related to autism symptom severity. These results suggest that impairments in higher-order social or attentional networks may underlie visual motion deficits observed in autism.

From body form to biological motion: the apparent velocity of human movement biases subjective time

In two experiments, we investigated time perception during apparent biological motion. Pictures of initial, intermediate, and final positions of a single movement were presented, with interstimulus intervals that were constant within trials but varied across trials. Movement paths were manipulated by changing the sequential order of body postures. Increasing the path length produced an increase in perceived movement velocity. To produce an implicit measure of apparent movement dynamics, we also asked participants to judge the duration of a frame surrounding the stimuli. Longer paths with higher apparent movement velocity produced shorter perceived durations. This temporal bias was attenuated for nonbody (Experiment 1) and inverted-body (Experiment 2) control stimuli. As an explanation for these findings, we propose an automatic top-down mechanism of biological-motion perception that binds successive body postures into a continuous perception of movement. We show that this mechanism is associated with velocity-dependent temporal compression. Furthermore, this mechanism operates on-line, bridging the intervals between static stimuli, and is specific to configural processing of body form.

Attention, biological motion, and action recognition

Interacting with others in the environment requires that we perceive and recognize their movements and actions. Neuroimaging and neuropsychological studies have indicated that a number of brain regions, particularly the superior temporal sulcus, are involved in a number of processes essential for action recognition, including the processing of biological motion and processing the intentions of actions. We review the behavioral and neuroimaging evidence suggesting that while some aspects of action recognition might be rapid and effective, they are not necessarily automatic. Attention is particularly important when visual information about actions is degraded or ambiguous, or if competing information is present. We present evidence indicating that neural responses associated with the processing of biological motion are strongly modulated by attention. In addition, behavioral and neuroimaging evidence shows that drawing inferences from the actions of others is attentionally demanding. The role of attention in action observation has implications for everyday social interactions and workplace applications that depend on observing, understanding and interpreting actions.

Perception of biological motion in schizophrenia and healthy individuals: a behavioral and FMRI study

BACKGROUND

Anomalous visual perception is a common feature of schizophrenia plausibly associated with impaired social cognition that, in turn, could affect social behavior. Past research suggests impairment in biological motion perception in schizophrenia. Behavioral and functional magnetic resonance imaging (fMRI) experiments were conducted to verify the existence of this impairment, to clarify its perceptual basis, and to identify accompanying neural concomitants of those deficits.

METHODOLOGY/FINDINGS

In Experiment 1, we measured ability to detect biological motion portrayed by point-light animations embedded within masking noise. Experiment 2 measured discrimination accuracy for pairs of point-light biological motion sequences differing in the degree of perturbation of the kinematics portrayed in those sequences. Experiment 3 measured BOLD signals using event-related fMRI during a biological motion categorization task. Compared to healthy individuals, schizophrenia patients performed significantly worse on both the detection (Experiment 1) and discrimination (Experiment 2) tasks. Consistent with the behavioral results, the fMRI study revealed that healthy individuals exhibited strong activation to biological motion, but not to scrambled motion in the posterior portion of the superior temporal sulcus (STSp). Interestingly, strong STSp activation was also observed for scrambled or partially scrambled motion when the healthy participants perceived it as normal biological motion. On the other hand, STSp activation in schizophrenia patients was not selective to biological or scrambled motion.

CONCLUSION

Schizophrenia is accompanied by difficulties discriminating biological from non-biological motion, and associated with those difficulties are altered patterns of neural responses within brain area STSp. The perceptual deficits exhibited by schizophrenia patients may be an exaggerated manifestation of neural events within STSp associated with perceptual errors made by healthy observers on these same tasks. The present findings fit within the context of theories of delusion involving perceptual and cognitive processes.