Effector-specific motor simulation supplements core action recognition processes in adverse conditions

Observing other people acting activates imitative motor plans in the observer. Whether, and if so when and how, such 'effector-specific motor simulation' contributes to action recognition remains unclear. We report that individuals born without upper limbs (IDs)-who cannot covertly imitate upper-limb movements-are significantly less accurate at recognizing degraded (but not intact) upper-limb than lower-limb actions (i.e. point-light animations). This finding emphasizes the need to reframe the current controversy regarding the role of effector-specific motor simulation in action recognition: instead of focusing on the dichotomy between motor and non-motor theories, the field would benefit from new hypotheses specifying when and how effector-specific motor simulation may supplement core action recognition processes to accommodate the full variety of action stimuli that humans can recognize.

Integration of event experiences to build relational knowledge in the human brain

We investigated how the human brain integrates experiences of specific events to build general knowledge about typical event structure. We examined an episodic memory area important for temporal relations, anterior-lateral entorhinal cortex, and a semantic memory area important for action concepts, middle temporal gyrus, to understand how and when these areas contribute to these processes. Participants underwent functional magnetic resonance imaging while learning and recalling temporal relations among novel events over two sessions 1 week apart. Across distinct contexts, individual temporal relations among events could either be consistent or inconsistent with each other. Within each context, during the recall phase, we measured associative coding as the difference of multivoxel correlations among related vs unrelated pairs of events. Neural regions that form integrative representations should exhibit stronger associative coding in the consistent than the inconsistent contexts. We found evidence of integrative representations that emerged quickly in anterior-lateral entorhinal cortex (at session 1), and only subsequently in middle temporal gyrus, which showed a significant change across sessions. A complementary pattern of findings was seen with signatures during learning. This suggests that integrative representations are established early in anterior-lateral entorhinal cortex and may be a pathway to the later emergence of semantic knowledge in middle temporal gyrus.

The Impact of Limited Previous Motor Experience on Action Possibility Judgments in People with Spinal Muscle Atrophy

Previous studies have shown that people with limited motor capabilities may rely on previous motor experience when making action possibility judgments for others. In the present study, we examined if having limited previous motor experience, as a consequence of spinal muscle atrophy (SMA), alters action possibility judgments. Participants with SMA and neurologically healthy (NH) sex- and age-matched controls performed a perceptual-motor judgment task using the Fitts's law paradigm. Participants observed apparent motion videos of reciprocal aiming movements with varying levels of difficulty. For each movement, participants predicted the shortest movement time (MT) at which a neurologically healthy young adult could accurately perform the task. Participants with SMA predicted significantly longer MTs compared to controls; however, the predicted MTs of both SMA and NH participants exhibited a Fitts's law relationship (i.e., the predicted MTs significantly increased as movement difficulty increased). Overall, these results provide evidence that participants with SMA who have limited, or no motor experience may make more conservative action possibility judgments for others. Critically, our finding that the pattern of action possibility judgments was not different between SMA and NH groups suggests that limited previous motor experience may not completely impair action possibility judgments.

Novel objects with causal event schemas elicit selective responses in tool- and hand-selective lateral occipitotemporal cortex

Tool-selective lateral occipitotemporal cortex (LOTC) responds preferentially to images of tools (hammers, brushes) relative to non-tool objects (clocks, shoes). What drives these responses? Unlike other objects, tools exert effects on their surroundings. We tested whether LOTC responses are influenced by event schemas that denote different temporal relations. Participants learned about novel objects embedded in different event sequences. Causer objects moved prior to the appearance of an environmental event (e.g. stars), while Reactor objects moved after an event. Visual features and motor association were controlled. During functional magnetic resonance imaging, participants viewed still images of the objects. We localized tool-selective LOTC and non-tool-selective parahippocampal cortex (PHC) by contrasting neural responses to images of familiar tools and non-tools. We found that LOTC responded more to Causers than Reactors, while PHC did not. We also measured responses to images of hands, which elicit overlapping responses with tools. Across inferior temporal cortex, voxels' tool and hand selectivity positively predicted a preferential response to Causers. We conclude that an event schema typical of tools is sufficient to drive LOTC and that category-preferential responses across the temporal lobe may reflect relational event structures typical of those domains.

Two distinct networks containing position-tolerant representations of actions in the human brain

Humans can recognize others' actions in the social environment. This action recognition ability is rarely hindered by the movement of people in the environment. The neural basis of this position tolerance for observed actions is not fully understood. Here, we aimed to identify brain regions capable of generalizing representations of actions across different positions and investigate the representational content of these regions. In a functional magnetic resonance imaging experiment, participants viewed point-light displays of different human actions. Stimuli were presented in either the upper or the lower visual field. Multivariate pattern analysis and a surface-based searchlight approach were employed to identify brain regions that contain position-tolerant action representation: Classifiers were trained with patterns in response to stimuli presented in one position and were tested with stimuli presented in another position. Results showed above-chance classification in the left and right lateral occipitotemporal cortices, right intraparietal sulcus, and right postcentral gyrus. Further analyses exploring the representational content of these regions showed that responses in the lateral occipitotemporal regions were more related to subjective judgments, while those in the parietal regions were more related to objective measures. These results provide evidence for two networks that contain abstract representations of human actions with distinct representational content.

Two 'what' pathways for action and object recognition

The ventral visual stream is conceived as a pathway for object recognition. However, we also recognize the actions an object can be involved in. Here, we show that action recognition critically depends on a pathway in lateral occipitotemporal cortex, partially overlapping and topographically aligned with object representations that are precursors for action recognition. By contrast, object features that are more relevant for object recognition, such as color and texture, are typically found in ventral occipitotemporal cortex. We argue that occipitotemporal cortex contains similarly organized lateral and ventral 'what' pathways for action and object recognition, respectively. This account explains a number of observed phenomena, such as the duplication of object domains and the specific representational profiles in lateral and ventral cortex.

What modulates the Mirror Neuron System during action observation?: Multiple factors involving the action, the actor, the observer, the relationship between actor and observer, and the context

Seeing an agent perform an action typically triggers a motor simulation of that action in the observer's Mirror Neuron System (MNS). Over the past few years, it has become increasingly clear that during action observation the patterns and strengths of responses in the MNS are modulated by multiple factors. The first aim of this paper is therefore to provide the most comprehensive survey to date of these factors. To that end, 22 distinct factors are described, broken down into the following sets: six involving the action; two involving the actor; nine involving the observer; four involving the relationship between actor and observer; and one involving the context. The second aim is to consider the implications of these findings for four prominent theoretical models of the MNS: the Direct Matching Model; the Predictive Coding Model; the Value-Driven Model; and the Associative Model. These assessments suggest that although each model is supported by a wide range of findings, each one is also challenged by other findings and relatively unaffected by still others. Hence, there is now a pressing need for a richer, more inclusive model that is better able to account for all of the modulatory factors that have been identified so far.

Motor Imagery Training of Reaching-to-Grasp Movement Supplemented by a Virtual Environment in an Individual With Congenital Bilateral Transverse Upper-Limb Deficiency

This study explored the effect of kinesthetic motor imagery training on reaching-to-grasp movement supplemented by a virtual environment in a patient with congenital bilateral transverse upper-limb deficiency. Based on a theoretical assumption, it is possible to conduct such training in this patient. The aim of this study was to evaluate whether cortical activity related to motor imagery of reaching and motor imagery of grasping of the right upper limb was changed by computer-aided imagery training (CAIT) in a patient who was born without upper limbs compared to a healthy control subject, as characterized by multi-channel electroencephalography (EEG) signals recorded before and 4, 8, and 12 weeks after CAIT. The main task during CAIT was to kinesthetically imagine the execution of reaching-to-grasp movements without any muscle activation, supplemented by computer visualization of movements provided by a special headset. Our experiment showed that CAIT can be conducted in the patient with higher vividness of imagery for reaching than grasping tasks. Our results confirm that CAIT can change brain activation patterns in areas related to motor planning and the execution of reaching and grasping movements, and that the effect was more pronounced in the patient than in the healthy control subject. The results show that CAIT has a different effect on the cortical activity related to the motor imagery of a reaching task than on the cortical activity related to the motor imagery of a grasping task. The change observed in the activation patterns could indicate CAIT-induced neuroplasticity, which could potentially be useful in rehabilitation or brain-computer interface purposes for such patients, especially before and after transplantation. This study was part of a registered experiment (ID: NCT04048083).

Typically Efficient Lipreading without Motor Simulation

All it takes is a face-to-face conversation in a noisy environment to realize that viewing a speaker's lip movements contributes to speech comprehension. What are the processes underlying the perception and interpretation of visual speech? Brain areas that control speech production are also recruited during lipreading. This finding raises the possibility that lipreading may be supported, at least to some extent, by a covert unconscious imitation of the observed speech movements in the observer's own speech motor system-a motor simulation. However, whether, and if so to what extent, motor simulation contributes to visual speech interpretation remains unclear. In two experiments, we found that several participants with congenital facial paralysis were as good at lipreading as the control population and performed these tasks in a way that is qualitatively similar to the controls despite severely reduced or even completely absent lip motor representations. Although it remains an open question whether this conclusion generalizes to other experimental conditions and to typically developed participants, these findings considerably narrow the space of hypothesis for a role of motor simulation in lipreading. Beyond its theoretical significance in the field of speech perception, this finding also calls for a re-examination of the more general hypothesis that motor simulation underlies action perception and interpretation developed in the frameworks of motor simulation and mirror neuron hypotheses.

Towards Strong Inference in Research on Embodiment - Possibilities and Limitations of Causal Paradigms

A central question in the cognitive sciences is which role embodiment plays for high-level cognitive functions, such as conceptual processing. Here, we propose that one reason why progress regarding this question has been slow is a lacking focus on what Platt (1964) called “strong inference”. Strong inference is possible when results from an experimental paradigm are not merely consistent with a hypothesis, but they provide decisive evidence for one particular hypothesis compared to competing hypotheses. We discuss how causal paradigms, which test the functional relevance of sensory-motor processes for high-level cognitive functions, can move the field forward. In particular, we explore how congenital sensory-motor disorders, acquired sensory-motor deficits, and interference paradigms with healthy participants can be utilized as an opportunity to better understand the role of sensory experience in conceptual processing. Whereas all three approaches can bring about valuable insights, we highlight that the study of congenitally and acquired sensorimotor disorders is particularly effective in the case of conceptual domains with strong unimodal basis (e.g., colors), whereas interference paradigms with healthy participants have a broader application, avoid many of the practical and interpretational limitations of patient studies, and allow a systematic and step-wise progressive inference approach to causal mechanisms.

Evidence for an effector-independent action system from people born without hands

What are the principles of brain organization? In the motor domain, separate pathways were found for reaching and grasping actions performed by the hand. To what extent is this organization specific to the hand or based on abstract action types, regardless of which body part performs them? We tested people born without hands who perform actions with their feet. Activity in frontoparietal association motor areas showed preference for an action type (reaching or grasping), regardless of whether it was performed by the foot in people born without hands or by the hand in typically-developed controls. These findings provide evidence that some association areas are organized based on abstract functions of action types, independent of specific sensorimotor experience and parameters of specific body parts.

Many parts of the visuomotor system guide daily hand actions, like reaching for and grasping objects. Do these regions depend exclusively on the hand as a specific body part whose movement they guide, or are they organized for the reaching task per se, for any body part used as an effector? To address this question, we conducted a neuroimaging study with people born without upper limbs—individuals with dysplasia—who use the feet to act, as they and typically developed controls performed reaching and grasping actions with their dominant effector. Individuals with dysplasia have no prior experience acting with hands, allowing us to control for hand motor imagery when acting with another effector (i.e., foot). Primary sensorimotor cortices showed selectivity for the hand in controls and foot in individuals with dysplasia. Importantly, we found a preference based on action type (reaching/grasping) regardless of the effector used in the association sensorimotor cortex, in the left intraparietal sulcus and dorsal premotor cortex, as well as in the basal ganglia and anterior cerebellum. These areas also showed differential response patterns between action types for both groups. Intermediate areas along a posterior–anterior gradient in the left dorsal premotor cortex gradually transitioned from selectivity based on the body part to selectivity based on the action type. These findings indicate that some visuomotor association areas are organized based on abstract action functions independent of specific sensorimotor parameters, paralleling sensory feature-independence in visual and auditory cortices in people born blind and deaf. Together, they suggest association cortices across action and perception may support specific computations, abstracted from low-level sensorimotor elements.

What Does Dorsal Cortex Contribute to Perception?

According to the influential "Two Visual Pathways" hypothesis, the cortical visual system is segregated into two pathways, with the ventral, occipitotemporal pathway subserving object perception, and the dorsal, occipitoparietal pathway subserving the visuomotor control of action. However, growing evidence suggests that the dorsal pathway also plays a functional role in object perception. In the current article, we present evidence that the dorsal pathway contributes uniquely to the perception of a range of visuospatial attributes that are not redundant with representations in ventral cortex. We describe how dorsal cortex is recruited automatically during perception, even when no explicit visuomotor response is required. Importantly, we propose that dorsal cortex may selectively process visual attributes that can inform the perception of potential actions on objects and environments, and we consider plausible developmental and cognitive mechanisms that might give rise to these representations. As such, we consider whether naturalistic stimuli, such as real-world solid objects, might engage dorsal cortex more so than simplified or artificial stimuli such as images that do not afford action, and how the use of suboptimal stimuli might limit our understanding of the functional contribution of dorsal cortex to visual perception.

Efficient recognition of facial expressions does not require motor simulation

What mechanisms underlie facial expression recognition? A popular hypothesis holds that efficient facial expression recognition cannot be achieved by visual analysis alone but additionally requires a mechanism of motor simulation — an unconscious, covert imitation of the observed facial postures and movements. Here, we first discuss why this hypothesis does not necessarily follow from extant empirical evidence. Next, we report experimental evidence against the central premise of this view: we demonstrate that individuals can achieve normotypical efficient facial expression recognition despite a congenital absence of relevant facial motor representations and, therefore, unaided by motor simulation. This underscores the need to reconsider the role of motor simulation in facial expression recognition.

Distinct roles of temporal and frontoparietal cortex in representing actions across vision and language

Both temporal and frontoparietal brain areas are associated with the representation of knowledge about the world, in particular about actions. However, what these brain regions represent and precisely how they differ remains unknown. Here, we reveal distinct functional profiles of lateral temporal and frontoparietal cortex using fMRI-based MVPA. Frontoparietal areas encode representations of observed actions and corresponding written sentences in an overlapping way, but these representations do not generalize across stimulus type. By contrast, only left lateral posterior temporal cortex (LPTC) encodes action representations that generalize across observed action scenes and written descriptions. The representational organization of stimulus-general action information in LPTC can be predicted from models that describe basic agent-patient relations (object- and person-directedness) and the general semantic similarity between actions. Our results suggest that LPTC encodes general, conceptual aspects of actions whereas frontoparietal representations appear to be tied to specific stimulus types.

Temporal and frontoparietal brain areas both encode representations of actions, but whether they do so in different ways is unclear. Here, the authors show that only lateral posterior temporal cortex (LPTC) encodes representations that generalize across directly observed action scenes and written descriptions.