Linguistic and motor representations of everyday complex actions: an fNIRS investigation

Awareness of embodiment enhances enjoyment and engages sensorimotor cortices

Whether in performing arts, sporting, or everyday contexts, when we watch others move, we tend to enjoy bodies moving in synchrony. Our enjoyment of body movements is further enhanced by our own prior experience with performing those movements, or our 'embodied experience'. The relationships between movement synchrony and enjoyment, as well as embodied experience and movement enjoyment, are well known. The interaction between enjoyment of movements, synchrony, and embodiment is less well understood, and may be central for developing new approaches for enriching social interaction. To examine the interplay between movement enjoyment, synchrony, and embodiment, we asked participants to copy another person's movements as accurately as possible, thereby gaining embodied experience of movement sequences. Participants then viewed other dyads performing the same or different sequences synchronously, and we assessed participants' recognition of having performed these sequences, as well as their enjoyment of each movement sequence. We used functional near-infrared spectroscopy to measure cortical activation over frontotemporal sensorimotor regions while participants performed and viewed movements. We found that enjoyment was greatest when participants had mirrored the sequence and recognised it, suggesting that awareness of embodiment may be central to enjoyment of synchronous movements. Exploratory analyses of relationships between cortical activation and enjoyment and recognition implicated the sensorimotor cortices, which subserve action observation and aesthetic processing. These findings hold implications for clinical research and therapies seeking to foster successful social interaction.

Differential responses in the mirror neuron system during imitation of individual emotional facial expressions and association with autistic traits

The mirror neuron system (MNS), including the inferior frontal gyrus (IFG), inferior parietal lobule (IPL) and superior temporal sulcus (STS) plays an important role in action representation and imitation and may be dysfunctional in autism spectrum disorder (ASD). However, it's not clear how these three regions respond and interact during the imitation of different basic facial expressions and whether the pattern of responses is influenced by autistic traits. Thus, we conducted a natural facial expression (happiness, angry, sadness and fear) imitation task in 100 healthy male subjects where expression intensity was measured using facial emotion recognition software (FaceReader) and MNS responses were recorded using functional near-infrared spectroscopy (fNIRS). Autistic traits were measured using the Autism Spectrum Quotient questionnaire. Results showed that imitation of happy expressions produced the highest expression intensity but a small deactivation in MNS responses, suggesting a lower processing requirement compared to other expressions. A cosine similarity analysis indicated a distinct pattern of MNS responses during imitation of each facial expression with functional intra-hemispheric connectivity between the left IPL and left STS being significantly higher during happy compared to other expressions, while inter-hemispheric connectivity between the left and right IPL differed between imitation of fearful and sad expressions. Furthermore, functional connectivity changes during imitation of each different expression could reliably predict autistic trait scores. Overall, the results provide evidence for distinct patterns of functional connectivity changes between MNS regions during imitation of different emotions which are also associated with autistic traits.

Body experience influences lexical-semantic knowledge of body parts in children with hemiplegic cerebral palsy

Background

Disorders in different levels of body representation (i.e., body schema, body structural description, and body image) are present in hemiplegic cerebral palsy (HCP). However, it remains unclear whether the body image develops from aspects of body schema and body structural description, and how this occurs in children with HCP.

Objective and methods

In a cross-sectional study, we investigated 53 children with HCP (mean age about 10 years) and 204 typically developing (TD) control children to qualitatively evaluate whether and how body schema (related sensorimotor experiences) and body structural description (related visuospatial experiences) affect the development of children’s body image and whether this development is delayed through HCP. Graph analysis was used to create a lexical-semantic map of body representation from data of a semantic word fluency task.

Results

Results indicated a similar qualitative pattern of influences of sensorimotor and visuospatial experiences on lexical-semantic knowledge of body parts, with a delayed developmental course in children with HCP compared to TD children.

Conclusion

These findings suggest that children’s body image seemed to be influenced by body schema and body structural descriptions as indicated by poorer lexical-semantic knowledge of body parts in children with HCP due to missing physical experiences of the affected body parts. This might imply that “body talk” may beneficially complement physical therapy for children with HCP to promote body image development.

Structured sparse multiset canonical correlation analysis of simultaneous fNIRS and EEG provides new insights into the human action-observation network

The action observation network (AON) is a network of brain regions involved in the execution and observation of a given action. The AON has been investigated in humans using mostly electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), but shared neural correlates of action observation and action execution are still unclear due to lack of ecologically valid neuroimaging measures. In this study, we used concurrent EEG and functional Near Infrared Spectroscopy (fNIRS) to examine the AON during a live-action observation and execution paradigm. We developed structured sparse multiset canonical correlation analysis (ssmCCA) to perform EEG-fNIRS data fusion. MCCA is a generalization of CCA to more than two sets of variables and is commonly used in medical multimodal data fusion. However, mCCA suffers from multi-collinearity, high dimensionality, unimodal feature selection, and loss of spatial information in interpreting the results. A limited number of participants (small sample size) is another problem in mCCA, which leads to overfitted models. Here, we adopted graph-guided (structured) fused least absolute shrinkage and selection operator (LASSO) penalty to mCCA to conduct feature selection, incorporating structural information amongst the variables (i.e., brain regions). Benefitting from concurrent recordings of brain hemodynamic and electrophysiological responses, the proposed ssmCCA finds linear transforms of each modality such that the correlation between their projections is maximized. Our analysis of 21 right-handed participants indicated that the left inferior parietal region was active during both action execution and action observation. Our findings provide new insights into the neural correlates of AON which are more fine-tuned than the results from each individual EEG or fNIRS analysis and validate the use of ssmCCA to fuse EEG and fNIRS datasets.

Autonomic system tuning during gesture observation and reproduction

Gestural communication allows providing information about thoughts and feelings, characterizing face-to-face interactions, also during non-verbal exchanges. In the present study, the autonomic responses and peripheral synchronization mechanisms of two individuals (encoder and decoder) were recorded simultaneously, through the use of biofeedback in hyperscanning, during two different experimental phases consisting in the observation (watching videos of gestures) and reproduction of positive and negative different types of gestures (affective, social and informative) supported by linguistic contexts. Therefore, the main aim of this study was focused on the analysis of simultaneous individuals' peripheral mechanisms during the performing of complex joint action, consisting of the observation (watching videos) and the reproduction of positive and negative social, affective, and informative gestures each supported by a linguistic script. Single-subject and inter-subject correlation analyses were conducted to observe individuals' autonomic responses and physiological synchronization. Single-subject results revealed an increase in emotional arousal, indicated by an increase in electrodermal activity (skin conductance level - SCL and response - SCR), during both the observation (watching videos) and reproduction of negative social and affective gestures contextualized by a linguistic context. Moreover, an increase of emotional engagement, expressed by an increase in heart rate (HR) activity, emerged in the encoder compare to the decoder during gestures reproduction (simulation of gestures). Inter-subject correlation results showed the presence of mirroring mechanisms, indicated by an increase in SCL, SCR, and HR synchronization, during the linguistic contexts and gesture observation (watching videos). Furthermore, an increase in SCL and SCR synchronization emerged during the observation (watching videos) and reproduction of negative social and affective gestures. Therefore, the present study allowed to obtain information on the mirroring mechanisms and physiological synchronization underlying the linguistic and gesture system during non-verbal interaction.

An fNIRS Study of Brain Lateralization During Observation and Execution of a Fine Motor Task

Brain activity in the action observation network (AON) is lateralized during action execution, with greater activation in the contralateral hemisphere to the side of the body used to perform the task. However, it is unknown whether the AON is also lateralized when watching another person perform an action. In this study, we use fNIRS to measure brain activity over the left and right cortex while participants completed actions with their left and right hands and watched an actor complete action with their left and right hands. We show that while activation is lateralized when the participants themselves are moving, brain lateralization is not affected by the side of the body when the participant is observing another person's action. In addition, we demonstrate that individual differences in hand preference and dexterity between the right and left hands are related to brain lateralization patterns.

Cerebral hemodynamic response during a live action-observation and action-execution task: A fNIRS study

Although many studies have examined the location of the action observation network (AON) in human adults, the shared neural correlates of action-observation and action-execution are still unclear partially due to lack of ecologically valid neuroimaging measures. In this study, we aim to demonstrate the feasibility of using functional near infrared spectroscopy (fNIRS) to measure the neural correlates of action-observation and action execution regions during a live task. Thirty adults reached for objects or observed an experimenter reaching for objects while their cerebral hemodynamic responses including oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) were recorded in the sensorimotor and parietal regions. Our results indicated that the parietal regions, including bilateral superior parietal lobule (SPL), bilateral inferior parietal lobule (IPL), right supra-marginal region (SMG) and right angular gyrus (AG) share neural activity during action-observation and action-execution. Our findings confirm the applicability of fNIRS for the study of the AON and lay the foundation for future work with developmental and clinical populations.

Data Processing in Functional Near-Infrared Spectroscopy (fNIRS) Motor Control Research

FNIRS pre-processing and processing methodologies are very important-how a researcher chooses to process their data can change the outcome of an experiment. The purpose of this review is to provide a guide on fNIRS pre-processing and processing techniques pertinent to the field of human motor control research. One hundred and twenty-three articles were selected from the motor control field and were examined on the basis of their fNIRS pre-processing and processing methodologies. Information was gathered about the most frequently used techniques in the field, which included frequency cutoff filters, wavelet filters, smoothing filters, and the general linear model (GLM). We discuss the methodologies of and considerations for these frequently used techniques, as well as those for some alternative techniques. Additionally, general considerations for processing are discussed.

Using Functional Connectivity to Examine the Correlation between Mirror Neuron Network and Autistic Traits in a Typically Developing Sample: A fNIRS Study

Mirror neuron network (MNN) is associated with one's ability to recognize and interpret others' actions and emotions and has a crucial role in cognition, perception, and social interaction. MNN connectivity and its relation to social attributes, such as autistic traits have not been thoroughly examined. This study aimed to investigate functional connectivity in the MNN and assess relationship between MNN connectivity and subclinical autistic traits in neurotypical adults. Hemodynamic responses, including oxy- and deoxy-hemoglobin were measured in the central and parietal cortex of 30 healthy participants using a 24-channel functional Near-Infrared spectroscopy (fNIRS) system during a live action-observation and action-execution task. Functional connectivity was derived from oxy-hemoglobin data. Connections with significantly greater connectivity in both tasks were assigned to MNN connectivity. Correlation between connectivity and autistic traits were performed using Pearson correlation. Connections within the right precentral, right supramarginal, left inferior parietal, left postcentral, and between left supramarginal-left angular regions were identified as MNN connections. In addition, individuals with higher subclinical autistic traits present higher connectivity in both action-execution and action-observation conditions. Positive correlation between MNN connectivity and subclinical autistic traits can be used in future studies to investigate MNN in a developing population with autism spectrum disorder.

Characterizing the Action-Observation Network Through Functional Near-Infrared Spectroscopy: A Review

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique that has undergone tremendous growth over the last decade due to methodological advantages over other measures of brain activation. The action-observation network (AON), a system of brain structures proposed to have “mirroring” abilities (e.g., active when an individual completes an action or when they observe another complete that action), has been studied in humans through neural measures such as fMRI and electroencephalogram (EEG); however, limitations of these methods are problematic for AON paradigms. For this reason, fNIRS is proposed as a solution to investigating the AON in humans. The present review article briefly summarizes previous neural findings in the AON and examines the state of AON research using fNIRS in adults. A total of 14 fNIRS articles are discussed, paying particular attention to methodological choices and considerations while summarizing the general findings to aid in developing better protocols to study the AON through fNIRS. Additionally, future directions of this work are discussed, specifically in relation to researching AON development and potential multimodal imaging applications.

The effect of interbrain synchronization in gesture observation: A fNIRS study

INTRODUCTION

Gestures characterize individuals' nonverbal communicative exchanges, taking on different functions. Several types of research in the neuroscientific field have been interested in the investigation of the neural correlates underlying the observation and implementation of different gestures categories. In particular, different studies have focused on the neural correlates underlying gestures observation, emphasizing the presence of mirroring mechanisms in specific brain areas, which appear to be involved in gesture observation and planning mechanisms.

MATERIALS AND METHODS

Specifically, the present study aimed to investigate the neural mechanisms, through the use of functional Near-Infrared Spectroscopy (fNIRS), underlying the observation of affective, social, and informative gestures with positive and negative valence in individuals' dyads composed by encoder and decoder. The variations of oxygenated (O2Hb) and deoxygenated (HHb) hemoglobin concentrations of both individuals were collected simultaneously through the use of hyperscanning paradigm, allowing the recording of brain responsiveness and interbrain connectivity.

RESULTS

The results showed a different brain activation and an increase of interbrain connectivity according to the type of gestures observed, with a significant increase of O2Hb brain responsiveness and interbrain connectivity and a decrease of HHb brain responsiveness for affective gestures in the dorsolateral prefrontal cortex (DLPFC) and for social gestures in the superior frontal gyrus (SFG). Furthermore, concerning the valence of the observed gestures, an increase of O2Hb brain activity and interbrain connectivity was observed in the left DLPFC for positive affective gestures compared to negative ones.

CONCLUSION

In conclusion, the present study showed different brain responses underlying the observation of different types of positive and negative gestures. Moreover, interbrain connectivity calculation allowed us to underline the presence of mirroring mechanisms involved in gesture-specific frontal regions during gestures observation and action planning.

Affective, Social, and Informative Gestures Reproduction in Human Interaction: Hyperscanning and Brain Connectivity

Gestural communication characterizes daily individuals' interactions in order to share information and to modify others' behavior. Social neuroscience has investigated the neural bases which support recognizing of different gestures. The present research, through the use of the hyperscanning approach, that allows the simultaneously recording of the activity of two or more individuals involved in a joint action, aims to investigate the neural bases of gestural communication. Moreover, by using hyperscanning paradigm we explore the inter-brain connectivity between two inter-agents, the one who performed the gesture (encoder) and the one who received it (decoder), with functional Near-infrared Spectroscopy (fNIRS) during the reproduction of affective, social and informative gestures with positive and negative valence. Result showed an increase in oxygenated hemoglobin concentration (O2Hb) and inter-brain connectivity in the dorsolateral prefrontal cortex (DLPFC) for affective gestures, in the superior frontal gyrus (SFG) for social gestures and the frontal eye fields (FEF) for informative gestures, for both encoder and decoder. Furthermore, it emerged that positive gestures activate more the left DLPFC, with an increase in inter-brain connectivity in DLPFC and SFG. The present study revealed the relevant function of the type and valence of gestures in affecting intra- and inter-brain connectivity.

Flexibility in Language Action Interaction: The Influence of Movement Type

Recent neuropsychological studies in neurological patients and healthy subjects suggest a close functional relationship between the brain systems for language and action. Facilitation and inhibition effects of motor system activity on language processing have been demonstrated as well as causal effects in the reverse direction, from language processes on motor excitability or performance. However, as the documented effects between motor and language systems were sometimes facilitatory and sometimes inhibitory, the “sign” of these effects still remains to be explained. In a previous study, we reported a word-category-specific differential impairment of verbal working memory for concordant arm- and leg-related action words brought about by complex sequential movements of the hands and feet. In this article, we seek to determine whether the sign of the functional interaction between language and action systems of the human brain can be changed in a predictable manner by changing movement type. We here report that the sign of the effect of motor movement on action word memory can be reversed from interference to facilitation if, instead of complex movement sequences, simple repetitive movements are performed. Specifically, when engaged in finger tapping, subjects were able to remember relatively more arm-related action words (as compared to control conditions), thus documenting an enhancement of working memory brought about by simple hand movements. In contrast, when performing complex sequences of finger movements, an effector-specific degradation of action word memory was found. By manipulating the sign of the effect in accord with theory-driven predictions, these findings provide support for shared neural bases for motor movement and verbal working memory for action-related words and strengthen the argument that motor systems play a causal and functionally relevant role in language processing semantically related to action.