Cortical activity during the recognition of cooperative actions

Movies and narratives as naturalistic stimuli in neuroimaging

Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.

Cortical Activation during Action Observation, Action Execution, and Interpersonal Synchrony in Adults: A functional Near-Infrared Spectroscopy (fNIRS) Study

Introduction: Humans engage in Interpersonal Synchrony (IPS) as they synchronize their own actions with that of a social partner over time. When humans engage in imitation/IPS behaviors, multiple regions in the frontal, temporal, and parietal cortices are activated including the putative Mirror Neuron Systems (Iacoboni, 2005; Buxbaum et al., 2014). In the present study, we compared fNIRS-based cortical activation patterns across three conditions of action observation (“Watch” partner), action execution (“Do” on your own), and IPS (move “Together”).

Methods: Fifteen typically developing adults completed a reach and cleanup task with the right arm while cortical activation was examined using a 24-channel, Hitachi fNIRS system. Each adult completed 8 trials across three conditions (Watch, Do, and Together). For each fNIRS channel, we obtained oxy hemoglobin (HbO₂) and deoxy hemoglobin (HHb) profiles. Spatial registration methods were applied to localize the cortical regions underneath each channel and to define six regions of interest (ROIs), right and left supero-anterior (SA or pre/post-central gyri), infero-posterior (IP or angular/supramarginal gyri), and infero-anterior (IA or superior/middle temporal gyri) regions.

Results: In terms of task-related differences, the majority of the ROIs were more active during Do and Together compared to Watch. Only the right/ipsilateral fronto-parietal and inferior parietal cortices had greater activation during Together compared to Do.

Conclusions: The similarities in cortical activation between action execution and IPS suggest that neural control of IPS is more similar to its execution than observational aspects. To be clear, the more complex the actions performed, the more difficult the IPS behaviors. Secondly, IPS behaviors required slightly more right-sided activation (vs. execution/observation) suggesting that IPS is a higher-order process involving more bilateral activation compared to its sub-components. These findings provide a neuroimaging framework to study imitation and IPS impairments in special populations such as infants at risk for and children with ASD.

Near-infrared spectroscopy: a report from the McDonnell infant methodology consortium

Near-infrared spectroscopy (NIRS) is a new and increasingly widespread brain imaging technique, particularly suitable for young infants. The laboratories of the McDonnell Consortium have contributed to the technological development and research applications of this technique for nearly a decade. The present paper provides a general introduction to the technique as well as a detailed report of the methodological innovations developed by the Consortium. The basic principles of NIRS and some of the existing developmental studies are reviewed. Issues concerning technological improvements, parameter optimization, possible experimental designs and data analysis techniques are discussed and illustrated by novel empirical data.

Testing the potential of a virtual reality neurorehabilitation system during performance of observation, imagery and imitation of motor actions recorded by wireless functional near-infrared spectroscopy (fNIRS)

BACKGROUND

Several neurorehabilitation strategies have been introduced over the last decade based on the so-called simulation hypothesis. This hypothesis states that a neural network located in primary and secondary motor areas is activated not only during overt motor execution, but also during observation or imagery of the same motor action. Based on this hypothesis, we investigated the combination of a virtual reality (VR) based neurorehabilitation system together with a wireless functional near infrared spectroscopy (fNIRS) instrument. This combination is particularly appealing from a rehabilitation perspective as it may allow minimally constrained monitoring during neurorehabilitative training.

METHODS

fNIRS was applied over F3 of healthy subjects during task performance in a virtual reality (VR) environment: 1) 'unilateral' group (N = 15), contralateral recording during observation, motor imagery, observation & motor imagery, and imitation of a grasping task performed by a virtual limb (first-person perspective view) using the right hand; 2) 'bilateral' group (N = 8), bilateral recording during observation and imitation of the same task using the right and left hand alternately.

RESULTS

In the unilateral group, significant within-condition oxy-hemoglobin concentration Δ[O2Hb] changes (mean ± SD μmol/l) were found for motor imagery (0.0868 ± 0.5201 μmol/l) and imitation (0.1715 ± 0.4567 μmol/l). In addition, the bilateral group showed a significant within-condition Δ[O2Hb] change for observation (0.0924 ± 0.3369 μmol/l) as well as between-conditions with lower Δ[O2Hb] amplitudes during observation compared to imitation, especially in the ipsilateral hemisphere (p < 0.001). Further, in the bilateral group, imitation using the non-dominant (left) hand resulted in larger Δ[O2Hb] changes in both the ipsi- and contralateral hemispheres as compared to using the dominant (right) hand.

CONCLUSIONS

This study shows that our combined VR-fNIRS based neurorehabilitation system can activate the action-observation system as described by the simulation hypothesis during performance of observation, motor imagery and imitation of hand actions elicited by a VR environment. Further, in accordance with previous studies, the findings of this study revealed that both inter-subject variability and handedness need to be taken into account when recording in untrained subjects. These findings are of relevance for demonstrating the potential of the VR-fNIRS instrument in neurofeedback applications.

Hemodynamic change in occipital lobe during visual search: visual attention allocation measured with NIRS

We examined the changes in regional cerebral blood volume (rCBV) around visual cortex using Near Infrared Spectroscopy (NIRS) when observers attended to visual scenes. The oxygenated and deoxygenated hemoglobin (Oxy-Hb and Deoxy-Hb) concentration changes at occipital lobe were monitored during a dual task. Observers were asked to name a digit superimposed on a scenery picture, while in parallel, they had to detect an on-and-off flickering object in a Change Blindness paradigm. Results showed the typical activation patterns in and around the visual cortex with increases in Oxy-Hb and decreases in Deoxy-Hb. The Oxy-Hb increase doubled when observers could not find the target, as opposed to trials in which they could. The results strongly suggest that active attention to a visual scene enhances Oxy-Hb change much stronger than passive watching, and that attention and Oxy-Hb increases are possibly correlated.

Observing human interaction with physical devices

Previous study has shown that if we observe another person operating a tool or physical device, then the action rule of the observed action is automatically activated and can subsequently facilitate own actions. In this study, the mechanisms responsible for this automatic priming of actions are investigated. In two experiments, the question is raised whether priming arises from the observation of the physical device and its movements, or whether it is modulated by aspects of the person’s behaviour. Whereas experiment 1 shows that priming effects are not influenced by the effector used by the observed person, experiment 2 demonstrates that they are modulated by the handle (and associated action rule) that is used to operate the device. These results suggest that motor resonance mechanisms are sensitive to the specific interaction between movements of an actor and associated movements of a physical device.

Understanding the flexibility of action-perception coupling

The idea that observing an action triggers an automatic and obligatory activation of an imitative action in the motor system of the observer has recently been questioned by studies examining complementary actions. Instead of a tendency for imitation, cooperative settings may facilitate the execution of dissimilar actions, resulting in a relative disadvantage for imitative actions. The present study aimed at clarifying the contribution of associative learning and interference of task representations to the reversal of congruency effects. To distinguish between the two, an experiment was designed, in which we increased the effects of associative learning and minimized the effects of task interference. Participants completed a series of imitation and complementary action runs, in which they continuously imitated or complemented the actions of a virtual co-actor. Each run was alternated with a test run showing the same actions but including color-cues, and the participants were instructed to respond to color instead of the actor's posture. Reaction times to test runs showed no reversal of facilitation effects between the imitation and complementary action conditions. This result strongly argues that associative learning cannot adequately account for reversed facilitation effects. Our study provides additional support for action-perception models that allow flexible selection of action-perception coupling and challenges the existing models purely based on stimulus-response associations.

Event-related potentials during the evaluation of the appropriateness of cooperative actions

Event-related potentials (ERPs) were recorded during the evaluation of the appropriateness of cooperative actions between two people. We used pictures of pass-and-receive actions as one type of cooperative actions, in which one person passed an object and another reached out to receive it with a preshaped hand. Eleven participants judged whether the receiving actions were appropriate or inappropriate in relation to the passing ones. The inappropriate actions elicited a widely distributed and parietal maximum N400 as compared with the appropriate ones. These results suggest that the N400 is evoked in the incongruous context in which two people perform inappropriate cooperative actions and is related to the semantic processing that involves the prediction of interpersonal action sequences.