Do we mind other minds when we mind other minds' actions? A functional magnetic resonance imaging study

Updating predictions in a complex repertoire of actions and its neural representation

Even though actions we observe in everyday life seem to unfold in a continuous manner, they are automatically divided into meaningful chunks, that are single actions or segments, which provide information for the formation and updating of internal predictive models. Specifically, boundaries between actions constitute a hub for predictive processing since the prediction of the current action comes to an end and calls for updating of predictions for the next action. In the current study, we investigated neural processes which characterize such boundaries using a repertoire of complex action sequences with a predefined probabilistic structure. Action sequences consisted of actions that started with the hand touching an object (T) and ended with the hand releasing the object (U). These action boundaries were determined using an automatic computer vision algorithm. Participants trained all action sequences by imitating demo videos. Subsequently, they returned for an fMRI session during which the original action sequences were presented in addition to slightly modified versions thereof. Participants completed a post-fMRI memory test to assess the retention of original action sequences. The exchange of individual actions, and thus a violation of action prediction, resulted in increased activation of the action observation network and the anterior insula. At U events, marking the end of an action, increased brain activation in supplementary motor area, striatum, and lingual gyrus was indicative of the retrieval of the previously encoded action repertoire. As expected, brain activation at U events also reflected the predefined probabilistic branching structure of the action repertoire. At T events, marking the beginning of the next action, midline and hippocampal regions were recruited, reflecting the selected prediction of the unfolding action segment. In conclusion, our findings contribute to a better understanding of the various cerebral processes characterizing prediction during the observation of complex action repertoires.

Functional role of the theory of mind network in integrating mentalistic prior information with action kinematics during action observation

Inferring intentions from verbal and nonverbal human behaviour is critical for everyday social life. Here, we combined Transcranial Magnetic Stimulation (TMS) with a behavioural priming paradigm to test whether key nodes of the Theory of Mind network (ToMn) contribute to understanding others' intentions by integrating prior knowledge about an agent with the observed action kinematics. We used a modified version of the Faked-Action Discrimination Task (FAD), a forced-choice paradigm in which participants watch videos of actors lifting a cube and judge whether the actors are trying to deceive them concerning the weight of the cube. Videos could be preceded or not by verbal description (prior) about the agent's truthful or deceitful intent. We applied single pulse TMS over three key nodes of the ToMn, namely dorsomedial prefrontal cortex (dmPFC), right posterior superior temporal sulcus (pSTS) and right temporo-parietal junction (rTPJ). Sham-TMS served as a control (baseline) condition. Following sham or rTPJ stimulation, we observed no consistent influence of priors on FAD performance. In contrast, following dmPFC stimulation, and to a lesser extent pSTS stimulation, truthful and deceitful actions were perceived as more deceptive only when the prior suggested a dishonest intention. These findings highlight a functional role of dmPFC and pSTS in coupling prior knowledge about deceptive intents with observed action kinematics in order to judge faked actions. Our study provides causal evidence that fronto-temporal nodes of the ToMn are functionally relevant to mental state inference during action observation.

Unimpaired social cognition in adult patients with ADHD: brain volumetric and behavioral results

The present study aimed to investigate whether adult patients with attention deficit hyperactivity disorder (ADHD) show deficits in social cognition and to identify the structural neural correlates of social cognitive skills in ADHD. Twenty-six adult patients with ADHD and 26 matched healthy control participants performed the Movie for the Assessment of Social Cognition and underwent a structural magnetic resonance imaging scan. We compared theory of mind (ToM) performance between ADHD patients and healthy controls. Using voxel-based morphometry, we further compared gray matter volumes in regions that are critical for social cognition between the two groups and examined whether ToM performance was correlated with brain morphometry measures. We did not observe any between-group differences in ToM abilities or regional gray matter volumes. Across both groups, performance on affective aspects of ToM correlated positively with gray matter volumes in the medial part of the superior frontal gyri, which is typically involved in social cognition. This study is the first to relate brain structure to social cognitive abilities in adult patients with ADHD. Although our sample was small and heterogeneous, with half of the patients showing mild-to-moderate psychiatric comorbidities, our results may encourage longitudinal studies that relate social cognitive development in childhood and adolescence to brain maturation of ADHD patients.

Representational similarity analysis reveals atypical age-related changes in brain regions supporting face and car recognition in autism

BACKGROUND

Autism Spectrum Disorder (ASD) is associated with atypical activation in the ventral stream during face processing. The current study further characterizes the development of face processing in ASD using a multivoxel pattern analysis, which assesses the similarity in the representation of exemplars from the same category.

METHODS

Ninety-two children, adolescents and adults - with and without ASD - performed the Cambridge Face Memory Test, the Australian Face Memory Test, and a matched car memory test. Regions of interest during these tasks included Fusiform Face Area (FFA), based on the literature, and additional, structurally-defined regions in the ventral stream. Group differences in the patterns of activity within these ROIs when memorizing exemplars were examined using a representational similarity analysis (RSA).

RESULTS

The RSA revealed significant interactions between age group and diagnostic group in R FFA, with increasing similarity within a category (faces, cars) into adulthood typically but not in those with ASD. This pattern was also evident in structurally defined ventral stream regions, namely L inferior frontal gyrus (IFG), bilateral temporoparietal junction (TPJ), L inferior temporal lobule, and the R fusiform gyrus.

CONCLUSIONS

The specialization of face and object processing from adolescence to adulthood evident in typical development may be impaired in ASD, undermining the ability to reach adult-level visual processing in those with ASD.

Effects of pseudoexperience on the understanding of hemiplegic movements in physical therapists: An fMRI study

Physical therapists (PTs) are required to obtain an accurate understanding of the physical and mental states of their patients through observational assessment. To perform comprehensive observational assessments of patients' movements, PTs likely need to engage their own neural systems involved in action understanding and theory of mind, such as the action observation network (AON) and the right temporoparietal junction (rTPJ). Both systems are modulated by the observer's actual experience with the observed movements. Although, most PTs do not have physical experience with neurological disabilities, they routinely examine hemiplegic movements in stroke patients, and are thus considered to have acquired pseudoexperience with hemiplegia. We hypothesized that the PTs' pseudoexperience with hemiplegia would modulate the neural system associated with the understanding of others to elaborately comprehend the physical and mental states associated with hemiplegia. To investigate our hypothesis, we recruited 19 PTs and 19 naïve participants (NPs) to undergo functional MRI (fMRI) for cortical activity measurement while viewing videos of hemiplegic (HHM) and non-hemiplegic (non-HHM) hand movements. The participants subsequently viewed the same videos again outside the MRI scanner, and evaluated the observed hand movements via a questionnaire. Compared to the NPs, the PTs showed greater activation in the AON and rTPJ while observing HHMs. Psychophysiological interaction analyses revealed increased connectivity between the rTPJ and AON when the PTs viewed the HHMs. Behavioral analyses further indicated that the PTs more accurately assessed feeling states associated with HHMs than did NPs. These findings suggest that the PTs' pseudoexperience modulates the AON and rTPJ, enabling them to better understand hemiplegia-associated feeling states.

Making sense of objects lying around: How contextual objects shape brain activity during action observation

Action recognition involves not only the readout of body movements and involved objects but also the integration of contextual information, e.g. the environment in which an action takes place. Notably, inferring superordinate goals and generating predictions about forthcoming action steps should benefit from screening the actor's immediate environment, in particular objects located in the actor's peripersonal space and thus potentially used in following action steps. Critically, if such contextual objects (COs) afford actions that are semantically related to the observed action, they may trigger or facilitate the inference of goals and the prediction of following actions. This fMRI study investigated the neural mechanisms underlying the integration of COs in semantic and spatial relation to observed actions. Specifically, we tested the hypothesis that the inferior frontal gyrus (IFG) subserves this integration. Participants observed action videos in which COs and observed actions had common overarching goals or not (goal affinity) and varied in their location relative to the actor. High goal affinity increased bilateral activity in action observation network nodes, i.e. the occipitotemporal cortex and the intraparietal sulcus, but also in the precuneus and middle frontal gyri. This finding suggests that the semantic relation between COs and actions is considered during action observation and triggers (rather than facilitates) processes beyond those usually involved in action observation. Moreover, COs with high goal affinity located close to the actor's dominant hand additionally engaged bilateral IFG, corroborating the view that IFG is critically involved in the integration of action steps under a common overarching goal.

Action Categories in Lateral Occipitotemporal Cortex Are Organized Along Sociality and Transitivity

How neural specificity for distinct conceptual knowledge categories arises is central for understanding the organization of semantic memory in the human brain. Although there is a large body of research on the neural processing of distinct object categories, the organization of action categories remains largely unknown. In particular, it is unknown whether different action categories follow a specific topographical organization on the cortical surface analogously to the category-specific organization of object knowledge. Here, we tested whether the neural representation of action knowledge is organized in terms of nonsocial versus social and object-unrelated versus object-related actions (sociality and transitivity, respectively, hereafter). We hypothesized a major distinction of sociality and transitivity along dorsal and ventral lateral occipitotemporal cortex (LOTC), respectively. Using fMRI-based multivoxel pattern analysis, we identified neural representations of action information associated with sociality and transitivity in bilateral LOTC. Representational similarity analysis revealed a dissociation between dorsal and ventral LOTC. We found that action representations in dorsal LOTC are segregated along features of sociality, whereas action representations in ventral LOTC are segregated along features of transitivity. In addition, representations of sociality and transitivity features were found more anteriorly in LOTC than representations of specific subtypes of actions, suggesting a posterior-anterior gradient from concrete to abstract action features. These findings elucidate how the neural representations of perceptually and conceptually diverse actions are organized in distinct subsystems in the LOTC.

SIGNIFICANCE STATEMENT

The lateral occipitotemporal cortex (LOTC) is critically involved in the recognition of objects and actions, but our knowledge about the underlying organizing principles is limited. Here, we discovered a dorsal-ventral distinction of actions in LOTC: dorsal LOTC represents actions based on sociality (how much an action is directed to another person) in proximity to person knowledge. In contrast, ventral LOTC represents actions based on transitivity (how much an action involves the interaction with inanimate objects) in proximity to tools/artifacts in ventral LOTC, suggesting a mutually dependent organization of actions and objects. In addition, we found a posterior-to-anterior organization of the LOTC for concrete and abstract representations, respectively. Our findings provide important insights about the organization of actions in LOTC.

Action Categories in Lateral Occipitotemporal Cortex Are Organized Along Sociality and Transitivity

How neural specificity for distinct conceptual knowledge categories arises is central for understanding the organization of semantic memory in the human brain. Although there is a large body of research on the neural processing of distinct object categories, the organization of action categories remains largely unknown. In particular, it is unknown whether different action categories follow a specific topographical organization on the cortical surface analogously to the category-specific organization of object knowledge. Here, we tested whether the neural representation of action knowledge is organized in terms of nonsocial versus social and object-unrelated versus object-related actions (sociality and transitivity, respectively, hereafter). We hypothesized a major distinction of sociality and transitivity along dorsal and ventral lateral occipitotemporal cortex (LOTC), respectively. Using fMRI-based multivoxel pattern analysis, we identified neural representations of action information associated with sociality and transitivity in bilateral LOTC. Representational similarity analysis revealed a dissociation between dorsal and ventral LOTC. We found that action representations in dorsal LOTC are segregated along features of sociality, whereas action representations in ventral LOTC are segregated along features of transitivity. In addition, representations of sociality and transitivity features were found more anteriorly in LOTC than representations of specific subtypes of actions, suggesting a posterior-anterior gradient from concrete to abstract action features. These findings elucidate how the neural representations of perceptually and conceptually diverse actions are organized in distinct subsystems in the LOTC.

SIGNIFICANCE STATEMENT

The lateral occipitotemporal cortex (LOTC) is critically involved in the recognition of objects and actions, but our knowledge about the underlying organizing principles is limited. Here, we discovered a dorsal-ventral distinction of actions in LOTC: dorsal LOTC represents actions based on sociality (how much an action is directed to another person) in proximity to person knowledge. In contrast, ventral LOTC represents actions based on transitivity (how much an action involves the interaction with inanimate objects) in proximity to tools/artifacts in ventral LOTC, suggesting a mutually dependent organization of actions and objects. In addition, we found a posterior-to-anterior organization of the LOTC for concrete and abstract representations, respectively. Our findings provide important insights about the organization of actions in LOTC.

Differentiating between self and others: an ALE meta-analysis of fMRI studies of self-recognition and theory of mind

The perception of self and others is a key aspect of social cognition. In order to investigate the neurobiological basis of this distinction we reviewed two classes of task that study self-awareness and awareness of others (theory of mind, ToM). A reliable task to measure self-awareness is the recognition of one's own face in contrast to the recognition of others' faces. False-belief tasks are widely used to identify neural correlates of ToM as a measure of awareness of others. We performed an activation likelihood estimation meta-analysis, using the fMRI literature on self-face recognition and false-belief tasks. The brain areas involved in performing false-belief tasks were the medial prefrontal cortex (MPFC), bilateral temporo-parietal junction, precuneus, and the bilateral middle temporal gyrus. Distinct self-face recognition regions were the right superior temporal gyrus, the right parahippocampal gyrus, the right inferior frontal gyrus/anterior cingulate cortex, and the left inferior parietal lobe. Overlapping brain areas were the superior temporal gyrus, and the more ventral parts of the MPFC. We confirmed that self-recognition in contrast to recognition of others' faces, and awareness of others involves a network that consists of separate, distinct neural pathways, but also includes overlapping regions of higher order prefrontal cortex where these processes may be combined. Insights derived from the neurobiology of disorders such as autism and schizophrenia are consistent with this notion.

Action observers implicitly expect actors to act goal-coherently, even if they do not: an fMRI study

Actions observed in everyday life normally consist of one person performing sequences of goal-directed actions. The present fMRI study tested the hypotheses that observers are influenced by the actor's identity, even when this information is task-irrelevant, and that this information shapes their expectation on subsequent actions of the same actor. Participants watched short video clips of action steps that either pertained to a common action with an overarching goal or not, and were performed by either one or by varying actors (2 × 2 design). Independent of goal coherence, actor coherence elicited activation in dorsolateral and ventromedial frontal cortex, together pointing to a spontaneous attempt to integrate all actions performed by one actor. Interestingly, watching an actor performing unrelated actions elicited additional activation in left inferior frontal gyrus, suggesting a search in semantic memory in an attempt to construct an overarching goal that can reconcile the disparate action steps with a coherent intention. Post-experimental surveys indicate that these processes occur mostly unconsciously. Findings strongly suggest a spontaneous expectation bias toward actor-related episodes in action observers, and hence to the immense impact of actor information on action observation.

Explicit authenticity and stimulus features interact to modulate BOLD response induced by emotional speech

Context has been found to have a profound effect on the recognition of social stimuli and correlated brain activation. The present study was designed to determine whether knowledge about emotional authenticity influences emotion recognition expressed through speech intonation. Participants classified emotionally expressive speech in an fMRI experimental design as sad, happy, angry, or fearful. For some trials, stimuli were cued as either authentic or play-acted in order to manipulate participant top-down belief about authenticity, and these labels were presented both congruently and incongruently to the emotional authenticity of the stimulus. Contrasting authentic versus play-acted stimuli during uncued trials indicated that play-acted stimuli spontaneously up-regulate activity in the auditory cortex and regions associated with emotional speech processing. In addition, a clear interaction effect of cue and stimulus authenticity showed up-regulation in the posterior superior temporal sulcus and the anterior cingulate cortex, indicating that cueing had an impact on the perception of authenticity. In particular, when a cue indicating an authentic stimulus was followed by a play-acted stimulus, additional activation occurred in the temporoparietal junction, probably pointing to increased load on perspective taking in such trials. While actual authenticity has a significant impact on brain activation, individual belief about stimulus authenticity can additionally modulate the brain response to differences in emotionally expressive speech.

Interaction vs. observation: distinctive modes of social cognition in human brain and behavior? A combined fMRI and eye-tracking study

Human cognition has usually been approached on the level of individual minds and brains, but social interaction is a challenging case. Is it best thought of as a self-contained individual cognitive process aiming at an "understanding of the other," or should it rather be approached as an collective, inter-personal process where individual cognitive components interact on a moment-to-moment basis to form coupled dynamics? In a combined fMRI and eye-tracking study we directly contrasted these models of social cognition. We found that the perception of situations affording social contingent responsiveness (e.g., someone offering or showing you an object) elicited activations in regions of the right posterior temporal sulcus and yielded greater pupil dilation corresponding to a model of coupled dynamics (joint action). In contrast, the social-cognitive perception of someone "privately" manipulating an object elicited activation in medial prefrontal cortex, the right inferior frontal gyrus and right inferior parietal lobus, regions normally associated with Theory of Mind and with the mirror neuron system. Our findings support a distinction in social cognition between social observation and social interaction, and demonstrate that simple ostensive cues may shift participants' experience, behavior, and brain activity between these modes. The identification of a distinct, interactive mode has implications for research on social cognition, both in everyday life and in clinical conditions.

Authenticity affects the recognition of emotions in speech: behavioral and fMRI evidence

The aim of the present study was to determine how authenticity of emotion expression in speech modulates activity in the neuronal substrates involved in emotion recognition. Within an fMRI paradigm, participants judged either the authenticity (authentic or play acted) or emotional content (anger, fear, joy, or sadness) of recordings of spontaneous emotions and reenactments by professional actors. When contrasting between task types, active judgment of authenticity, more than active judgment of emotion, indicated potential involvement of the theory of mind (ToM) network (medial prefrontal cortex, temporoparietal cortex, retrosplenium) as well as areas involved in working memory and decision making (BA 47). Subsequently, trials with authentic recordings were contrasted with those of reenactments to determine the modulatory effects of authenticity. Authentic recordings were found to enhance activity in part of the ToM network (medial prefrontal cortex). This effect of authenticity suggests that individuals integrate recollections of their own experiences more for judgments involving authentic stimuli than for those involving play-acted stimuli. The behavioral and functional results show that authenticity of emotional prosody is an important property influencing human responses to such stimuli, with implications for studies using play-acted emotions.