Endogenous variation in ventromedial prefrontal cortex state dynamics during naturalistic viewing reflects affective experience

Stimulus Selection Influences Prediction of Individual Phenotypes in Naturalistic Conditions

While the use of naturalistic stimuli such as movie clips for understanding individual differences and brain-behaviour relationships attracts increasing interest, the influence of stimulus selection remains largely unclear. By using machine learning to predict individual traits (phenotypes) from brain activity evoked during various movie clips, we show that different movie stimuli can result in distinct prediction performances. In brain regions related to lower-level processing of the stimulus, prediction to a certain degree benefits from stronger synchronisation of brain activity across subjects. By contrast, better predictions in frontoparietal brain regions are mainly associated with larger inter-subject variability. Furthermore, we demonstrate that while movie clips with rich social content in general achieve better predictions, the importance of specific movie features for prediction highly depends on the phenotype under investigation. Overall, our findings underscore the importance of careful stimulus selection and provide novel insights into stimulus selection for phenotype prediction in naturalistic conditions, opening new avenues for future research.

Individual differences in wellbeing are supported by separable sets of co-active self- and visual-attention-related brain networks

How does the brain support 'wellbeing'? Because it is a multidimensional construct, it is likely the product of multiple co-active brain networks that vary across individuals. This is perhaps why prior neuroimaging studies have found inconsistent anatomical associations with wellbeing. Furthermore, these used 'laboratory-style' or 'resting-state' methods not amenable to finding manifold networks. To address these issues, we had participants watch a full-length romantic comedy-drama film during functional magnetic resonance imaging. We hypothesised that individual differences in wellbeing measured before scanning would be correlated with individual differences in brain networks associated with 'embodied' and 'narrative' self-related processing. Indeed, searchlight spatial inter-participant representational similarity and subsequent analyses revealed seven sets of co-activated networks associated with individual differences in wellbeing. Two were 'embodied self' related, including brain regions associated with autonomic and affective processing. Three sets were 'narrative self' related, involving speech, language, and autobiographical memory-related regions. Finally, two sets of visual-attention-related networks emerged. These results suggest that the neurobiology of wellbeing in the real world is supported by diverse but functionally definable and separable sets of networks. This has implications for psychotherapy where individualised interventions might target, e.g., neuroplasticity in language-related narrative over embodied self or visual-attentional related processes.

A core set of neural states underlying memory reactivation of naturalistic events in the posterior medial cortex

In the posterior midline default mode network, spatial activity patterns similar to those during the initial experience are reactivated during the successful recall of past events. Prior studies have shown that these event-specific activity patterns are consistent across individuals recalling a shared experience, suggesting that common functional responses underlying episodic recall do exist. However, the spatial organization of function during episodic encoding and subsequent recall, especially in the absence of external stimuli, remains poorly understood. To address this, we leverage fMRI data collected during the encoding and recall of naturalistic movies to identify a core set of neural states in the posterior medial cortex. These states are stimulus-locked, reactivated during recall, and have a shared spatial organization across brains (i.e., individuals). We show that a surprisingly small number of these states (16 states across hemispheres) is sufficient to achieve the same levels of reactivation in the posterior medial cortex as when using the standard methods of the field. Additionally, these states are linked to actions and social-affective features of events in the movies. Our findings elucidate the properties of a common, spatially organized code within the posterior default mode network which appears during natural recollection of memories.

Network state dynamics underpin basal craving in a transdiagnostic population

Emerging fMRI methods quantifying brain dynamics present an opportunity to capture how fluctuations in brain responses give rise to individual variations in affective and motivation states. Although the experience and regulation of affective states affect psychopathology, their underlying time-varying brain responses remain unclear. Here, we present a novel framework to identify network states matched to an affective experience and examine how the dynamic engagement of these network states contributes to this experience. We apply this framework to investigate network state dynamics underlying basal craving, an affective experience with important clinical implications. In a transdiagnostic sample of healthy controls and individuals diagnosed with or at risk for craving-related disorders (total N = 252), we utilized connectome-based predictive modeling (CPM) to identify brain networks predictive of basal craving. An edge-centric timeseries approach was leveraged to quantify the moment-to-moment engagement of the craving-positive and craving-negative subnetworks during independent scan runs. We found that dynamic markers of network engagement, namely more persistence in a craving-positive network state and less dwelling in a craving-negative network state, characterized individuals with higher craving. We replicated the latter results in a separate dataset, incorporating distinct participants (N = 173) and experimental stimuli. The associations between basal craving and network state dynamics were consistently observed even when craving-predictive networks were defined in the replication dataset. These robust findings suggest that network state dynamics underpin individual differences in basal craving. Our framework additionally presents a new avenue to explore how the moment-to-moment engagement of behaviorally meaningful network states supports our affective experiences.

Mapping patterns of thought onto brain activity during movie-watching

Movie-watching is a central aspect of our lives and an important paradigm for understanding the brain mechanisms behind cognition as it occurs in daily life. Contemporary views of ongoing thought argue that the ability to make sense of events in the 'here and now' depend on the neural processing of incoming sensory information by auditory and visual cortex, which are kept in check by systems in association cortex. However, we currently lack an understanding of how patterns of ongoing thoughts map onto the different brain systems when we watch a film, partly because methods of sampling experience disrupt the dynamics of brain activity and the experience of movie-watching. Our study established a novel method for mapping thought patterns onto the brain activity that occurs at different moments of a film, which does not disrupt the time course of brain activity or the movie-watching experience. We found moments when experience sampling highlighted engagement with multi-sensory features of the film or highlighted thoughts with episodic features, regions of sensory cortex were more active and subsequent memory for events in the movie was better-on the other hand, periods of intrusive distraction emerged when activity in regions of association cortex within the frontoparietal system was reduced. These results highlight the critical role sensory systems play in the multi-modal experience of movie-watching and provide evidence for the role of association cortex in reducing distraction when we watch films.

Aberrant neural event segmentation during a continuous social narrative in trauma-exposed older adolescents and young adults

Post-traumatic stress and major depressive disorders are associated with "overgeneral" autobiographical memory, or impaired recall of specific life events. Interpersonal trauma exposure, a risk factor for both conditions, may influence how symptomatic trauma-exposed (TE) individuals segment everyday events. The ability to parse experience into units (event segmentation) supports memory. Neural state transitions occur within a cortical hierarchy and play a key role in event segmentation, with regions like the occipital cortex, angular gyrus, and striatum involved in parsing event structure. We examined whether interpersonal trauma exposure was associated with alterations in the cortical hierarchy and striatal activity at neural state transitions in symptomatic TE versus healthy control (HC) individuals. Fifty older adolescents and young adults (29 TE, 21 HC) viewed the film "Partly Cloudy" during functional magnetic resonance imaging. A greedy-state boundary search algorithm assessed the optimal number of events, quality, and segmentation agreement of neural state transitions in the occipital cortex and angular gyrus. Striatal (nucleus accumbens, caudate, and putamen) activity was assessed at occipital and angular gyrus-evoked state transitions. Compared to HCs, TE participants displayed less occipital and greater angular gyrus-evoked optimal number of neural state transitions. TE participants also displayed lower quality of neural state segmentation solutions in occipital and angular cortices compared to HCs. Additionally, TE participants had less putamen activity at angular gyrus-evoked state transitions than HCs. This investigation provides neurobiological insights into aberrant event segmentation in symptomatic TE individuals, shedding light on mechanisms influencing overgeneral memory in trauma-related disorders.

Brain network dynamics predict moments of surprise across contexts

We experience surprise when reality conflicts with our expectations. When we encounter such expectation violations in psychological tasks and daily life, are we experiencing completely different forms of surprise? Or is surprise a fundamental psychological process with shared neural bases across contexts? To address this question, we identified a brain network model, the surprise edge-fluctuation-based predictive model (EFPM), whose regional interaction dynamics measured with functional magnetic resonance imaging (fMRI) predicted surprise in an adaptive learning task. The same model generalized to predict surprise as a separate group of individuals watched suspenseful basketball games and as a third group watched videos violating psychological expectations. The surprise EFPM also uniquely predicts surprise, capturing expectation violations better than models built from other brain networks, fMRI measures and behavioural metrics. These results suggest that shared neurocognitive processes underlie surprise across contexts and that distinct experiences can be translated into the common space of brain dynamics.

Youth Generalized Anxiety and Brain Activation States During Socioemotional Processing

Importance

The brain enters distinct activation states to support differential cognitive and emotional processes, but little is known about how brain activation states differ in youths with clinical anxiety.

Objective

To characterize brain activation states during socioemotional processing (movie stimuli) and assess associations between state characteristics and movie features and anxiety symptoms.

Design, Setting, and Participants

The Healthy Brain Network is an ongoing cross-sectional study of individuals aged 5 to 21 years experiencing difficulties in school, of whom approximately 45% met criteria for a lifetime anxiety disorder diagnosis. Data used in this study are from the first 9 releases (collected in a nonclinical research setting in the New York City metropolitan area from 2015 to 2020) and include 620 youths aged 5 to 15 years (53% of whom met criteria for a lifetime anxiety disorder diagnosis) who watched an emotional video during functional magnetic resonance imaging and completed questionnaires and clinical evaluation. Of those with functional magnetic resonance imaging data, 432 youths aged 7 to 15 years also self-reported on anxiety symptoms. Data were processed and analyzed between February 2020 and August 2024.

Main Outcomes and Measures

A hidden Markov model was trained to identify brain activation states across participants during video watching. Time spent in each state and the moment-to-moment probability of being in each state were extracted. Videos were annotated for emotion-specific and nonspecific information using the EmoCodes system. Self-reported anxiety symptoms were assessed using the Screen for Child Anxiety Related Disorders. Time spent in each state across the video and during and outside of peaks in negative content correlated with generalized and social anxiety scores.

Results

Among the 620 youths in the overall analysis, 369 were male and the mean (SD) age was 10.4 (2.8) years. In the anxiety symptom analysis, 263 of 432 youths were male and the mean (SD) age was 11.5 (2.2) years. Three brain activation states were identified: a high somatomotor activation state (state 1), a high cingulo-opercular network activation state (state 2), and a high ventral attention and default mode state (state 3). The probability of being in state 3 was correlated with video content that was more negative, quieter, and with less visual motion (ρ < 0.08; P < .001). Increased generalized anxiety was associated with greater time in state 3 (B, 0.10; 95% CI, 0.01 to 0.20; false discovery rate [FDR]-corrected P = .048) and less time in state 2 (B, -0.11; 95% CI, -0.21 to -0.02; FDR-corrected P = .048) when negative social cues were present.

Conclusions and Relevance

Youths entered 3 distinct brain activation states during movie watching, and youths with anxiety spent more time in a state with high ventral attention and default activation during negative socioemotional processing. Youths high in generalized anxiety may be more engaged in deeply processing negative emotional content, which may influence self-regulation. Interventions that focus on changing physiological and psychological state during negative social interactions in youths with anxiety should be considered.

Behaviorally-relevant features of observed actions dominate cortical representational geometry in natural vision

We effortlessly extract behaviorally relevant information from dynamic visual input in order to understand the actions of others. In the current study, we develop and test a number of models to better understand the neural representational geometries supporting action understanding. Using fMRI, we measured brain activity as participants viewed a diverse set of 90 different video clips depicting social and nonsocial actions in real-world contexts. We developed five behavioral models using arrangement tasks: two models reflecting behavioral judgments of the purpose (transitivity) and the social content (sociality) of the actions depicted in the video stimuli; and three models reflecting behavioral judgments of the visual content (people, objects, and scene) depicted in still frames of the stimuli. We evaluated how well these models predict neural representational geometry and tested them against semantic models based on verb and nonverb embeddings and visual models based on gaze and motion energy. Our results revealed that behavioral judgments of similarity better reflect neural representational geometry than semantic or visual models throughout much of cortex. The sociality and transitivity models in particular captured a large portion of unique variance throughout the action observation network, extending into regions not typically associated with action perception, like ventral temporal cortex. Overall, our findings expand the action observation network and indicate that the social content and purpose of observed actions are predominant in cortical representation.

Worry and rumination elicit similar neural representations: neuroimaging evidence for repetitive negative thinking

Repetitive negative thinking (RNT) captures shared cognitive and emotional features of content-specific cognition, including future-focused worry and past-focused rumination. The degree to which these distinct but related processes recruit overlapping neural structures is undetermined, because most neuroscientific studies only examine worry or rumination in isolation. To address this, we developed a paradigm to elicit idiographic worries and ruminations during an fMRI scan in 39 young adults with a range of trait RNT scores. We measured concurrent emotion ratings and heart rate as a physiological metric of arousal. Multivariate representational similarity analysis revealed that regions distributed across default mode, salience, and frontoparietal control networks encode worry and rumination similarly. Moreover, heart rate did not differ between worry and rumination. Capturing the shared neural features between worry and rumination throughout networks supporting self-referential processing, memory, salience detection, and cognitive control provides novel empirical evidence to bolster cognitive and clinical models of RNT.

Proximity within adolescent peer networks predicts neural similarity during affective experience

Individuals befriend others who are similar to them. One important source of similarity in relationships is similarity in felt emotion. In the present study, we used novel methods to assess whether greater similarity in the multivoxel brain representation of affective stimuli was associated with adolescents' proximity within real-world school-based social networks. We examined dyad-level neural similarity within a set of brain regions associated with the representation of affect including the ventromedial prefrontal cortex (vmPFC), amygdala, insula, and temporal pole. Greater proximity was associated with greater vmPFC neural similarity during pleasant and neutral experiences. Moreover, we used unsupervised clustering on social networks to identify groups of friends and observed that individuals from the same (versus different) friend groups were more likely to have greater vmPFC neural similarity during pleasant and negative experiences. These findings suggest that similarity in the multivoxel brain representation of affect may play an important role in adolescent friendships.

Preserved Spontaneous Mentalizing Amid Reduced Intersubject Variability in Autism During a Movie Narrative

BACKGROUND

While individuals with autism often face challenges in everyday social interactions, they may demonstrate proficiency in structured theory of mind (ToM) tasks that assess their ability to infer others' mental states. Using functional magnetic resonance imaging and pupillometry, we investigated whether these discrepancies stem from diminished spontaneous mentalizing or broader difficulties in unstructured contexts.

METHODS

Fifty-two adults diagnosed with autism and 52 neurotypical control participants viewed the animated short Partly Cloudy, a nonverbal animated film with a dynamic social narrative known to engage the ToM brain network during specific scenes. Analysis focused on comparing brain and pupil responses to these ToM events. Additionally, dynamic intersubject correlations were used to explore the variability of these responses throughout the film.

RESULTS

Both groups showed similar brain and pupil responses to ToM events and provided comparable descriptions of the characters' mental states. However, participants with autism exhibited significantly stronger correlations in their responses across the film's social narrative, indicating reduced interindividual variability. This distinct pattern emerged well before any ToM events and involved brain regions beyond the ToM network.

CONCLUSIONS

Our findings provide functional evidence of spontaneous mentalizing in autism, demonstrating this capacity in a context that affords but does not require mentalizing. Rather than responses to ToM events, a novel neurocognitive signature-interindividual variability in brain and pupil responses to evolving social narratives-differentiated neurotypical individuals from individuals with autism. These results suggest that idiosyncratic narrative processing in unstructured settings, a common element of everyday social interactions, may offer a more sensitive scenario for understanding the autistic mind.

Deciphering the neural responses to a naturalistic persuasive message

Effective health promotion may benefit from understanding how persuasion emerges. While earlier research has identified brain regions implicated in persuasion, these studies often relied on event-related analyses and frequently simplified persuasive communications. The present study investigates the neural basis of valuation change induced by a persuasive healthy eating call, employing naturalistic stimuli. Fifty healthy participants performed two blocks of a bidding task, in which they had to bid on sugar-containing, sugar-free, and nonedible products during functional MRI. In between the two blocks, they listened to a persuasive healthy eating call that influenced their bidding behavior. Intriguingly, participants who resisted persuasion exhibited increased synchronization of brain activity during listening in several regions, including default mode network structures. Additionally, intersubject functional connectivity among these brain regions was found to be weaker in persuaded individuals. These results emphasize the individualized nature of processing persuasive messages, challenging conventional interpretations of synchronized neural activity. Our findings support the emerging practice of tailoring persuasive messages in health promotion campaigns.

Movement synchrony among dance performers predicts brain synchrony among dance spectators

Performing dance is an intrinsically social art form where at least one person moves while another person watches. Dancing in groups promotes social bonding, but how does group dance affect the people watching? A group of dancers and dance novices watched a 30 min dance video individually in an fMRI scanner. In a follow-up behavioural study, the same people watched the video again and provided continuous enjoyment ratings. Firstly, we computed cross-recurrence of continuous enjoyment ratings and inter-subject correlations (ISCs) in fMRI separately for both groups, and with the choreographer of the dance work. At both behavioural and neural levels, dancers responded more similarly to each other than novices. ISCs among dancers extended beyond brain areas involved in audio-visual integration and sensory areas of human movement perception into motor areas, suggesting greater sensorimotor familiarity with the observed dance movements in the expert group. Secondly, we show that dancers' brain activations and continuous ratings are more similar to the choreographer's ratings in keeping with sharing an aesthetic and artistic perspective when viewing the dance. Thirdly, we show that movement synchrony among performers is the best predictor of brain synchrony among both expert and novice spectators. This is consistent with the idea that changes in emergent movement synchrony are a key aesthetic feature of performing dance. Finally, ISCs across perceptual and motor brain areas were primarily driven by movement acceleration and synchrony, whereas ISCs in orbital and pre-frontal brain areas were overall weaker and better explained by the continuous enjoyment ratings of each group. Our findings provide strong evidence that the aesthetic appreciation of dance involves a common experience between dance spectators and the choreographer. Moreover, the similarity of brain activations and of enjoyment increases with shared knowledge of - and practice in - the artform that is being experienced, in this case contemporary performing dance.

Auditory areas are recruited for naturalistic visual meaning in early deaf people

Congenital deafness enhances responses of auditory cortices to non-auditory tasks, yet the nature of the reorganization is not well understood. Here, naturalistic stimuli are used to induce neural synchrony across early deaf and hearing individuals. Participants watch a silent animated film in an intact version and three versions with gradually distorted meaning. Differences between groups are observed in higher-order auditory cortices in all stimuli, with no statistically significant effects in the primary auditory cortex. Comparison between levels of scrambling revealed a heterogeneity of function in secondary auditory areas. Both hemispheres show greater synchrony in the deaf than in the hearing participants for the intact movie and high-level variants. However, only the right hemisphere shows an increased inter-subject synchrony in the deaf people for the low-level movie variants. An event segmentation validates these results: the dynamics of the right secondary auditory cortex in the deaf people consist of shorter-length events with more transitions than the left. Our results reveal how deaf individuals use their auditory cortex to process visual meaning.

High-level cognition is supported by information-rich but compressible brain activity patterns

To efficiently yet reliably represent and process information, our brains need to produce information-rich signals that differentiate between moments or cognitive states, while also being robust to noise or corruption. For many, though not all, natural systems, these two properties are often inversely related: More information-rich signals are less robust, and vice versa. Here, we examined how these properties change with ongoing cognitive demands. To this end, we applied dimensionality reduction algorithms and pattern classifiers to functional neuroimaging data collected as participants listened to a story, temporally scrambled versions of the story, or underwent a resting state scanning session. We considered two primary aspects of the neural data recorded in these different experimental conditions. First, we treated the maximum achievable decoding accuracy across participants as an indicator of the "informativeness" of the recorded patterns. Second, we treated the number of features (components) required to achieve a threshold decoding accuracy as a proxy for the "compressibility" of the neural patterns (where fewer components indicate greater compression). Overall, we found that the peak decoding accuracy (achievable without restricting the numbers of features) was highest in the intact (unscrambled) story listening condition. However, the number of features required to achieve comparable classification accuracy was also lowest in the intact story listening condition. Taken together, our work suggests that our brain networks flexibly reconfigure according to ongoing task demands and that the activity patterns associated with higher-order cognition and high engagement are both more informative and more compressible than the activity patterns associated with lower-order tasks and lower engagement.

The causal structure and computational value of narratives

Many human behavioral and brain imaging studies have used narratively structured stimuli (e.g., written, audio, or audiovisual stories) to better emulate real-world experience in the laboratory. However, narratives are a special class of real-world experience, largely defined by their causal connections across time. Much contemporary neuroscience research does not consider this key property. We review behavioral and neuroscientific work that speaks to how causal structure shapes comprehension of and memory for narratives. We further draw connections between this work and reinforcement learning, highlighting how narratives help link causes to outcomes in complex environments. By incorporating the plausibility of causal connections between classes of actions and outcomes, reinforcement learning models may become more ecologically valid, while simultaneously elucidating the value of narratives.

Inter- and intra-subject similarity in network functional connectivity across a full narrative movie

Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging, are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. Previous studies have primarily focused on using shorter movie clips, geared toward eliciting specific and often isolated emotions, while the potential behind using full narratives depicted in commercial movies as a proxy for real-life experiences has barely been explored. Here, we offer preliminary evidence that a full narrative movie (FNM), that is, a movie covering a complete narrative arc, can capture complex socio-affective dynamics and their links to individual differences. Using the studyforrest dataset, we investigated inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into eight consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results show that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. We conclude that FNMs offer complex content and dynamics that might be particularly valuable for studying individual differences. Further characterization of movie features, such as the overarching narratives, that enhance individual differences is needed for advancing the potential of NV research.

The emotion paradox in the aging body and brain

With age, parasympathetic activity decreases, while sympathetic activity increases. Thus, the typical older adult has low heart rate variability (HRV) and high noradrenaline levels. Younger adults with this physiological profile tend to be unhappy and stressed. Yet, with age, emotional experience tends to improve. Why does older adults' emotional well-being not suffer as their HRV decreases? To address this apparent paradox, I present the autonomic compensation model. In this model, failing organs, the initial phases of Alzheimer's pathology, and other age-related diseases trigger noradrenergic hyperactivity. To compensate, older brains increase autonomic regulatory activity in the pregenual prefrontal cortex (PFC). Age-related declines in nerve conduction reduce the ability of the pregenual PFC to reduce hyperactive noradrenergic activity and increase peripheral HRV. But these pregenual PFC autonomic compensation efforts have a significant impact in the brain, where they bias processing in favor of stimuli that tend to increase parasympathetic activity (e.g., stimuli that increase feelings of safety) and against stimuli that tend to increase sympathetic activity (e.g., threatening stimuli). In summary, the autonomic compensation model posits that age-related chronic sympathetic/noradrenergic hyperactivity stimulates regulatory attempts that have the side effect of enhancing emotional well-being.

Inter- and intra-subject similarity in network functional connectivity across a full narrative movie

Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging (fMRI), are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. However, in how far NV elicits similarity within and between subjects on a network level, particularly depending on emotions portrayed in movies, is currently unknown. We used the studyforrest dataset to investigate the inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into 8 consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results showed that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. Lowest similarity both within and between subjects was seen in the emotional regulation network and networks associated with long-term memory processing, which might be explained by specific features and content of the movie. We conclude that detailed characterization of movie features is crucial for NV research.

Value Estimation versus Effort Mobilization: A General Dissociation between Ventromedial and Dorsomedial Prefrontal Cortex

Deciding on a course of action requires both an accurate estimation of option values and the right amount of effort invested in deliberation to reach sufficient confidence in the final choice. In a previous study, we have provided evidence, across a series of judgment and choice tasks, for a dissociation between the ventromedial prefrontal cortex (vmPFC), which would represent option values, and the dorsomedial prefrontal cortex (dmPFC), which would represent the duration of deliberation. Here, we first replicate this dissociation and extend it to the case of an instrumental learning task, in which 24 human volunteers (13 women) choose between options associated with probabilistic gains and losses. According to fMRI data recorded during decision-making, vmPFC activity reflects the sum of option values generated by a reinforcement learning model and dmPFC activity the deliberation time. To further generalize the role of the dmPFC in mobilizing effort, we then analyze fMRI data recorded in the same participants while they prepare to perform motor and cognitive tasks (squeezing a handgrip or making numerical comparisons) to maximize gains or minimize losses. In both cases, dmPFC activity is associated with the output of an effort regulation model, and not with response time. Taken together, these results strengthen a general theory of behavioral control that implicates the vmPFC in the estimation of option values and the dmPFC in the energization of relevant motor and cognitive processes.

Vicarious punishment of moral violations in naturalistic drama narratives predicts cortical synchronization

Punishment of moral norm violators is instrumental for human cooperation. Yet, social and affective neuroscience research has primarily focused on second- and third-party norm enforcement, neglecting the neural architecture underlying observed (vicarious) punishment of moral wrongdoers. We used naturalistic television drama as a sampling space for observing outcomes of morally-relevant behaviors to assess how individuals cognitively process dynamically evolving moral actions and their consequences. Drawing on Affective Disposition Theory, we derived hypotheses linking character morality with viewers' neural processing of characters' rewards and punishments. We used functional magnetic resonance imaging (fMRI) to examine neural responses of 28 female participants while free-viewing 15 short story summary video clips of episodes from a popular US television soap opera. Each summary included a complete narrative structure, fully crossing main character behaviors (moral/immoral) and the consequences (reward/punishment) characters faced for their actions. Narrative engagement was examined via intersubject correlation and representational similarity analysis. Highest cortical synchronization in 9 specifically selected regions previously implicated in processing moral information was observed when characters who act immorally are punished for their actions with participants' empathy as an important moderator. The results advance our understanding of the moral brain and the role of normative considerations and character outcomes in viewers' engagement with popular narratives.

A shared neural code for perceiving and remembering social interactions in the human superior temporal sulcus

Recognizing and remembering social information is a crucial cognitive skill. Neural patterns in the superior temporal sulcus (STS) support our ability to perceive others' social interactions. However, despite the prominence of social interactions in memory, the neural basis of remembering social interactions is still unknown. To fill this gap, we investigated the brain mechanisms underlying memory of others' social interactions during free spoken recall of a naturalistic movie. By applying machine learning-based fMRI encoding analyses to densely labeled movie and recall data we found that a subset of the STS activity evoked by viewing social interactions predicted neural responses in not only held-out movie data, but also during memory recall. These results provide the first evidence that activity in the STS is reinstated in response to specific social content and that its reactivation underlies our ability to remember others' interactions. These findings further suggest that the STS contains representations of social interactions that are not only perceptually driven, but also more abstract or conceptual in nature.

Intersubject representational similarity analysis uncovers the impact of state anxiety on brain activation patterns in the human extrastriate cortex

The current study used functional magnetic resonance imaging (fMRI) and showed that state anxiety modulated extrastriate cortex activity in response to emotionally-charged visual images. State anxiety and neuroimaging data from 53 individuals were subjected to an intersubject representational similarity analysis (ISRSA), wherein the geometries between neural and behavioral data were compared. This analysis identified the extrastriate cortex (fusiform gyrus and area MT) to be the sole regions whose activity patterns covaried with state anxiety. Importantly, we show that this brain-behavior association is revealed when treating state anxiety data as a multidimensional response pattern, rather than a single composite score. This suggests that ISRSA using multivariate distances may be more sensitive in identifying the shared geometries between self-report questionnaires and brain imaging data. Overall, our findings demonstrate that a transient state of anxiety may influence how visual information - especially those relevant to the valence dimension - is processed in the extrastriate cortex.

Neural patterns associated with mixed valence feelings differ in consistency and predictability throughout the brain

Mixed feelings, the simultaneous presence of feelings with positive and negative valence, remain an understudied topic. They pose a specific set of challenges due to individual variation, and their investigation requires analtyic approaches focusing on individually self-reported states. We used functional magnetic resonance imaging (fMRI) to scan 27 subjects watching an animated short film chosen to induce bittersweet mixed feelings. The same subjects labeled when they had experienced positive, negative, and mixed feelings. Using hidden-Markov models, we found that various brain regions could predict the onsets of new feeling states as determined by self-report. The ability of the models to identify these transitions suggests that these states may exhibit unique and consistent neural signatures. We next used the subjects' self-reports to evaluate the spatiotemporal consistency of neural patterns for positive, negative, and mixed states. The insula had unique and consistent neural signatures for univalent states, but not for mixed valence states. The anterior cingulate and ventral medial prefrontal cortex had consistent neural signatures for both univalent and mixed states. This study is the first to demonstrate that subjectively reported changes in feelings induced by naturalistic stimuli can be predicted from fMRI and the first to show direct evidence for a neurally consistent representation of mixed feelings.

Dissecting abstract, modality-specific and experience-dependent coding of affect in the human brain

Emotion and perception are tightly intertwined, as affective experiences often arise from the appraisal of sensory information. Nonetheless, whether the brain encodes emotional instances using a sensory-specific code or in a more abstract manner is unclear. Here, we answer this question by measuring the association between emotion ratings collected during a unisensory or multisensory presentation of a full-length movie and brain activity recorded in typically developed, congenitally blind and congenitally deaf participants. Emotional instances are encoded in a vast network encompassing sensory, prefrontal, and temporal cortices. Within this network, the ventromedial prefrontal cortex stores a categorical representation of emotion independent of modality and previous sensory experience, and the posterior superior temporal cortex maps the valence dimension using an abstract code. Sensory experience more than modality affects how the brain organizes emotional information outside supramodal regions, suggesting the existence of a scaffold for the representation of emotional states where sensory inputs during development shape its functioning.

The Emerging Science of Interacting Minds

For over a century, psychology has focused on uncovering mental processes of a single individual. However, humans rarely navigate the world in isolation. The most important determinants of successful development, mental health, and our individual traits and preferences arise from interacting with other individuals. Social interaction underpins who we are, how we think, and how we behave. Here we discuss the key methodological challenges that have limited progress in establishing a robust science of how minds interact and the new tools that are beginning to overcome these challenges. A deep understanding of the human mind requires studying the context within which it originates and exists: social interaction.

Neural signatures of shared subjective affective engagement and disengagement during movie viewing

When watching a negative emotional movie, we differ from person to person in the ease with which we engage and the difficulty with which we disengage throughout a temporally evolving narrative. We investigated neural responses of emotional processing, by considering inter-individual synchronization in subjective emotional engagement and disengagement. The neural underpinnings of these shared responses are ideally studied in naturalistic scenarios like movie viewing, wherein individuals emotionally engage and disengage at their own time and pace throughout the course of a narrative. Despite the rich data that naturalistic designs can bring to the study, there is a challenge in determining time-resolved behavioral markers of subjective engagement and disengagement and their underlying neural responses. We used a within-subject cross-over design instructing 22 subjects to watch clips of either neutral or sad content while undergoing functional magnetic resonance imaging (fMRI). Participants watched the same movies a second time while continuously annotating the perceived emotional intensity, thus enabling the mapping of brain activity and emotional experience. Our analyses revealed that between-participant similarity in waxing (engagement) and waning (disengagement) of emotional intensity was directly related to the between-participant similarity in spatiotemporal patterns of brain activation during the movie(s). Similar patterns of engagement reflected common activation in the bilateral ventromedial prefrontal cortex, regions often involved in self-referenced evaluation and generation of negative emotions. Similar patterns of disengagement reflected common activation in central executive and default mode network regions often involved in top-down emotion regulation. Together this work helps to better understand cognitive and neural mechanisms underpinning engagement and disengagement from emotionally evocative narratives.

Hostile Attribution Bias Shapes Neural Synchrony in the Left Ventromedial Prefrontal Cortex during Ambiguous Social Narratives

Hostile attribution bias refers to the tendency to interpret social situations as intentionally hostile. While previous research has focused on its developmental origins and behavioral consequences, the underlying neural mechanisms remain underexplored. Here, we employed functional near-infrared spectroscopy (fNIRS) to investigate the neural correlates of hostile attribution bias. While undergoing fNIRS, male and female participants listened to and provided attribution ratings for 21 hypothetical scenarios where a character's actions resulted in a negative outcome for the listener. Ratings of hostile intentions were averaged to measure hostile attribution bias. Using intersubject representational similarity analysis, we found that participants with similar levels of hostile attribution bias exhibited higher levels of neural synchrony during narrative listening, suggesting shared interpretations of the scenarios. This effect was localized to the left ventromedial prefrontal cortex (VMPFC) and was particularly prominent in scenarios where the character's intentions were highly ambiguous. We then grouped participants into high and low bias groups based on a median split of their hostile attribution bias scores. A similarity-based classifier trained on the neural data classified participants as having high or low bias with 75% accuracy, indicating that the neural time courses during narrative listening was systematically different between the two groups. Furthermore, hostile attribution bias correlated negatively with attributional complexity, a measure of one's tendency to consider multifaceted causes when explaining behavior. Our study sheds light on the neural mechanisms underlying hostile attribution bias and highlights the potential of using fNIRS to develop nonintrusive and cost-effective neural markers of this sociocognitive bias.

Deep social neuroscience: the promise and peril of using artificial neural networks to study the social brain

This review offers an accessible primer to social neuroscientists interested in neural networks. It begins by providing an overview of key concepts in deep learning. It then discusses three ways neural networks can be useful to social neuroscientists: (i) building statistical models to predict behavior from brain activity; (ii) quantifying naturalistic stimuli and social interactions; and (iii) generating cognitive models of social brain function. These applications have the potential to enhance the clinical value of neuroimaging and improve the generalizability of social neuroscience research. We also discuss the significant practical challenges, theoretical limitations and ethical issues faced by deep learning. If the field can successfully navigate these hazards, we believe that artificial neural networks may prove indispensable for the next stage of the field's development: deep social neuroscience.

Effects of a dissociative drug on fronto-limbic resting-state functional connectivity in individuals with posttraumatic stress disorder: a randomized controlled pilot study

RATIONALE

A subanesthetic dose of ketamine, a non-competitive N-methyl-D-aspartate glutamate receptor (NMDAR) antagonist, elicits dissociation in individuals with posttraumatic stress disorder (PTSD), who also often suffer from chronic dissociative symptoms in daily life. These debilitating symptoms have not only been linked to worse PTSD trajectories, but also to increased resting-state functional connectivity (RSFC) between medial prefrontal cortex (mPFC) and amygdala, supporting the conceptualization of dissociation as emotion overmodulation. Yet, as studies were observational, causal evidence is lacking.

OBJECTIVES

The present randomized controlled pilot study examines the effect of ketamine, a dissociative drug, on RSFC between mPFC subregions and amygdala in individuals with PTSD.

METHODS

Twenty-six individuals with PTSD received either ketamine (0.5mg/kg; n = 12) or the control drug midazolam (0.045mg/kg; n = 14) during functional magnetic resonance imaging (fMRI). RSFC between amygdala and mPFC subregions, i.e., ventromedial PFC (vmPFC), dorsomedial PFC (dmPFC) and anterior-medial PFC (amPFC), was assessed at baseline and during intravenous drug infusion.

RESULTS

Contrary to pre-registered predictions, ketamine did not promote a greater increase in RSFC between amygdala and mPFC subregions from baseline to infusion compared to midazolam. Instead, ketamine elicited a stronger transient decrease in vmPFC-amygdala RSFC compared to midazolam.

CONCLUSIONS

A dissociative drug did not increase fronto-limbic RSFC in individuals with PTSD. These preliminary experimental findings contrast with prior correlative findings and call for further exploration and, potentially, a more differentiated view on the neurobiological underpinning of dissociative phenomena in PTSD.

Individuals who see the good in the bad engage distinctive default network coordination during post-encoding rest

Focusing on the upside of negative events often promotes resilience. Yet, the underlying mechanisms that allow some people to spontaneously see the good in the bad remain unclear. The broaden-and-build theory of positive emotion has long suggested that positive affect, including positivity in the face of negative events, is linked to idiosyncratic thought patterns (i.e., atypical cognitive responses). Yet, evidence in support of this view has been limited, in part, due to difficulty in measuring idiosyncratic cognitive processes as they unfold. To overcome this barrier, we applied Inter-Subject Representational Similarity Analysis to test whether and how idiosyncratic neural responding supports positive reactions to negative experience. We found that idiosyncratic functional connectivity patterns in the brain's default network while resting after a negative experience predicts more positive descriptions of the event. This effect persisted when controlling for connectivity 1) before and during the negative experience, 2) before, during, and after a neutral experience, and 3) between other relevant brain regions (i.e., the limbic system). The relationship between idiosyncratic default network responding and positive affect was largely driven by functional connectivity patterns between the ventromedial prefrontal cortex and the rest of the default network and occurred relatively quickly during rest. We identified post-encoding rest as a key moment and the default network as a key brain system in which idiosyncratic responses correspond with seeing the good in the bad.

Characterizing the mechanisms of social connection

Understanding how individuals form and maintain strong social networks has emerged as a significant public health priority as a result of the increased focus on the epidemic of loneliness and the myriad protective benefits conferred by social connection. In this review, we highlight the psychological and neural mechanisms that enable us to connect with others, which in turn help buffer against the consequences of stress and isolation. Central to this process is the experience of rewards derived from positive social interactions, which encourage the sharing of perspectives and preferences that unite individuals. Sharing affective states with others helps us to align our understanding of the world with another's, thereby continuing to reinforce bonds and strengthen relationships. These psychological processes depend on neural systems supporting reward and social cognitive function. Lastly, we also consider limitations associated with pursuing healthy social connections and outline potential avenues of future research.

Cortico-hippocampal networks carry information about characters and their relationships in an extended narrative

Social information is a centerpiece of human experience. Despite a wealth of research into the way we understand social relationships and how aspects of social life might be supported by the brain, relatively little is known about how the brain represents individual people and their relationships with others. How do intrinsic networks in the brain track people and their connections in complex situations? Here, we sought to understand this issue using an open neuroimaging dataset in which people freely viewed "The Grand Budapest Hotel." Using support vector machine classification of fMRI activity patterns, we found that character identity could be decoded throughout subsystems of the brain's "Default Mode" Network, especially in regions of an Anterior Temporal and a Medial Prefrontal subsystem, as well as a Medial Temporal Network (MTN). We tested character relationships in two ways - onscreen co-occurrence and shared semantic information from an independent sample of character descriptions - and found evidence for these representations throughout the "Default Mode" Network, and the MTN. The extent to which each variant of character relationships fit neural patterns differed across networks, with abstract semantic relatedness being especially prominent in regions of Anterior Temporal and Medial Prefrontal Networks. These data show that subsystems of the brain's "Default Mode" Network and MTN carry information about individual people as well as their connections, and highlight a particularly strong role for the Anterior Temporal network in representing this information.

Py-Feat: Python Facial Expression Analysis Toolbox

Studying facial expressions is a notoriously difficult endeavor. Recent advances in the field of affective computing have yielded impressive progress in automatically detecting facial expressions from pictures and videos. However, much of this work has yet to be widely disseminated in social science domains such as psychology. Current state-of-the-art models require considerable domain expertise that is not traditionally incorporated into social science training programs. Furthermore, there is a notable absence of user-friendly and open-source software that provides a comprehensive set of tools and functions that support facial expression research. In this paper, we introduce Py-Feat, an open-source Python toolbox that provides support for detecting, preprocessing, analyzing, and visualizing facial expression data. Py-Feat makes it easy for domain experts to disseminate and benchmark computer vision models and also for end users to quickly process, analyze, and visualize face expression data. We hope this platform will facilitate increased use of facial expression data in human behavior research.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-023-00191-4.

Neural signatures of emotional inference and experience align during social consensus

Humans seamlessly transform dynamic social signals into inferences about the internal states of the people around them. To understand the neural processes that sustain this transformation, we collected fMRI data from participants (N = 100) while they rated the emotional intensity of people (targets) describing significant life events. Targets rated themselves on the same scale to indicate the intended "ground truth" emotional intensity of their videos. Next, we developed two multivariate models of observer brain activity- the first predicted the "ground truth" (r = 0.50, p < 0.0001) and the second predicted observer inferences (r = 0.53, p < 0.0001). When individuals make more accurate inferences, there is greater moment-by-moment concordance between these two models, suggesting that an observer's brain activity contains latent representations of other people's emotional states. Using naturalistic socioemotional stimuli and machine learning, we developed reliable brain signatures that predict what an observer thinks about a target, what the target thinks about themselves, and the correspondence between them. These signatures can be applied in clinical data to better our understanding of socioemotional dysfunction.

Perceived financial well-being and its association with frontostriatal functional connectivity, real-life anticipatory experiences, and everyday happiness

Perceived financial well-being (FWB) is an important aspect of life that can affect one's attitude toward future experiences and happiness. However, the relationship between FWB, anticipatory experiences, and happiness, and the brain's functional architecture underlying this relationship remain unknown. Here, we combined an experience sampling method, multilevel modeling, and functional neuroimaging to identify the neural correlates of FWB and their associations with real-world anticipatory experiences and everyday happiness. Behaviorally, we found that individuals with greater FWB felt more positive and more interested when they expected positive events to occur, which in turn resulted in increased everyday happiness. Furthermore, the level of FWB was significantly associated with the strength of functional connectivity (FC) between the nucleus accumbens (NAc) and ventromedial prefrontal cortex (vmPFC) and the local coherence within the vmPFC. The frontostriatal FC and local coherence within the vmPFC were further predictive of everyday happiness via the anticipatory response involving interestedness during positive expectations. Our findings suggest that individual differences in FWB could be reflected in the functional architecture of brain's reward system that may contribute to shaping positive anticipatory experiences and happiness in daily life.

Synchronized affect in shared experiences strengthens social connection

People structure their days to experience events with others. We gather to eat meals, watch TV, and attend concerts together. What constitutes a shared experience and how does it manifest in dyadic behavior? The present study investigates how shared experiences-measured through emotional, motoric, physiological, and cognitive alignment-promote social bonding. We recorded the facial expressions and electrodermal activity (EDA) of participants as they watched four episodes of a TV show for a total of 4 h with another participant. Participants displayed temporally synchronized and spatially aligned emotional facial expressions and the degree of synchronization predicted the self-reported social connection ratings between viewing partners. We observed a similar pattern of results for dyadic physiological synchrony measured via EDA and their cognitive impressions of the characters. All four of these factors, temporal synchrony of positive facial expressions, spatial alignment of expressions, EDA synchrony, and character impression similarity, contributed to a latent factor of a shared experience that predicted social connection. Our findings suggest that the development of interpersonal affiliations in shared experiences emerges from shared affective experiences comprising synchronous processes and demonstrate that these complex interpersonal processes can be studied in a holistic and multi-modal framework leveraging naturalistic experimental designs.

Emotion representations in context: maturation and convergence pathways

How does the human brain develop stable emotion representations? According to recent work by Camacho et al., neural representations of contextualized emotional cues are distinct and fairly stable by mid-to-late childhood and activation patterns become increasingly similar between individuals during adolescence. Here, I propose a framework for investigating contextualized emotion processing.

Topographic representation of visually evoked emotional experiences in the human cerebral cortex

Affective neuroscience seeks to uncover the neural underpinnings of emotions that humans experience. However, it remains unclear whether an affective space underlies the discrete emotion categories in the human brain, and how it relates to the hypothesized affective dimensions. To address this question, we developed a voxel-wise encoding model to investigate the cortical organization of human emotions. Results revealed that the distributed emotion representations are constructed through a fundamental affective space. We further compared each dimension of this space to 14 hypothesized affective dimensions, and found that many affective dimensions are captured by the fundamental affective space. Our results suggest that emotional experiences are represented by broadly spatial overlapping cortical patterns and form smooth gradients across large areas of the cortex. This finding reveals the specific structure of the affective space and its relationship to hypothesized affective dimensions, while highlighting the distributed nature of emotional representations in the cortex.

Intersubject similarity in neural representations underlies shared episodic memory content

Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.

Large-scale encoding of emotion concepts becomes increasingly similar between individuals from childhood to adolescence

Humans require a shared conceptualization of others' emotions for adaptive social functioning. A concept is a mental blueprint that gives our brains parameters for predicting what will happen next. Emotion concepts undergo refinement with development, but it is not known whether their neural representations change in parallel. Here, in a sample of 5-15-year-old children (n = 823), we show that the brain represents different emotion concepts distinctly throughout the cortex, cerebellum and caudate. Patterns of activation to each emotion changed little across development. Using a model-free approach, we show that activation patterns were more similar between older children than between younger children. Moreover, scenes that required inferring negative emotional states elicited higher default mode network activation similarity in older children than younger children. These results suggest that representations of emotion concepts are relatively stable by mid to late childhood and synchronize between individuals during adolescence.

Integrating media content analysis, reception analysis, and media effects studies

Every day, the world of media is at our fingertips, whether it is watching movies, listening to the radio, or browsing online media. On average, people spend over 8 h per day consuming messages from the mass media, amounting to a total lifetime dose of more than 20 years in which conceptual content stimulates our brains. Effects from this flood of information range from short-term attention bursts (e.g., by breaking news features or viral 'memes') to life-long memories (e.g., of one's favorite childhood movie), and from micro-level impacts on an individual's memory, attitudes, and behaviors to macro-level effects on nations or generations. The modern study of media's influence on society dates back to the 1940s. This body of mass communication scholarship has largely asked, "what is media's effect on the individual?" Around the time of the cognitive revolution, media psychologists began to ask, "what cognitive processes are involved in media processing?" More recently, neuroimaging researchers started using real-life media as stimuli to examine perception and cognition under more natural conditions. Such research asks: "what can media tell us about brain function?" With some exceptions, these bodies of scholarship often talk past each other. An integration offers new insights into the neurocognitive mechanisms through which media affect single individuals and entire audiences. However, this endeavor faces the same challenges as all interdisciplinary approaches: Researchers with different backgrounds have different levels of expertise, goals, and foci. For instance, neuroimaging researchers label media stimuli as "naturalistic" although they are in many ways rather artificial. Similarly, media experts are typically unfamiliar with the brain. Neither media creators nor neuroscientifically oriented researchers approach media effects from a social scientific perspective, which is the domain of yet another species. In this article, we provide an overview of approaches and traditions to studying media, and we review the emerging literature that aims to connect these streams. We introduce an organizing scheme that connects the causal paths from media content → brain responses → media effects and discuss network control theory as a promising framework to integrate media content, reception, and effects analyses.

Functional organization of social perception in the human brain

Humans rapidly extract diverse and complex information from ongoing social interactions, but the perceptual and neural organization of the different aspects of social perception remains unresolved. We showed short movie clips with rich social content to 97 healthy participants while their haemodynamic brain activity was measured with fMRI. The clips were annotated moment-to-moment for a large set of social features and 45 of the features were evaluated reliably between annotators. Cluster analysis of the social features revealed that 13 dimensions were sufficient for describing the social perceptual space. Three different analysis methods were used to map the social perceptual processes in the human brain. Regression analysis mapped regional neural response profiles for different social dimensions. Multivariate pattern analysis then established the spatial specificity of the responses and intersubject correlation analysis connected social perceptual processing with neural synchronization. The results revealed a gradient in the processing of social information in the brain. Posterior temporal and occipital regions were broadly tuned to most social dimensions and the classifier revealed that these responses showed spatial specificity for social dimensions; in contrast Heschl gyri and parietal areas were also broadly associated with different social signals, yet the spatial patterns of responses did not differentiate social dimensions. Frontal and subcortical regions responded only to a limited number of social dimensions and the spatial response patterns did not differentiate social dimension. Altogether these results highlight the distributed nature of social processing in the brain.

Valence-partitioned learning signals drive choice behavior and phenomenal subjective experience in humans

How the human brain generates conscious phenomenal experience is a fundamental problem. In particular, it is unknown how variable and dynamic changes in subjective affect are driven by interactions with objective phenomena. We hypothesize a neurocomputational mechanism that generates valence-specific learning signals associated with 'what it is like' to be rewarded or punished. Our hypothesized model maintains a partition between appetitive and aversive information while generating independent and parallel reward and punishment learning signals. This valence-partitioned reinforcement learning (VPRL) model and its associated learning signals are shown to predict dynamic changes in 1) human choice behavior, 2) phenomenal subjective experience, and 3) BOLD-imaging responses that implicate a network of regions that process appetitive and aversive information that converge on the ventral striatum and ventromedial prefrontal cortex during moments of introspection. Our results demonstrate the utility of valence-partitioned reinforcement learning as a neurocomputational basis for investigating mechanisms that may drive conscious experience.

Negative affect homogenizes and positive affect diversifies social memory consolidation across people

We are often surprised when an interaction we remember positively is recalled by a peer negatively. What colors social memories with positive versus negative hues? We show that when resting after a social experience, individuals showing similar default network responding subsequently remember more negative information, while individuals showing idiosyncratic default network responding remember more positive information. Results were specific to rest after the social experience (as opposed to before or during the social experience, or rest after a nonsocial experience). The results provide novel neural evidence in support of the "broaden and build" theory of positive emotion, which posits that while negative affect confines, positive affect broadens idiosyncrasy in cognitive processing. For the first time, we identified post-encoding rest as a key moment and the default network as a key brain system in which negative affect homogenizes, whereas positive affect diversifies social memories.

Representational integration and differentiation in the human hippocampus following goal-directed navigation

As we learn, dynamic memory processes build structured knowledge across our experiences. Such knowledge enables the formation of internal models of the world that we use to plan, make decisions, and act. Recent theorizing posits that mnemonic mechanisms of differentiation and integration - which at one level may seem to be at odds - both contribute to the emergence of structured knowledge. We tested this possibility using fMRI as human participants learned to navigate within local and global virtual environments over the course of 3 days. Pattern similarity analyses on entorhinal cortical and hippocampal patterns revealed evidence that differentiation and integration work concurrently to build local and global environmental representations, and that variability in integration relates to differences in navigation efficiency. These results offer new insights into the neural machinery and the underlying mechanisms that translate experiences into structured knowledge that allows us to navigate to achieve goals.

Shared neural representations and temporal segmentation of political content predict ideological similarity

Despite receiving the same sensory input, opposing partisans often interpret political content in disparate ways. Jointly analyzing controlled and naturalistic functional magnetic resonance imaging data, we uncover the neurobiological mechanisms explaining how these divergent political viewpoints arise. Individuals who share an ideology have more similar neural representations of political words, experience greater neural synchrony during naturalistic political content, and temporally segment real-world information into the same meaningful units. In the striatum and amygdala, increasing intersubject similarity in neural representations of political concepts during a word reading task predicts enhanced synchronization of blood oxygen level-dependent time courses when viewing real-time, inflammatory political videos, revealing that polarization can arise from differences in the brain's affective valuations of political concepts. Together, this research shows that political ideology is shaped by semantic representations of political concepts processed in an environment free of any polarizing agenda and that these representations bias how real-world political information is construed into a polarized perspective.

Don't You Forget About Me: The Importance of Studying the Brain Basis of Real-world Interpersonal Memory

Despite the fact that humans are a highly social species, we know relatively little about how people remember the rich interpersonal information filling their social lives. This gap is surprising: One function of memory has been suggested to be relationship maintenance [Neisser, U. Time present and time past. Practical Aspects of Memory: Current Research and Issues, 2, 545-560, 1988]. A major barrier to understanding the brain basis of interpersonal memory is that traditional brain imaging methods are not ideally suited to study memory for the nuanced interpersonal experiences comprising our social lives. Yet, recent and rapidly developing advances in the analysis of brain responses to naturalistic social information can help researchers surpass this methodological barrier. This perspective piece articulates the importance of studying the brain basis of real-world social memories and suggests new directions in interpersonal memory research. This includes investigating the brain mechanisms that represent the content and structure of real-world interpersonal memories as well as how they are altered in mental health conditions associated with social memory biases.

Recovering Individual Emotional States from Sparse Ratings Using Collaborative Filtering

A fundamental challenge in emotion research is measuring feeling states with high granularity and temporal precision without disrupting the emotion generation process. Here we introduce and validate a new approach in which responses are sparsely sampled and the missing data are recovered using a computational technique known as collaborative filtering (CF). This approach leverages structured covariation across individual experiences and is available in Neighbors, an open-source Python toolbox. We validate our approach across three different experimental contexts by recovering dense individual ratings using only a small subset of the original data. In dataset 1, participants (n=316) separately rated 112 emotional images on 6 different discrete emotions. In dataset 2, participants (n=203) watched 8 short emotionally engaging autobiographical stories while simultaneously providing moment-by-moment ratings of the intensity of their affective experience. In dataset 3, participants (n=60) with distinct social preferences made 76 decisions about how much money to return in a hidden multiplier trust game. Across all experimental contexts, CF was able to accurately recover missing data and importantly outperformed mean and multivariate imputation, particularly in contexts with greater individual variability. This approach will enable new avenues for affective science research by allowing researchers to acquire high dimensional ratings from emotional experiences with minimal disruption to the emotion-generation process.

Supplementary Information

The online version contains supplementary material available at 10.1007/s42761-022-00161-2.