1
|
Chen J, Bornstein AM. The causal structure and computational value of narratives. Trends Cogn Sci 2024; 28:769-781. [PMID: 38734531 PMCID: PMC11305923 DOI: 10.1016/j.tics.2024.04.003] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/08/2023] [Revised: 04/03/2024] [Accepted: 04/08/2024] [Indexed: 05/13/2024]
Abstract
Many human behavioral and brain imaging studies have used narratively structured stimuli (e.g., written, audio, or audiovisual stories) to better emulate real-world experience in the laboratory. However, narratives are a special class of real-world experience, largely defined by their causal connections across time. Much contemporary neuroscience research does not consider this key property. We review behavioral and neuroscientific work that speaks to how causal structure shapes comprehension of and memory for narratives. We further draw connections between this work and reinforcement learning, highlighting how narratives help link causes to outcomes in complex environments. By incorporating the plausibility of causal connections between classes of actions and outcomes, reinforcement learning models may become more ecologically valid, while simultaneously elucidating the value of narratives.
Collapse
Affiliation(s)
- Janice Chen
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA.
| | - Aaron M Bornstein
- Department of Cognitive Sciences, University of California, Irvine, CA, USA; Center for the Neurobiology of Learning and Memory, University of California, Irvine, CA, USA
| |
Collapse
|
2
|
Lee SA, Lee JJ, Han J, Choi M, Wager TD, Woo CW. Brain representations of affective valence and intensity in sustained pleasure and pain. Proc Natl Acad Sci U S A 2024; 121:e2310433121. [PMID: 38857402 PMCID: PMC11194486 DOI: 10.1073/pnas.2310433121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2023] [Accepted: 04/18/2024] [Indexed: 06/12/2024] Open
Abstract
Pleasure and pain are two fundamental, intertwined aspects of human emotions. Pleasurable sensations can reduce subjective feelings of pain and vice versa, and we often perceive the termination of pain as pleasant and the absence of pleasure as unpleasant. This implies the existence of brain systems that integrate them into modality-general representations of affective experiences. Here, we examined representations of affective valence and intensity in an functional MRI (fMRI) study (n = 58) of sustained pleasure and pain. We found that the distinct subpopulations of voxels within the ventromedial and lateral prefrontal cortices, the orbitofrontal cortex, the anterior insula, and the amygdala were involved in decoding affective valence versus intensity. Affective valence and intensity predictive models showed significant decoding performance in an independent test dataset (n = 62). These models were differentially connected to distinct large-scale brain networks-the intensity model to the ventral attention network and the valence model to the limbic and default mode networks. Overall, this study identified the brain representations of affective valence and intensity across pleasure and pain, promoting a systems-level understanding of human affective experiences.
Collapse
Affiliation(s)
- Soo Ahn Lee
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon16419, Republic of Korea
- Department of Biomedical Engineering, Sungkyunkwan University, Suwon16419, Republic of Korea
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University, Suwon16419, Republic of Korea
| | - Jae-Joong Lee
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon16419, Republic of Korea
| | - Jisoo Han
- Korea Brain Research Institute, Daegu41062, Republic of Korea
| | - Myunghwan Choi
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon16419, Republic of Korea
- School of Biological Sciences, Seoul National University, Seoul08826, Republic of Korea
| | - Tor D. Wager
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH03755
| | - Choong-Wan Woo
- Center for Neuroscience Imaging Research, Institute for Basic Science, Suwon16419, Republic of Korea
- Department of Biomedical Engineering, Sungkyunkwan University, Suwon16419, Republic of Korea
- Department of Intelligent Precision Healthcare Convergence, Sungkyunkwan University, Suwon16419, Republic of Korea
- Life-inspired Neural Network for Prediction and Optimization Research Group, Suwon16419, Republic of Korea
| |
Collapse
|
3
|
Lee Masson H, Chen J, Isik L. A shared neural code for perceiving and remembering social interactions in the human superior temporal sulcus. Neuropsychologia 2024; 196:108823. [PMID: 38346576 DOI: 10.1016/j.neuropsychologia.2024.108823] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2023] [Revised: 01/15/2024] [Accepted: 02/09/2024] [Indexed: 02/20/2024]
Abstract
Recognizing and remembering social information is a crucial cognitive skill. Neural patterns in the superior temporal sulcus (STS) support our ability to perceive others' social interactions. However, despite the prominence of social interactions in memory, the neural basis of remembering social interactions is still unknown. To fill this gap, we investigated the brain mechanisms underlying memory of others' social interactions during free spoken recall of a naturalistic movie. By applying machine learning-based fMRI encoding analyses to densely labeled movie and recall data we found that a subset of the STS activity evoked by viewing social interactions predicted neural responses in not only held-out movie data, but also during memory recall. These results provide the first evidence that activity in the STS is reinstated in response to specific social content and that its reactivation underlies our ability to remember others' interactions. These findings further suggest that the STS contains representations of social interactions that are not only perceptually driven, but also more abstract or conceptual in nature.
Collapse
Affiliation(s)
- Haemy Lee Masson
- Department of Psychology, Durham University, Durham, DH1 3LE, United Kingdom; Department of Cognitive Science, Johns Hopkins University, Baltimore, MD, 21218, United States.
| | - Janice Chen
- Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, 21218, United States
| | - Leyla Isik
- Department of Cognitive Science, Johns Hopkins University, Baltimore, MD, 21218, United States.
| |
Collapse
|
4
|
Ju U, Wallraven C. Decoding the dynamic perception of risk and speed using naturalistic stimuli: A multivariate, whole-brain analysis. Hum Brain Mapp 2024; 45:e26652. [PMID: 38488473 PMCID: PMC10941534 DOI: 10.1002/hbm.26652] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/13/2023] [Revised: 02/20/2024] [Accepted: 02/25/2024] [Indexed: 03/18/2024] Open
Abstract
Time-resolved decoding of speed and risk perception in car driving is important for understanding the perceptual processes related to driving safety. In this study, we used an fMRI-compatible trackball with naturalistic stimuli to record dynamic ratings of perceived risk and speed and investigated the degree to which different brain regions were able to decode these. We presented participants with first-person perspective videos of cars racing on the same course. These videos varied in terms of subjectively perceived speed and risk profiles, as determined during a behavioral pilot. During the fMRI experiment, participants used the trackball to dynamically rate subjective risk in a first and speed in a second session and assessed overall risk and speed after watching each video. A standard multivariate correlation analysis based on these ratings revealed sparse decodability in visual areas only for the risk ratings. In contrast, the dynamic rating-based correlation analysis uncovered frontal, visual, and temporal region activation for subjective risk and dorsal visual stream and temporal region activation for subjectively perceived speed. Interestingly, further analyses showed that the brain regions for decoding risk changed over time, whereas those for decoding speed remained constant. Overall, our results demonstrate the advantages of time-resolved decoding to help our understanding of the dynamic networks associated with decoding risk and speed perception in realistic driving scenarios.
Collapse
Affiliation(s)
- Uijong Ju
- Department of Information DisplayKyung Hee UniversitySeoulSouth Korea
| | - Christian Wallraven
- Department of Brain and Cognitive EngineeringKorea UniversitySouth Korea
- Department of Artificial IntelligenceKorea UniversitySouth Korea
| |
Collapse
|
5
|
Lee S, Kim J. Testing the bipolar assumption of Singer-Loomis Type Deployment Inventory for Korean adults using classification and multidimensional scaling. Front Psychol 2024; 14:1249185. [PMID: 38356992 PMCID: PMC10864660 DOI: 10.3389/fpsyg.2023.1249185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/28/2023] [Accepted: 12/26/2023] [Indexed: 02/16/2024] Open
Abstract
In this study, we explored whether the Korean version of Singer Loomis Type Deployment Inventory II (K-SLTDI) captures the opposing tendencies of Jung's theory of psychological type. The types are Extroverted Sensing, Extroverted Intuition, Extroverted Feeling, Extroverted Thinking, Introverted Sensing, Introverted Intuition, Introverted Feeling, and Introverted Thinking. A nationwide online survey was conducted in South Korea. We performed multidimensional scaling and classification analyses based on 521 Korean adult profiles with eight psychological types to test the bipolarity assumption. The results showed that the Procrustes-rotated four-dimensional space successfully represented four types of opposing tendencies. Moreover, the bipolarity assumption in the four dimensions of Jungian typology was tested and compared between lower and higher psychological distress populations via cluster analysis. Lastly, we explored patterns of responses in lower and higher psychological distress populations using intersubject correlation. Both similarity analyses and classification results consistently support the theoretical considerations on the conceptualization of Jung's type in independent order that the types could be derived without bipolar assumption as Singer and Loomis expected in their Type Development Inventory. Limitations in our study include the sample being randomly selected internet users during the COVID-19 pandemic, despite excellence in the use of the internet in the general Korean population.
Collapse
Affiliation(s)
| | - Jongwan Kim
- Psychology Department, Jeonbuk National University, Jeonju, Republic of Korea
| |
Collapse
|
6
|
Mahrukh R, Shakil S, Malik AS. Sentiments analysis of fMRI using automatically generated stimuli labels under naturalistic paradigm. Sci Rep 2023; 13:7267. [PMID: 37142654 PMCID: PMC10160115 DOI: 10.1038/s41598-023-33734-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 04/18/2023] [Indexed: 05/06/2023] Open
Abstract
Our emotions and sentiments are influenced by naturalistic stimuli such as the movies we watch and the songs we listen to, accompanied by changes in our brain activation. Comprehension of these brain-activation dynamics can assist in identification of any associated neurological condition such as stress and depression, leading towards making informed decision about suitable stimuli. A large number of open-access functional magnetic resonance imaging (fMRI) datasets collected under naturalistic conditions can be used for classification/prediction studies. However, these datasets do not provide emotion/sentiment labels, which limits their use in supervised learning studies. Manual labeling by subjects can generate these labels, however, this method is subjective and biased. In this study, we are proposing another approach of generating automatic labels from the naturalistic stimulus itself. We are using sentiment analyzers (VADER, TextBlob, and Flair) from natural language processing to generate labels using movie subtitles. Subtitles generated labels are used as the class labels for positive, negative, and neutral sentiments for classification of brain fMRI images. Support vector machine, random forest, decision tree, and deep neural network classifiers are used. We are getting reasonably good classification accuracy (42-84%) for imbalanced data, which is increased (55-99%) for balanced data.
Collapse
Affiliation(s)
| | - Sadia Shakil
- Institute of Space Technology, Islamabad, Pakistan.
- Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic.
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia.
| | - Aamir Saeed Malik
- Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic.
| |
Collapse
|
7
|
Lee KM, Lee S, Satpute AB. Sinful pleasures and pious woes? Using fMRI to examine evaluative and hedonic emotion knowledge. Soc Cogn Affect Neurosci 2022; 17:986-994. [PMID: 35348768 PMCID: PMC9629474 DOI: 10.1093/scan/nsac024] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/24/2021] [Revised: 02/24/2022] [Accepted: 03/22/2022] [Indexed: 01/12/2023] Open
Abstract
Traditionally, lust and pride have been considered pleasurable, yet sinful in the West. Conversely, guilt is often considered aversive, yet valuable. These emotions illustrate how evaluations about specific emotions and beliefs about their hedonic properties may often diverge. Evaluations about specific emotions may shape important aspects of emotional life (e.g. in emotion regulation, emotion experience and acquisition of emotion concepts). Yet these evaluations are often understudied in affective neuroscience. Prior work in emotion regulation, affective experience, evaluation/attitudes and decision-making point to anterior prefrontal areas as candidates for supporting evaluative emotion knowledge. Thus, we examined the brain areas associated with evaluative and hedonic emotion knowledge, with a focus on the anterior prefrontal cortex. Participants (N = 25) made evaluative and hedonic ratings about emotion knowledge during functional magnetic resonance imaging (fMRI). We found that greater activity in the medial prefrontal cortex (mPFC), ventromedial PFC (vmPFC) and precuneus was associated with an evaluative (vs hedonic) focus on emotion knowledge. Our results suggest that the mPFC and vmPFC, in particular, may play a role in evaluating discrete emotions.
Collapse
Affiliation(s)
- Kent M Lee
- Correspondence should be addressed to Kent M. Lee, Department of Psychology, Northeastern University, 125 Nightingale Hall, Boston, MA, USA. E-mail:
| | - SuhJin Lee
- Department of Neurobiology, University of Pittsburgh School of Medicine, Pittsburgh, PA 15213, USA
| | - Ajay B Satpute
- Department of Psychology, Northeastern University, Boston, MA 02115, USA
| |
Collapse
|
8
|
Functional selectivity for social interaction perception in the human superior temporal sulcus during natural viewing. Neuroimage 2021; 245:118741. [PMID: 34800663 DOI: 10.1016/j.neuroimage.2021.118741] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2021] [Revised: 09/15/2021] [Accepted: 11/16/2021] [Indexed: 11/22/2022] Open
Abstract
Recognizing others' social interactions is a crucial human ability. Using simple stimuli, previous studies have shown that social interactions are selectively processed in the superior temporal sulcus (STS), but prior work with movies has suggested that social interactions are processed in the medial prefrontal cortex (mPFC), part of the theory of mind network. It remains unknown to what extent social interaction selectivity is observed in real world stimuli when controlling for other covarying perceptual and social information, such as faces, voices, and theory of mind. The current study utilizes a functional magnetic resonance imaging (fMRI) movie paradigm and advanced machine learning methods to uncover the brain mechanisms uniquely underlying naturalistic social interaction perception. We analyzed two publicly available fMRI datasets, collected while both male and female human participants (n = 17 and 18) watched two different commercial movies in the MRI scanner. By performing voxel-wise encoding and variance partitioning analyses, we found that broad social-affective features predict neural responses in social brain regions, including the STS and mPFC. However, only the STS showed robust and unique selectivity specifically to social interactions, independent from other covarying features. This selectivity was observed across two separate fMRI datasets. These findings suggest that naturalistic social interaction perception recruits dedicated neural circuity in the STS, separate from the theory of mind network, and is a critical dimension of human social understanding.
Collapse
|
9
|
Lee KM, Ferreira-Santos F, Satpute AB. Predictive processing models and affective neuroscience. Neurosci Biobehav Rev 2021; 131:211-228. [PMID: 34517035 DOI: 10.1016/j.neubiorev.2021.09.009] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2019] [Revised: 02/10/2021] [Accepted: 09/07/2021] [Indexed: 01/17/2023]
Abstract
The neural bases of affective experience remain elusive. Early neuroscience models of affect searched for specific brain regions that uniquely carried out the computations that underlie dimensions of valence and arousal. However, a growing body of work has failed to identify these circuits. Research turned to multivariate analyses, but these strategies, too, have made limited progress. Predictive processing models offer exciting new directions to address this problem. Here, we use predictive processing models as a lens to critique prevailing functional neuroimaging research practices in affective neuroscience. Our review highlights how much work relies on rigid assumptions that are inconsistent with a predictive processing approach. We outline the central aspects of a predictive processing model and draw out their implications for research in affective and cognitive neuroscience. Predictive models motivate a reformulation of "reverse inference" in cognitive neuroscience, and placing a greater emphasis on external validity in experimental design.
Collapse
Affiliation(s)
- Kent M Lee
- Northeastern University, 360 Huntington Ave, 125 NI, Boston, MA 02118, USA.
| | - Fernando Ferreira-Santos
- Laboratory of Neuropsychophysiology, Faculty of Psychology and Education Sciences, University of Porto, Portugal
| | - Ajay B Satpute
- Northeastern University, 360 Huntington Ave, 125 NI, Boston, MA 02118, USA
| |
Collapse
|
10
|
Marxen M, Jacob MJ, Hellrung L, Riedel P, Smolka MN. Questioning the role of amygdala and insula in an attentional capture by emotional stimuli task. Hum Brain Mapp 2021; 42:1257-1267. [PMID: 33216427 PMCID: PMC7927307 DOI: 10.1002/hbm.25290] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2020] [Revised: 10/16/2020] [Accepted: 10/22/2020] [Indexed: 12/13/2022] Open
Abstract
Our senses are constantly monitoring the environment for emotionally salient stimuli that are potentially relevant for survival. Because of our limited cognitive resources, emotionally salient distractors prolong reaction times (RTs) as compared to neutral distractors. In addition, many studies have reported fMRI blood oxygen level-dependent (BOLD) activation of both the amygdala and the anterior insula for similar valence contrasts. However, a direct correlation of trail-by-trial BOLD activity with RTs has not been shown, yet, which would be a crucial piece of evidence to relate the two observations. To investigate the role of the above two regions in the context of emotional distractor effects, we study here the correlation between BOLD activity and RTs for a simple attentional capture by emotional stimuli (ACES) choice reaction time task using a general linear subject-level model with a parametric RT regressor. We found significant regression coefficients in the anterior insula, supplementary motor cortex, medial precentral regions, sensory-motor areas and others, but not in the amygdala, despite activation of both insula and amygdala in the traditional valence contrast across trials (i.e., negative vs. neutral pictures). In addition, we found that subjects that exhibit a stronger RT distractor effect across trials also show a stronger BOLD valence contrast in the right anterior insula but not in the amygdala. Our results indicate that the current neuroimaging-based evidence for the involvement of the amygdala in RT slowing is limited. We advocate that models of emotional capture should incorporate both the amygdala and the anterior insula as separate entities.
Collapse
Affiliation(s)
- Michael Marxen
- Department of Psychiatry and Neuroimaging CenterTechnische Universität DresdenDresdenGermany
| | - Mark J. Jacob
- Department of Psychiatry and Neuroimaging CenterTechnische Universität DresdenDresdenGermany
| | - Lydia Hellrung
- Department of Psychiatry and Neuroimaging CenterTechnische Universität DresdenDresdenGermany
- Department of EconomicsUniversity of ZurichZurichSwitzerland
| | - Philipp Riedel
- Department of Psychiatry and Neuroimaging CenterTechnische Universität DresdenDresdenGermany
| | - Michael N. Smolka
- Department of Psychiatry and Neuroimaging CenterTechnische Universität DresdenDresdenGermany
| |
Collapse
|
11
|
Shinkareva SV, Gao C, Wedell D. Audiovisual Representations of Valence: a Cross-study Perspective. ACTA ACUST UNITED AC 2020; 1:237-246. [DOI: 10.1007/s42761-020-00023-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 10/22/2020] [Indexed: 01/25/2023]
|