1
|
Bacha-Trams M, Yorulmaz GE, Glerean E, Ryyppö E, Tapani K, Virmavirta E, Saaristo J, Jääskeläinen IP, Sams M. Sisterhood predicts similar neural processing of a film. Neuroimage 2024; 297:120712. [PMID: 38945181 DOI: 10.1016/j.neuroimage.2024.120712] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2024] [Revised: 06/10/2024] [Accepted: 06/25/2024] [Indexed: 07/02/2024] Open
Abstract
Relationships between humans are essential for how we see the world. Using fMRI, we explored the neural basis of homophily, a sociological concept that describes the tendency to bond with similar others. Our comparison of brain activity between sisters, friends and acquaintances while they watched a movie, indicate that sisters' brain activity is more similar than that of friends and friends' activity is more similar than that of acquaintances. The increased similarity in brain activity measured as inter-subject correlation (ISC) was found both in higher-order brain areas including the default-mode network (DMN) and sensory areas. Increased ISC could not be explained by genetic relation between sisters neither by similarities in eye-movements, emotional experiences, and physiological activity. Our findings shed light on the neural basis of homophily by revealing that similarity in brain activity in the DMN and sensory areas is the stronger the closer is the relationship between the people.
Collapse
Affiliation(s)
- Mareike Bacha-Trams
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland; Institute of Research Methods in Psychology - Media-based Knowledge Construction, Computer Science and Applied Cognitive Science, University of Duisburg-Essen, Germany.
| | - Gökce Ertas Yorulmaz
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Enrico Glerean
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Elisa Ryyppö
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Karoliina Tapani
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Eero Virmavirta
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Jenni Saaristo
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Iiro P Jääskeläinen
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland; Advanced Magnetic Imaging (AMI) Centre, Aalto NeuroImaging, Aalto University, Espoo, Finland.
| | - Mikko Sams
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland; Aalto Studios - MAGICS, Aalto University, Espoo, Finland
| |
Collapse
|
2
|
Lerner Y, Raz G, Bloch M, Krasnoshtein M, Tevet M, Hendler T, Tene O. Empathy-related abnormalities among women with premenstrual dysphoric disorder: clinical and functional magnetic resonance imaging study. BJPsych Open 2024; 10:e138. [PMID: 39101206 DOI: 10.1192/bjo.2024.723] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 08/06/2024] Open
Abstract
BACKGROUND Empathy refers to the cognitive and emotional reactions of an individual to the experiences of another. Women with premenstrual dysphoric disorder (PMDD) report severe social difficulties during the luteal phase of their menstrual cycle. AIMS This clinical and functional magnetic resonance imaging study aimed to explore affective and cognitive empathy in women with PMDD, during the highly symptomatic luteal phase. METHOD Overall, 32 women with PMDD and 20 healthy controls participated in the study. The neuroimaging data were collected using a highly empathy-engaging movie. First, we characterised the synchrony of neural responses within PMDD and healthy groups, using the inter-individual correlation approach. Next, using network cohesion analysis, we compared connectivity within and between brain networks associated with affective and cognitive empathy between groups, and assessed the association of these network patterns with empathic measures. RESULTS A consistent, although complex, picture of empathy abnormalities was found. Patients with PMDD showed decreased neural synchrony in parietal and frontal key nodes of cognitive empathy processing (theory-of-mind network), but higher neural synchrony in the anterior insula and anterior cingulate cortex, a part of the salience network, implicated in affective empathy. Positive correlations between cognitive perspective-taking scores and neural synchrony were found within the theory-of-mind network. Interestingly, during highly emotional moments, the PMDD group showed increased functional connectivity within this network. CONCLUSIONS Similar to major depression, individuals with PMDD show enhanced affective empathy and reduced cognitive empathy. These findings echo clinical observations reported when women with PMDD have a dysregulated emotional response to negative stimuli.
Collapse
Affiliation(s)
- Yulia Lerner
- Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Israel; Faculty of Medicine, Tel Aviv University, Israel; and Sagol School of Neuroscience, Tel Aviv University, Israel
| | - Gal Raz
- Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Israel; Sagol School of Neuroscience, Tel Aviv University, Israel; and Steve Tisch School of Film and Television, Faculty of the Arts, Tel Aviv University, Israel
| | - Miki Bloch
- Faculty of Medicine, Tel Aviv University, Israel; Psychiatric Division, Tel Aviv Sourasky Medical Center, Israel; and Brull Ramat Chen Mental Health Center, Tel Aviv, Israel
| | - Michael Krasnoshtein
- Faculty of Medicine, Tel Aviv University, Israel; Psychiatric Division, Tel Aviv Sourasky Medical Center, Israel; and Ambulatory Department, Yehuda Abarbanel Mental Health Medical Center, Bat Yam, Israel
| | - Michal Tevet
- Psychiatric Division, Tel Aviv Sourasky Medical Center, Israel
| | - Talma Hendler
- Sagol Brain Institute, Tel Aviv Sourasky Medical Center, Israel; Faculty of Medicine, Tel Aviv University, Israel; Sagol School of Neuroscience, Tel Aviv University, Israel; and School of Psychological Sciences, Tel Aviv University, Israel
| | - Oren Tene
- Faculty of Medicine, Tel Aviv University, Israel; and Psychiatric Division, Tel Aviv Sourasky Medical Center, Israel
| |
Collapse
|
3
|
Mochalski LN, Friedrich P, Li X, Kröll J, Eickhoff SB, Weis S. Inter- and intra-subject similarity in network functional connectivity across a full narrative movie. Hum Brain Mapp 2024; 45:e26802. [PMID: 39086203 PMCID: PMC11291869 DOI: 10.1002/hbm.26802] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2023] [Revised: 07/11/2024] [Accepted: 07/17/2024] [Indexed: 08/02/2024] Open
Abstract
Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging, are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. Previous studies have primarily focused on using shorter movie clips, geared toward eliciting specific and often isolated emotions, while the potential behind using full narratives depicted in commercial movies as a proxy for real-life experiences has barely been explored. Here, we offer preliminary evidence that a full narrative movie (FNM), that is, a movie covering a complete narrative arc, can capture complex socio-affective dynamics and their links to individual differences. Using the studyforrest dataset, we investigated inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into eight consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results show that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. We conclude that FNMs offer complex content and dynamics that might be particularly valuable for studying individual differences. Further characterization of movie features, such as the overarching narratives, that enhance individual differences is needed for advancing the potential of NV research.
Collapse
Affiliation(s)
- Lisa N. Mochalski
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
- Institute of Systems Neuroscience, Medical FacultyHeinrich Heine University DüsseldorfDüsseldorfGermany
| | - Patrick Friedrich
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
| | - Xuan Li
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
| | - Jean‐Philippe Kröll
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
| | - Simon B. Eickhoff
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
- Institute of Systems Neuroscience, Medical FacultyHeinrich Heine University DüsseldorfDüsseldorfGermany
| | - Susanne Weis
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM‐7)Research Centre JülichJülichGermany
- Institute of Systems Neuroscience, Medical FacultyHeinrich Heine University DüsseldorfDüsseldorfGermany
| |
Collapse
|
4
|
te Rietmolen N, Mercier MR, Trébuchon A, Morillon B, Schön D. Speech and music recruit frequency-specific distributed and overlapping cortical networks. eLife 2024; 13:RP94509. [PMID: 39038076 PMCID: PMC11262799 DOI: 10.7554/elife.94509] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/24/2024] Open
Abstract
To what extent does speech and music processing rely on domain-specific and domain-general neural networks? Using whole-brain intracranial EEG recordings in 18 epilepsy patients listening to natural, continuous speech or music, we investigated the presence of frequency-specific and network-level brain activity. We combined it with a statistical approach in which a clear operational distinction is made between shared, preferred, and domain-selective neural responses. We show that the majority of focal and network-level neural activity is shared between speech and music processing. Our data also reveal an absence of anatomical regional selectivity. Instead, domain-selective neural responses are restricted to distributed and frequency-specific coherent oscillations, typical of spectral fingerprints. Our work highlights the importance of considering natural stimuli and brain dynamics in their full complexity to map cognitive and brain functions.
Collapse
Affiliation(s)
- Noémie te Rietmolen
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| | - Manuel R Mercier
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| | - Agnès Trébuchon
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
- APHM, Hôpital de la Timone, Service de Neurophysiologie CliniqueMarseilleFrance
| | - Benjamin Morillon
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| | - Daniele Schön
- Institute for Language, Communication, and the Brain, Aix-Marseille UniversityMarseilleFrance
- Aix Marseille Université, INSERM, INS, Institut de Neurosciences des SystèmesMarseilleFrance
| |
Collapse
|
5
|
Hoshal BD, Holmes CM, Bojanek K, Salisbury J, Berry MJ, Marre O, Palmer SE. Stimulus invariant aspects of the retinal code drive discriminability of natural scenes. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2023.08.08.552526. [PMID: 37609259 PMCID: PMC10441377 DOI: 10.1101/2023.08.08.552526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/24/2023]
Abstract
Everything that the brain sees must first be encoded by the retina, which maintains a reliable representation of the visual world in many different, complex natural scenes while also adapting to stimulus changes. This study quantifies whether and how the brain selectively encodes stimulus features about scene identity in complex naturalistic environments. While a wealth of previous work has dug into the static and dynamic features of the population code in retinal ganglion cells, less is known about how populations form both flexible and reliable encoding in natural moving scenes. We record from the larval salamander retina responding to five different natural movies, over many repeats, and use these data to characterize the population code in terms of single-cell fluctuations in rate and pairwise couplings between cells. Decomposing the population code into independent and cell-cell interactions reveals how broad scene structure is encoded in the retinal output. while the single-cell activity adapts to different stimuli, the population structure captured in the sparse, strong couplings is consistent across natural movies as well as synthetic stimuli. We show that these interactions contribute to encoding scene identity. We also demonstrate that this structure likely arises in part from shared bipolar cell input as well as from gap junctions between retinal ganglion cells and amacrine cells.
Collapse
|
6
|
Noad KN, Watson DM, Andrews TJ. Familiarity enhances functional connectivity between visual and nonvisual regions of the brain during natural viewing. Cereb Cortex 2024; 34:bhae285. [PMID: 39038830 DOI: 10.1093/cercor/bhae285] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Revised: 06/18/2024] [Accepted: 06/26/2024] [Indexed: 07/24/2024] Open
Abstract
We explored the neural correlates of familiarity with people and places using a naturalistic viewing paradigm. Neural responses were measured using functional magnetic resonance imaging, while participants viewed a movie taken from Game of Thrones. We compared inter-subject correlations and functional connectivity in participants who were either familiar or unfamiliar with the TV series. Higher inter-subject correlations were found between familiar participants in regions, beyond the visual brain, that are typically associated with the processing of semantic, episodic, and affective information. However, familiarity also increased functional connectivity between face and scene regions in the visual brain and the nonvisual regions of the familiarity network. To determine whether these regions play an important role in face recognition, we measured responses in participants with developmental prosopagnosia (DP). Consistent with a deficit in face recognition, the effect of familiarity was significantly attenuated across the familiarity network in DP. The effect of familiarity on functional connectivity between face regions and the familiarity network was also attenuated in DP. These results show that the neural response to familiarity involves an extended network of brain regions and that functional connectivity between visual and nonvisual regions of the brain plays an important role in the recognition of people and places during natural viewing.
Collapse
Affiliation(s)
- Kira N Noad
- Department of Psychology, University of York, York Y010 5DD, United Kingdom
| | - David M Watson
- Department of Psychology, University of York, York Y010 5DD, United Kingdom
| | - Timothy J Andrews
- Department of Psychology, University of York, York Y010 5DD, United Kingdom
| |
Collapse
|
7
|
Oakes RA, Peschel L, Barraclough NE. Inter-subject correlation of audience facial expressions predicts audience engagement during theatrical performances. iScience 2024; 27:109843. [PMID: 38779478 PMCID: PMC11109022 DOI: 10.1016/j.isci.2024.109843] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Revised: 01/10/2024] [Accepted: 04/25/2024] [Indexed: 05/25/2024] Open
Abstract
During performances, audiences experience various emotional states, and these are reflected in their ongoing facial expressions. We investigated if audience engagement could be determined by measuring the inter-subject correlation (ISC) of non-invasively recorded audience facial expressions. We filmed the faces of multiple audience members at theatrical performances and determined the intensity of their different facial expressions throughout the performances. Neutral, happy, anger, and disgust expression ISCs accounted for up to 24% of the performance dramaturge's predictions of audience engagement. Expression synchrony was greater between individuals in close proximity, suggesting effects of emotional contagion or cognitive similarities between neighboring individuals, whereas expression synchrony was greatest between individuals who were younger, female, and with greater levels of empathy, showing that individual characteristics impact shared audience experiences. Together, our results show that facial expression synchronization could be used as a real-time non-invasive indicator of engagement in audiences larger than achieved using previous approaches.
Collapse
Affiliation(s)
- Richard A. Oakes
- Department of Psychology, University of York, Heslington, YO10 5DD York, UK
- Department of Theatre, Film and Television and Interactive Media, University of York, Heslington, YO10 5DD York, UK
| | - Lisa Peschel
- Department of Theatre, Film and Television and Interactive Media, University of York, Heslington, YO10 5DD York, UK
| | | |
Collapse
|
8
|
Cohen MS, Leong YC, Ruby K, Pape RA, Decety J. Intersubject correlations in reward and mentalizing brain circuits separately predict persuasiveness of two types of ISIS video propaganda. Sci Rep 2024; 14:13455. [PMID: 38862592 PMCID: PMC11166951 DOI: 10.1038/s41598-024-62341-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2023] [Accepted: 05/15/2024] [Indexed: 06/13/2024] Open
Abstract
The Islamist group ISIS has been particularly successful at recruiting Westerners as terrorists. A hypothesized explanation is their simultaneous use of two types of propaganda: Heroic narratives, emphasizing individual glory, alongside Social narratives, which emphasize oppression against Islamic communities. In the current study, functional MRI was used to measure brain responses to short ISIS propaganda videos distributed online. Participants were shown 4 Heroic and 4 Social videos categorized as such by another independent group of subjects. Persuasiveness was measured using post-scan predictions of recruitment effectiveness. Inter-subject correlation (ISC) was used to measure commonality of brain activity time courses across individuals. ISCs in ventral striatum predicted rated persuasiveness for Heroic videos, while ISCs in mentalizing and default networks, especially in dmPFC, predicted rated persuasiveness for Social videos. This work builds on past findings that engagement of the reward circuit and of mentalizing brain regions predicts preferences and persuasion. The observed dissociation as a function of stimulus type is novel, as is the finding that intersubject synchrony in ventral striatum predicts rated persuasiveness. These exploratory results identify possible neural mechanisms by which political extremists successfully recruit prospective members and specifically support the hypothesized distinction between Heroic and Social narratives for ISIS propaganda.
Collapse
Affiliation(s)
- Michael S Cohen
- Department of Psychology, University of Chicago, 5848 S. University Ave, Chicago, IL, 60637, USA.
| | - Yuan Chang Leong
- Department of Psychology, University of Chicago, 5848 S. University Ave, Chicago, IL, 60637, USA
| | - Keven Ruby
- Department of Political Science, University of Chicago, Chicago, IL, USA
- Chicago Project on Security and Threats, University of Chicago, Chicago, IL, USA
| | - Robert A Pape
- Department of Political Science, University of Chicago, Chicago, IL, USA
- Chicago Project on Security and Threats, University of Chicago, Chicago, IL, USA
| | - Jean Decety
- Department of Psychology, University of Chicago, 5848 S. University Ave, Chicago, IL, 60637, USA.
- Department of Psychiatry and Behavioral Neuroscience, University of Chicago, Chicago, IL, USA.
| |
Collapse
|
9
|
Tripathi SC, Garg R. Consistent movement of viewers' facial keypoints while watching emotionally evocative videos. PLoS One 2024; 19:e0302705. [PMID: 38758739 PMCID: PMC11101037 DOI: 10.1371/journal.pone.0302705] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2023] [Accepted: 04/09/2024] [Indexed: 05/19/2024] Open
Abstract
Neuropsychological research aims to unravel how diverse individuals' brains exhibit similar functionality when exposed to the same stimuli. The evocation of consistent responses when different subjects watch the same emotionally evocative stimulus has been observed through modalities like fMRI, EEG, physiological signals and facial expressions. We refer to the quantification of these shared consistent signals across subjects at each time instant across the temporal dimension as Consistent Response Measurement (CRM). CRM is widely explored through fMRI, occasionally with EEG, physiological signals and facial expressions using metrics like Inter-Subject Correlation (ISC). However, fMRI tools are expensive and constrained, while EEG and physiological signals are prone to facial artifacts and environmental conditions (such as temperature, humidity, and health condition of subjects). In this research, facial expression videos are used as a cost-effective and flexible alternative for CRM, minimally affected by external conditions. By employing computer vision-based automated facial keypoint tracking, a new metric similar to ISC, called the Average t-statistic, is introduced. Unlike existing facial expression-based methodologies that measure CRM of secondary indicators like inferred emotions, keypoint, and ICA-based features, the Average t-statistic is closely associated with the direct measurement of consistent facial muscle movement using the Facial Action Coding System (FACS). This is evidenced in DISFA dataset where the time-series of Average t-statistic has a high correlation (R2 = 0.78) with a metric called AU consistency, which directly measures facial muscle movement through FACS coding of video frames. The simplicity of recording facial expressions with the automated Average t-statistic expands the applications of CRM such as measuring engagement in online learning, customer interactions, etc., and diagnosing outliers in healthcare conditions like stroke, autism, depression, etc. To promote further research, we have made the code repository publicly available.
Collapse
Affiliation(s)
- Shivansh Chandra Tripathi
- Department of Computer Science and Engineering, Indian Institute of Technology Delhi, New Delhi, India
| | - Rahul Garg
- Department of Computer Science and Engineering, Indian Institute of Technology Delhi, New Delhi, India
- Amar Nath and Shashi Khosla School of Information Technology, Indian Institute of Technology Delhi, New Delhi, India
- National Resource Centre for Value Education in Engineering, Indian Institute of Technology Delhi, New Delhi, India
| |
Collapse
|
10
|
Mochalski LN, Friedrich P, Li X, Kröll JP, Eickhoff SB, Weis S. Inter- and intra-subject similarity in network functional connectivity across a full narrative movie. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.05.14.594107. [PMID: 38798405 PMCID: PMC11118367 DOI: 10.1101/2024.05.14.594107] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/29/2024]
Abstract
Naturalistic paradigms, such as watching movies during functional magnetic resonance imaging (fMRI), are thought to prompt the emotional and cognitive processes typically elicited in real life situations. Therefore, naturalistic viewing (NV) holds great potential for studying individual differences. However, in how far NV elicits similarity within and between subjects on a network level, particularly depending on emotions portrayed in movies, is currently unknown. We used the studyforrest dataset to investigate the inter- and intra-subject similarity in network functional connectivity (NFC) of 14 meta-analytically defined networks across a full narrative, audio-visual movie split into 8 consecutive movie segments. We characterized the movie segments by valence and arousal portrayed within the sequences, before utilizing a linear mixed model to analyze which factors explain inter- and intra-subject similarity. Our results showed that the model best explaining inter-subject similarity comprised network, movie segment, valence and a movie segment by valence interaction. Intra-subject similarity was influenced significantly by the same factors and an additional three-way interaction between movie segment, valence and arousal. Overall, inter- and intra-subject similarity in NFC were sensitive to the ongoing narrative and emotions in the movie. Lowest similarity both within and between subjects was seen in the emotional regulation network and networks associated with long-term memory processing, which might be explained by specific features and content of the movie. We conclude that detailed characterization of movie features is crucial for NV research.
Collapse
|
11
|
Sievers B, Welker C, Hasson U, Kleinbaum AM, Wheatley T. Consensus-building conversation leads to neural alignment. Nat Commun 2024; 15:3936. [PMID: 38729961 PMCID: PMC11087652 DOI: 10.1038/s41467-023-43253-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2020] [Accepted: 11/03/2023] [Indexed: 05/12/2024] Open
Abstract
Conversation is a primary means of social influence, but its effects on brain activity remain unknown. Previous work on conversation and social influence has emphasized public compliance, largely setting private beliefs aside. Here, we show that consensus-building conversation aligns future brain activity within groups, with alignment persisting through novel experiences participants did not discuss. Participants watched ambiguous movie clips during fMRI scanning, then conversed in groups with the goal of coming to a consensus about each clip's narrative. After conversation, participants' brains were scanned while viewing the clips again, along with novel clips from the same movies. Groups that reached consensus showed greater similarity of brain activity after conversation. Participants perceived as having high social status spoke more and signaled disbelief in others, and their groups had unequal turn-taking and lower neural alignment. By contrast, participants with central positions in their real-world social networks encouraged others to speak, facilitating greater group neural alignment. Socially central participants were also more likely to become neurally aligned to others in their groups.
Collapse
Affiliation(s)
- Beau Sievers
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA.
- Department of Psychology, Stanford University, Stanford, CA, 94305, USA.
| | - Christopher Welker
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA
| | - Uri Hasson
- Princeton Neuroscience Institute and Department of Psychology, Princeton University, Princeton, NJ, 08544, USA
| | - Adam M Kleinbaum
- Tuck School of Business, Dartmouth College, Hanover, NH, 03755, USA
| | - Thalia Wheatley
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, 03755, USA
- Santa Fe Institute, Santa Fe, NM, 87501, USA
| |
Collapse
|
12
|
Tripathy K, Fogarty M, Svoboda AM, Schroeder ML, Rafferty SM, Richter EJ, Tracy C, Mansfield PK, Booth M, Fishell AK, Sherafati A, Markow ZE, Wheelock MD, Arbeláez AM, Schlaggar BL, Smyser CD, Eggebrecht AT, Culver JP. Mapping brain function in adults and young children during naturalistic viewing with high-density diffuse optical tomography. Hum Brain Mapp 2024; 45:e26684. [PMID: 38703090 PMCID: PMC11069306 DOI: 10.1002/hbm.26684] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/01/2023] [Revised: 03/27/2024] [Accepted: 04/03/2024] [Indexed: 05/06/2024] Open
Abstract
Human studies of early brain development have been limited by extant neuroimaging methods. MRI scanners present logistical challenges for imaging young children, while alternative modalities like functional near-infrared spectroscopy have traditionally been limited by image quality due to sparse sampling. In addition, conventional tasks for brain mapping elicit low task engagement, high head motion, and considerable participant attrition in pediatric populations. As a result, typical and atypical developmental trajectories of processes such as language acquisition remain understudied during sensitive periods over the first years of life. We evaluate high-density diffuse optical tomography (HD-DOT) imaging combined with movie stimuli for high resolution optical neuroimaging in awake children ranging from 1 to 7 years of age. We built an HD-DOT system with design features geared towards enhancing both image quality and child comfort. Furthermore, we characterized a library of animated movie clips as a stimulus set for brain mapping and we optimized associated data analysis pipelines. Together, these tools could map cortical responses to movies and contained features such as speech in both adults and awake young children. This study lays the groundwork for future research to investigate response variability in larger pediatric samples and atypical trajectories of early brain development in clinical populations.
Collapse
Affiliation(s)
- Kalyan Tripathy
- Division of Biological and Biomedical SciencesWashington University in St. LouisSt. LouisMissouriUSA
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
- Western Psychiatric HospitalUniversity of Pittsburgh Medical CenterPittsburghPennsylvaniaUSA
| | - Morgan Fogarty
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
- Imaging Science ProgramWashington University in St. LouisSt. LouisMissouriUSA
| | - Alexandra M. Svoboda
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Mariel L. Schroeder
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Sean M. Rafferty
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Edward J. Richter
- Department of Electrical and Systems EngineeringWashington University in St. LouisSt. LouisMissouriUSA
| | - Christopher Tracy
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Patricia K. Mansfield
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Madison Booth
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Andrew K. Fishell
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Arefeh Sherafati
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
- Department of PhysicsWashington University in St. LouisSt. LouisMissouriUSA
| | - Zachary E. Markow
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
- Department of Biomedical EngineeringWashington University in St. LouisSt. LouisMissouriUSA
| | - Muriah D. Wheelock
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Ana María Arbeláez
- Department of PediatricsWashington University School of MedicineSt. LouisMissouriUSA
| | - Bradley L. Schlaggar
- Kennedy Krieger InstituteBaltimoreMarylandUSA
- Department of NeurologyJohns Hopkins University School of MedicineBaltimoreMarylandUSA
- Department of PediatricsJohns Hopkins University School of MedicineBaltimoreMarylandUSA
| | - Christopher D. Smyser
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
- Department of PediatricsWashington University School of MedicineSt. LouisMissouriUSA
- Department of NeurologyWashington University School of MedicineSt. LouisMissouriUSA
| | - Adam T. Eggebrecht
- Division of Biological and Biomedical SciencesWashington University in St. LouisSt. LouisMissouriUSA
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
- Imaging Science ProgramWashington University in St. LouisSt. LouisMissouriUSA
- Department of Electrical and Systems EngineeringWashington University in St. LouisSt. LouisMissouriUSA
- Department of PhysicsWashington University in St. LouisSt. LouisMissouriUSA
- Department of Biomedical EngineeringWashington University in St. LouisSt. LouisMissouriUSA
| | - Joseph P. Culver
- Mallinckrodt Institute of RadiologyWashington University School of MedicineSt. LouisMissouriUSA
- Imaging Science ProgramWashington University in St. LouisSt. LouisMissouriUSA
- Department of PhysicsWashington University in St. LouisSt. LouisMissouriUSA
- Department of Biomedical EngineeringWashington University in St. LouisSt. LouisMissouriUSA
| |
Collapse
|
13
|
Kim SG, De Martino F, Overath T. Linguistic modulation of the neural encoding of phonemes. Cereb Cortex 2024; 34:bhae155. [PMID: 38687241 PMCID: PMC11059272 DOI: 10.1093/cercor/bhae155] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 03/21/2024] [Accepted: 03/22/2024] [Indexed: 05/02/2024] Open
Abstract
Speech comprehension entails the neural mapping of the acoustic speech signal onto learned linguistic units. This acousto-linguistic transformation is bi-directional, whereby higher-level linguistic processes (e.g. semantics) modulate the acoustic analysis of individual linguistic units. Here, we investigated the cortical topography and linguistic modulation of the most fundamental linguistic unit, the phoneme. We presented natural speech and "phoneme quilts" (pseudo-randomly shuffled phonemes) in either a familiar (English) or unfamiliar (Korean) language to native English speakers while recording functional magnetic resonance imaging. This allowed us to dissociate the contribution of acoustic vs. linguistic processes toward phoneme analysis. We show that (i) the acoustic analysis of phonemes is modulated by linguistic analysis and (ii) that for this modulation, both of acoustic and phonetic information need to be incorporated. These results suggest that the linguistic modulation of cortical sensitivity to phoneme classes minimizes prediction error during natural speech perception, thereby aiding speech comprehension in challenging listening situations.
Collapse
Affiliation(s)
- Seung-Goo Kim
- Department of Psychology and Neuroscience, Duke University, 308 Research Dr, Durham, NC 27708, United States
- Research Group Neurocognition of Music and Language, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, Frankfurt am Main 60322, Germany
| | - Federico De Martino
- Faculty of Psychology and Neuroscience, University of Maastricht, Universiteitssingel 40, 6229 ER Maastricht, Netherlands
| | - Tobias Overath
- Department of Psychology and Neuroscience, Duke University, 308 Research Dr, Durham, NC 27708, United States
- Duke Institute for Brain Sciences, Duke University, 308 Research Dr, Durham, NC 27708, United States
- Center for Cognitive Neuroscience, Duke University, 308 Research Dr, Durham, NC 27708, United States
| |
Collapse
|
14
|
Rahrig H, Beloboradova P, Castro C, Sabet K, Johnson M, Pearce O, Brown KW. Managing emotions in the age of political polarization: A randomized controlled trial comparing mindfulness to cognitive reappraisal. RESEARCH SQUARE 2024:rs.3.rs-3947259. [PMID: 38586010 PMCID: PMC10996818 DOI: 10.21203/rs.3.rs-3947259/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/09/2024]
Abstract
Emotional appraisals of political stimuli (e.g., videos) have been shown to drive shared neural encoding, which correspond to shared, yet divisive, interpretations of such stimuli. However, mindfulness practice may entrain a form of emotion regulation that de-automatizes social biases, possibly through alteration of such neural mechanisms. The present study combined a naturalistic neuroimaging paradigm and a randomized controlled trial to examine the effects of short-term mindfulness training (MT) (n = 35) vs structurally equivalent Cognitive Reappraisal training (CT) (n = 37) on politically-situated emotions while evaluating the mechanistic role of prefrontal cortical neural synchrony. Participants underwent functional near-infrared spectroscopy (fNIRS) recording while viewing inflammatory partisan news clips and continuously rating their momentary discrete emotions. MT participants were more likely to respond with extreme levels of anger (odds ratio = 0.12, p < .001) and disgust (odds ratio = 0.08, p < .001) relative to CT participants. Neural synchrony-based analyses suggested that participants with extreme emotion reactions exhibited greater prefrontal cortical neural synchrony, but that this pattern was less prominent in participants receiving MT relative to CT (CT > MT; channel 1 ISC = .040, p = .030).
Collapse
Affiliation(s)
- Hadley Rahrig
- Department of Psychology, University of Wisconsin-Madison, Madison, WI, 53703, United States of America
| | - Polina Beloboradova
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, 23284, United States of America
| | - Christina Castro
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, 23284, United States of America
| | - Kayla Sabet
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, 23284, United States of America
| | - Melina Johnson
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, 23284, United States of America
| | - Orion Pearce
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, 23284, United States of America
| | - Kirk Warren Brown
- Department of Psychology, Virginia Commonwealth University, Richmond, VA, 23284, United States of America
- Health and Human Performance Lab, Carnegie Mellon University, Pittsburgh, PA, 15213, United States of America
| |
Collapse
|
15
|
Thye M, Hoffman P, Mirman D. The neural basis of naturalistic semantic and social cognition. Sci Rep 2024; 14:6796. [PMID: 38514738 PMCID: PMC10957894 DOI: 10.1038/s41598-024-56897-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Accepted: 03/11/2024] [Indexed: 03/23/2024] Open
Abstract
Decoding social environments and engaging meaningfully with other people are critical aspects of human cognition. Multiple cognitive systems, including social and semantic cognition, work alongside each other to support these processes. This study investigated shared processing between social and semantic systems using neuroimaging data collected during movie-viewing, which captures the multimodal environment in which social knowledge is exchanged. Semantic and social content from movie events (event-level) and movie transcripts (word-level) were used in parametric modulation analyses to test (1) the degree to which semantic and social information is processed within each respective network and (2) engagement of the same cross-network regions or the same domain-general hub located within the semantic network during semantic and social processing. Semantic word and event-level content engaged the same fronto-temporo-parietal network and a portion of the semantic hub in the anterior temporal lobe (ATL). Social word and event-level content engaged the supplementary motor area and right angular gyrus within the social network, but only social words engaged the domain-general semantic hub in left ATL. There was evidence of shared processing between the social and semantic systems in the dorsolateral portion of right ATL which was engaged by word and event-level semantic and social content. Overlap between the semantic and social word and event results was highly variable within and across participants, with the most consistent loci of overlap occurring in left inferior frontal, bilateral precentral and supramarginal gyri for social and semantic words and in bilateral superior temporal gyrus extending from ATL posteriorly into supramarginal gyri for social and semantic events. These results indicate a complex pattern of shared and distinct regions for social and semantic cognition during naturalistic processing. PROTOCOL REGISTRATION: The stage 1 protocol for this Registered Report was accepted in principle on October 11, 2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/ACWQY .
Collapse
Affiliation(s)
- Melissa Thye
- Department of Psychology, University of Edinburgh, Edinburgh, EH8 9JZ, UK.
| | - Paul Hoffman
- Department of Psychology, University of Edinburgh, Edinburgh, EH8 9JZ, UK
| | - Daniel Mirman
- Department of Psychology, University of Edinburgh, Edinburgh, EH8 9JZ, UK
| |
Collapse
|
16
|
Gu C, Peng Y, Nastase SA, Mayer RE, Li P. Onscreen presence of instructors in video lectures affects learners' neural synchrony and visual attention during multimedia learning. Proc Natl Acad Sci U S A 2024; 121:e2309054121. [PMID: 38466840 PMCID: PMC10963011 DOI: 10.1073/pnas.2309054121] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Accepted: 01/20/2024] [Indexed: 03/13/2024] Open
Abstract
COVID-19 forced students to rely on online learning using multimedia tools, and multimedia learning continues to impact education beyond the pandemic. In this study, we combined behavioral, eye-tracking, and neuroimaging paradigms to identify multimedia learning processes and outcomes. College students viewed four video lectures including slides with either an onscreen human instructor, an animated instructor, or no onscreen instructor. Brain activity was recorded via fMRI, visual attention was recorded via eye-tracking, and learning outcome was assessed via post-tests. Onscreen presence of instructor, compared with no instructor presence, resulted in superior post-test performance, less visual attention on the slide, more synchronized eye movements during learning, and higher neural synchronization in cortical networks associated with socio-emotional processing and working memory. Individual variation in cognitive and socio-emotional abilities and intersubject neural synchronization revealed different levels of cognitive and socio-emotional processing in different learning conditions. The instructor-present condition evoked increased synchronization, likely reflecting extra processing demands in attentional control, working memory engagement, and socio-emotional processing. Although human instructors and animated instructors led to comparable learning outcomes, the effects were due to the dynamic interplay of information processing vs. attentional distraction. These findings reflect a benefit-cost trade-off where multimedia learning outcome is enhanced only when the cognitive benefits motivated by the social presence of onscreen instructor outweigh the cognitive costs brought about by concurrent attentional distraction unrelated to learning.
Collapse
Affiliation(s)
- Chanyuan Gu
- Department of Chinese and Bilingual Studies, Faculty of Humanities, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Yingying Peng
- Department of Chinese and Bilingual Studies, Faculty of Humanities, The Hong Kong Polytechnic University, Hong Kong SAR, China
| | - Samuel A. Nastase
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ08540
| | - Richard E. Mayer
- Department of Psychological and Brain Sciences, University of California, Santa Barbara, CA93106
| | - Ping Li
- Department of Chinese and Bilingual Studies, Faculty of Humanities, The Hong Kong Polytechnic University, Hong Kong SAR, China
- Centre for Immersive Learning and Metaverse in Education, The Hong Kong Polytechnic University, Hong Kong SAR, China
| |
Collapse
|
17
|
Shain C, Schuler W. A Deep Learning Approach to Analyzing Continuous-Time Cognitive Processes. Open Mind (Camb) 2024; 8:235-264. [PMID: 38528907 PMCID: PMC10962694 DOI: 10.1162/opmi_a_00126] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Accepted: 01/31/2024] [Indexed: 03/27/2024] Open
Abstract
The dynamics of the mind are complex. Mental processes unfold continuously in time and may be sensitive to a myriad of interacting variables, especially in naturalistic settings. But statistical models used to analyze data from cognitive experiments often assume simplistic dynamics. Recent advances in deep learning have yielded startling improvements to simulations of dynamical cognitive processes, including speech comprehension, visual perception, and goal-directed behavior. But due to poor interpretability, deep learning is generally not used for scientific analysis. Here, we bridge this gap by showing that deep learning can be used, not just to imitate, but to analyze complex processes, providing flexible function approximation while preserving interpretability. To do so, we define and implement a nonlinear regression model in which the probability distribution over the response variable is parameterized by convolving the history of predictors over time using an artificial neural network, thereby allowing the shape and continuous temporal extent of effects to be inferred directly from time series data. Our approach relaxes standard simplifying assumptions (e.g., linearity, stationarity, and homoscedasticity) that are implausible for many cognitive processes and may critically affect the interpretation of data. We demonstrate substantial improvements on behavioral and neuroimaging data from the language processing domain, and we show that our model enables discovery of novel patterns in exploratory analyses, controls for diverse confounds in confirmatory analyses, and opens up research questions in cognitive (neuro)science that are otherwise hard to study.
Collapse
Affiliation(s)
- Cory Shain
- Department of Brain & Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, MA, USA
| | - William Schuler
- Department of Linguistics, The Ohio State University, Columbus, OH, USA
| |
Collapse
|
18
|
Chen C, Dupré la Tour T, Gallant JL, Klein D, Deniz F. The cortical representation of language timescales is shared between reading and listening. Commun Biol 2024; 7:284. [PMID: 38454134 PMCID: PMC11245628 DOI: 10.1038/s42003-024-05909-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Accepted: 02/12/2024] [Indexed: 03/09/2024] Open
Abstract
Language comprehension involves integrating low-level sensory inputs into a hierarchy of increasingly high-level features. Prior work studied brain representations of different levels of the language hierarchy, but has not determined whether these brain representations are shared between written and spoken language. To address this issue, we analyze fMRI BOLD data that were recorded while participants read and listened to the same narratives in each modality. Levels of the language hierarchy are operationalized as timescales, where each timescale refers to a set of spectral components of a language stimulus. Voxelwise encoding models are used to determine where different timescales are represented across the cerebral cortex, for each modality separately. These models reveal that between the two modalities timescale representations are organized similarly across the cortical surface. Our results suggest that, after low-level sensory processing, language integration proceeds similarly regardless of stimulus modality.
Collapse
Affiliation(s)
- Catherine Chen
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA.
| | - Tom Dupré la Tour
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| | - Jack L Gallant
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| | - Daniel Klein
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, CA, USA
| | - Fatma Deniz
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA.
- Institute of Software Engineering and Theoretical Computer Science, Technische Universität Berlin, Berlin, Germany.
- Bernstein Center for Computational Neuroscience, Berlin, Germany.
| |
Collapse
|
19
|
Mecklenbrauck F, Gruber M, Siestrup S, Zahedi A, Grotegerd D, Mauritz M, Trempler I, Dannlowski U, Schubotz RI. The significance of structural rich club hubs for the processing of hierarchical stimuli. Hum Brain Mapp 2024; 45:e26543. [PMID: 38069537 PMCID: PMC10915744 DOI: 10.1002/hbm.26543] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Revised: 10/17/2023] [Accepted: 11/09/2023] [Indexed: 03/07/2024] Open
Abstract
The brain's structural network follows a hierarchy that is described as rich club (RC) organization, with RC hubs forming the well-interconnected top of this hierarchy. In this study, we tested whether RC hubs are involved in the processing of hierarchically higher structures in stimulus sequences. Moreover, we explored the role of previously suggested cortical gradients along anterior-posterior and medial-lateral axes throughout the frontal cortex. To this end, we conducted a functional magnetic resonance imaging (fMRI) experiment and presented participants with blocks of digit sequences that were structured on different hierarchically nested levels. We additionally collected diffusion weighted imaging data of the same subjects to identify RC hubs. This classification then served as the basis for a region of interest analysis of the fMRI data. Moreover, we determined structural network centrality measures in areas that were found as activation clusters in the whole-brain fMRI analysis. Our findings support the previously found anterior and medial shift for processing hierarchically higher structures of stimuli. Additionally, we found that the processing of hierarchically higher structures of the stimulus structure engages RC hubs more than for lower levels. Areas involved in the functional processing of hierarchically higher structures were also more likely to be part of the structural RC and were furthermore more central to the structural network. In summary, our results highlight the potential role of the structural RC organization in shaping the cortical processing hierarchy.
Collapse
Affiliation(s)
- Falko Mecklenbrauck
- Department of Psychology, Biological PsychologyUniversity of MünsterMünsterGermany
- Otto Creutzfeldt Center for Cognitive and Behavioral NeuroscienceUniversity of MünsterMünsterGermany
| | - Marius Gruber
- Institute for Translational PsychiatryUniversity of MünsterMünsterGermany
- Department for Psychiatry, Psychosomatic Medicine and PsychotherapyUniversity Hospital Frankfurt, Goethe UniversityFrankfurtGermany
| | - Sophie Siestrup
- Department of Psychology, Biological PsychologyUniversity of MünsterMünsterGermany
- Otto Creutzfeldt Center for Cognitive and Behavioral NeuroscienceUniversity of MünsterMünsterGermany
| | - Anoushiravan Zahedi
- Department of Psychology, Biological PsychologyUniversity of MünsterMünsterGermany
- Otto Creutzfeldt Center for Cognitive and Behavioral NeuroscienceUniversity of MünsterMünsterGermany
| | - Dominik Grotegerd
- Institute for Translational PsychiatryUniversity of MünsterMünsterGermany
| | - Marco Mauritz
- Institute for Translational PsychiatryUniversity of MünsterMünsterGermany
- Institute for Computational and Applied MathematicsUniversity of MünsterMünsterGermany
| | - Ima Trempler
- Department of Psychology, Biological PsychologyUniversity of MünsterMünsterGermany
- Otto Creutzfeldt Center for Cognitive and Behavioral NeuroscienceUniversity of MünsterMünsterGermany
| | - Udo Dannlowski
- Otto Creutzfeldt Center for Cognitive and Behavioral NeuroscienceUniversity of MünsterMünsterGermany
- Institute for Translational PsychiatryUniversity of MünsterMünsterGermany
| | - Ricarda I. Schubotz
- Department of Psychology, Biological PsychologyUniversity of MünsterMünsterGermany
- Otto Creutzfeldt Center for Cognitive and Behavioral NeuroscienceUniversity of MünsterMünsterGermany
| |
Collapse
|
20
|
Fink L, Simola J, Tavano A, Lange E, Wallot S, Laeng B. From pre-processing to advanced dynamic modeling of pupil data. Behav Res Methods 2024; 56:1376-1412. [PMID: 37351785 PMCID: PMC10991010 DOI: 10.3758/s13428-023-02098-1] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/20/2023] [Indexed: 06/24/2023]
Abstract
The pupil of the eye provides a rich source of information for cognitive scientists, as it can index a variety of bodily states (e.g., arousal, fatigue) and cognitive processes (e.g., attention, decision-making). As pupillometry becomes a more accessible and popular methodology, researchers have proposed a variety of techniques for analyzing pupil data. Here, we focus on time series-based, signal-to-signal approaches that enable one to relate dynamic changes in pupil size over time with dynamic changes in a stimulus time series, continuous behavioral outcome measures, or other participants' pupil traces. We first introduce pupillometry, its neural underpinnings, and the relation between pupil measurements and other oculomotor behaviors (e.g., blinks, saccades), to stress the importance of understanding what is being measured and what can be inferred from changes in pupillary activity. Next, we discuss possible pre-processing steps, and the contexts in which they may be necessary. Finally, we turn to signal-to-signal analytic techniques, including regression-based approaches, dynamic time-warping, phase clustering, detrended fluctuation analysis, and recurrence quantification analysis. Assumptions of these techniques, and examples of the scientific questions each can address, are outlined, with references to key papers and software packages. Additionally, we provide a detailed code tutorial that steps through the key examples and figures in this paper. Ultimately, we contend that the insights gained from pupillometry are constrained by the analysis techniques used, and that signal-to-signal approaches offer a means to generate novel scientific insights by taking into account understudied spectro-temporal relationships between the pupil signal and other signals of interest.
Collapse
Affiliation(s)
- Lauren Fink
- Department of Music, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322, Frankfurt am Main, Germany.
- Department of Psychology, Neuroscience & Behavior, McMaster University, 1280 Main St. West, Hamilton, Ontario, L8S 4L8, Canada.
| | - Jaana Simola
- Helsinki Collegium for Advanced Studies, University of Helsinki, Helsinki, Finland
- Department of Education, University of Helsinki, Helsinki, Finland
| | - Alessandro Tavano
- Department of Cognitive Neuropsychology, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| | - Elke Lange
- Department of Music, Max Planck Institute for Empirical Aesthetics, Grüneburgweg 14, 60322, Frankfurt am Main, Germany
| | - Sebastian Wallot
- Department of Literature, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
- Institute for Sustainability Education and Psychologyy, Leuphana University, Lüneburg, Germany
| | - Bruno Laeng
- Department of Psychology, University of Oslo, Oslo, Norway
- RITMO Centre for Interdisciplinary studies in Rhythm, Time, and Motion, University of Oslo, Oslo, Norway
| |
Collapse
|
21
|
Nanni-Zepeda M, DeGutis J, Wu C, Rothlein D, Fan Y, Grimm S, Walter M, Esterman M, Zuberer A. Neural signatures of shared subjective affective engagement and disengagement during movie viewing. Hum Brain Mapp 2024; 45:e26622. [PMID: 38488450 DOI: 10.1002/hbm.26622] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2023] [Revised: 01/10/2024] [Accepted: 01/26/2024] [Indexed: 03/19/2024] Open
Abstract
When watching a negative emotional movie, we differ from person to person in the ease with which we engage and the difficulty with which we disengage throughout a temporally evolving narrative. We investigated neural responses of emotional processing, by considering inter-individual synchronization in subjective emotional engagement and disengagement. The neural underpinnings of these shared responses are ideally studied in naturalistic scenarios like movie viewing, wherein individuals emotionally engage and disengage at their own time and pace throughout the course of a narrative. Despite the rich data that naturalistic designs can bring to the study, there is a challenge in determining time-resolved behavioral markers of subjective engagement and disengagement and their underlying neural responses. We used a within-subject cross-over design instructing 22 subjects to watch clips of either neutral or sad content while undergoing functional magnetic resonance imaging (fMRI). Participants watched the same movies a second time while continuously annotating the perceived emotional intensity, thus enabling the mapping of brain activity and emotional experience. Our analyses revealed that between-participant similarity in waxing (engagement) and waning (disengagement) of emotional intensity was directly related to the between-participant similarity in spatiotemporal patterns of brain activation during the movie(s). Similar patterns of engagement reflected common activation in the bilateral ventromedial prefrontal cortex, regions often involved in self-referenced evaluation and generation of negative emotions. Similar patterns of disengagement reflected common activation in central executive and default mode network regions often involved in top-down emotion regulation. Together this work helps to better understand cognitive and neural mechanisms underpinning engagement and disengagement from emotionally evocative narratives.
Collapse
Affiliation(s)
- Melanni Nanni-Zepeda
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
- Department of Psychiatry and Psychotherapy, University of Tübingen, Tübingen, Germany
| | - Joseph DeGutis
- Boston Attention and Learning Laboratory, VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Psychiatry, Harvard Medical School, Boston, Massachusetts, USA
| | - Charley Wu
- Human and Machine Cognition Lab, University of Tübingen, Tübingen, Germany
| | - David Rothlein
- Boston Attention and Learning Laboratory, VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Psychiatry, Boston University School of Medicine, Boston, Massachusetts, USA
| | - Yan Fan
- Department Psychology and Neurosciences, Leibniz Research Centre for Working Environment and Human Factors at the TU Dortmund (IfADo), Dortmund, Germany
| | - Simone Grimm
- Berlin Institute of Health, Campus Benjamin Franklin, Charité-Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin, Humboldt-Universität zu Berlin, Berlin, Germany
- Department of Psychiatry, Psychotherapy and Psychosomatics, Psychiatric Hospital, University of Zurich, Zurich, Switzerland
- Department of Psychology, MSB Medical School Berlin, Berlin, Germany
| | - Martin Walter
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
- Department of Psychiatry and Psychotherapy, University of Tübingen, Tübingen, Germany
- Clinical Affective Neuroimaging Laboratory, Otto-von-Guericke-University, Magdeburg, Germany
- Department of Behavioral Neurology, Leibniz Institute for Neurobiology, Magdeburg, Germany
| | - Michael Esterman
- Boston Attention and Learning Laboratory, VA Boston Healthcare System, Boston, Massachusetts, USA
- Department of Psychiatry, Boston University School of Medicine, Boston, Massachusetts, USA
- National Center for PTSD, VA Boston Healthcare System, Boston, Massachusetts, USA
| | - Agnieszka Zuberer
- Department of Psychiatry and Psychotherapy, Jena University Hospital, Jena, Germany
- Department of Psychiatry and Psychotherapy, University of Tübingen, Tübingen, Germany
- Boston Attention and Learning Laboratory, VA Boston Healthcare System, Boston, Massachusetts, USA
| |
Collapse
|
22
|
Wang L, Yang Y, Hu X, Zhao S, Jiang X, Guo L, Han J, Liu T. Frequency-specific functional difference between gyri and sulci in naturalistic paradigm fMRI. Brain Struct Funct 2024; 229:431-442. [PMID: 38193918 DOI: 10.1007/s00429-023-02746-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2023] [Accepted: 12/04/2023] [Indexed: 01/10/2024]
Abstract
Disentangling functional difference between cortical folding patterns of gyri and sulci provides novel insights into the relationship between brain structure and function. Previous studies using resting-state functional magnetic resonance imaging (rsfMRI) have revealed that sulcal signals exhibit stronger high-frequency but weaker low-frequency components compared to gyral ones, suggesting that gyri may serve as functional integration centers while sulci are segregated local processing units. In this study, we utilize naturalistic paradigm fMRI (nfMRI) to explore the functional difference between gyri and sulci as it has proven to record stronger functional integrations compared to rsfMRI. We adopt a convolutional neural network (CNN) to classify gyral and sulcal fMRI signals in the whole brain (the global model) and within functional brain networks (the local models). The frequency-specific difference between gyri and sulci is then inferred from the power spectral density (PSD) profiles of the learned filters in the CNN model. Our experimental results show that nfMRI shows higher gyral-sulcal PSD contrast effect sizes in the global model compared to rsfMRI. In the local models, the effect sizes are either increased or decreased depending on frequency bands and functional complexity of the FBNs. This study highlights the advantages of nfMRI in depicting the functional difference between gyri and sulci, and provides novel insights into unraveling the relationship between brain structure and function.
Collapse
Affiliation(s)
- Liting Wang
- School of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi, China
| | - Yang Yang
- School of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi, China
| | - Xintao Hu
- School of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi, China.
| | - Shijie Zhao
- School of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi, China
| | - Xi Jiang
- School of Life Science and Technology, MOE Key Laboratory for Neuroinformation, University of Electronic Science and Technology of China, Chengdu, China
| | - Lei Guo
- School of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi, China
| | - Junwei Han
- School of Automation, Northwestern Polytechnical University, Xi'an, Shaanxi, China
| | - Tianming Liu
- School of Computing, University of Georgia, Athens, USA
| |
Collapse
|
23
|
Keles U, Dubois J, Le KJM, Tyszka JM, Kahn DA, Reed CM, Chung JM, Mamelak AN, Adolphs R, Rutishauser U. Multimodal single-neuron, intracranial EEG, and fMRI brain responses during movie watching in human patients. Sci Data 2024; 11:214. [PMID: 38365977 PMCID: PMC10873379 DOI: 10.1038/s41597-024-03029-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2023] [Accepted: 01/31/2024] [Indexed: 02/18/2024] Open
Abstract
We present a multimodal dataset of intracranial recordings, fMRI, and eye tracking in 20 participants during movie watching. Recordings consist of single neurons, local field potential, and intracranial EEG activity acquired from depth electrodes targeting the amygdala, hippocampus, and medial frontal cortex implanted for monitoring of epileptic seizures. Participants watched an 8-min long excerpt from the video "Bang! You're Dead" and performed a recognition memory test for movie content. 3 T fMRI activity was recorded prior to surgery in 11 of these participants while performing the same task. This NWB- and BIDS-formatted dataset includes spike times, field potential activity, behavior, eye tracking, electrode locations, demographics, and functional and structural MRI scans. For technical validation, we provide signal quality metrics, assess eye tracking quality, behavior, the tuning of cells and high-frequency broadband power field potentials to familiarity and event boundaries, and show brain-wide inter-subject correlations for fMRI. This dataset will facilitate the investigation of brain activity during movie watching, recognition memory, and the neural basis of the fMRI-BOLD signal.
Collapse
Affiliation(s)
- Umit Keles
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Julien Dubois
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Kevin J M Le
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA
| | - J Michael Tyszka
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - David A Kahn
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA
| | - Chrystal M Reed
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Jeffrey M Chung
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Adam N Mamelak
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA
| | - Ralph Adolphs
- Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, CA, USA.
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
| | - Ueli Rutishauser
- Department of Neurosurgery, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Division of Biology and Biological Engineering, California Institute of Technology, Pasadena, CA, USA.
- Department of Neurology, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
- Center for Neural Science and Medicine, Department of Biomedical Sciences, Cedars-Sinai Medical Center, Los Angeles, CA, USA.
| |
Collapse
|
24
|
Yu J. Age-related decrease in inter-subject similarity of cortical morphology and task and resting-state functional connectivity. GeroScience 2024; 46:697-711. [PMID: 38006514 PMCID: PMC10828367 DOI: 10.1007/s11357-023-01008-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2023] [Accepted: 11/02/2023] [Indexed: 11/27/2023] Open
Abstract
"All old people are the same" is an unfortunate characterization of the perceived homogeneity in the older age group. This study attempts to debunk this myth in the context of the structural and functional brain. Within older relative to younger age groups, individuals are hypothesized to be more dissimilar to their similar-aged peers-thus demonstrating an age-related divergence. This study analyzed functional connectivity (FC) during multiple fMRI paradigms (2 rest + 5 tasks) and cortical thickness (CT) data from two lifespan datasets (Ntotal = 1161). On average, between-subject FC/CT correlations became weaker in the older age groups. Further analyses ruled out the possibility that more rapid age-related changes in older brains have increased the dissimilarity in these older age groups. Brain-wide analyses revealed significant effects of age-related divergence across most of the brain. Finally, CT similarity between a dyad significantly predicted their FC similarity across multiple fMRI task paradigms-demonstrating a close relationship between brain structure and function even at the between-dyad level. Contrary to the myth that "all old people are the same," these findings suggest young people are more similar to each other. This study presents major implications in the study of neural fingerprinting and brain-behavior associations.
Collapse
Affiliation(s)
- Junhong Yu
- Psychology, School of Social Sciences, Nanyang Technological University, 48 Nanyang Avenue, Singapore, 639798, Singapore.
| |
Collapse
|
25
|
Omigie D, Mencke I. A model of time-varying music engagement. Philos Trans R Soc Lond B Biol Sci 2024; 379:20220421. [PMID: 38104598 PMCID: PMC10725767 DOI: 10.1098/rstb.2022.0421] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Accepted: 11/13/2023] [Indexed: 12/19/2023] Open
Abstract
The current paper offers a model of time-varying music engagement, defined as changes in curiosity, attention and positive valence, as music unfolds over time. First, we present research (including new data) showing that listeners tend to allocate attention to music in a manner that is guided by both features of the music and listeners' individual differences. Next, we review relevant predictive processing literature before using this body of work to inform our model. In brief, we propose that music engagement, over the course of an extended listening episode, may constitute several cycles of curiosity, attention and positive valence that are interspersed with moments of mind-wandering. Further, we suggest that refocusing on music after an episode of mind-wandering can be due to triggers in the music or, conversely, mental action that occurs when the listener realizes they are mind-wandering. Finally, we argue that factors that modulate both overall levels of music engagement and how it changes over time include music complexity, listener background and the listening context. Our paper highlights how music can be used to provide insights into the temporal dynamics of attention and into how curiosity might emerge in everyday contexts. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.
Collapse
Affiliation(s)
- Diana Omigie
- Department of Psychology, Goldsmiths University of London, London, SE14 6NW, UK
| | - Iris Mencke
- Music Perception and Processing Lab, Department of Medical Physics and Acoustics, University of Oldenburg, 26129 Oldenberg, Germany
- Hanse-Wissenschaftskolleg—Institute for Advanced Studies, 27753 Delmenhorst, Germany
- Department of Music, Max Planck Institute for Empirical Aesthetics, Frankfurt/Main 60322, Germany
| |
Collapse
|
26
|
Liu Q, Cui H, Huang B, Huang Y, Sun H, Ru X, Zhang M, Chen W. Inter-brain neural mechanism and influencing factors underlying different cooperative behaviors: a hyperscanning study. Brain Struct Funct 2024; 229:75-95. [PMID: 37899406 DOI: 10.1007/s00429-023-02700-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/17/2023] [Accepted: 08/21/2023] [Indexed: 10/31/2023]
Abstract
Cooperative behavior is a vital social interaction which plays a vital role in improving human survival and reproduction. However, few empirical studies have examined the differences between cooperative behaviors and the underlying neural substrates. In the present study, the brain activity of familiar dyads of the same sex was measured using functional near-infrared spectroscopy during three cooperative tasks (cooperative button-press, tangram, and Jenga tasks). We also measured the dyads' empathic abilities and personality traits to investigate the relationships between individual characteristics and neural markers. The results showed that first, there were significant differences in intra-brain activation and inter-brain synchronization among different cooperative tasks in three dimensions: social cognition, behavioral response, and cognitive processing. Second, male participants require stronger intra-brain activation to achieve the same inter-brain synchronization level as women in cooperative tasks. Third, when performing cooperative tasks involving high cognitive demands, Big Five Neuroticism may be an important predictor of neural activation in female participants. Inter-brain synchronization plays an important role in the frontal and temporoparietal junctions during interpersonal cooperation. Furthermore, this study demonstrates that mutual prediction theory is crucial for understanding the neural mechanisms of cooperative behavior.
Collapse
Affiliation(s)
- Qingming Liu
- Center for Brain, Mind and Education, Shaoxing University, Shaoxing, China
- Department of Psychology, Shaoxing University, Shaoxing, China
| | - Huimin Cui
- Department of Psychology, Shaoxing University, Shaoxing, China
| | - Bincan Huang
- Department of Primary Education, Shaoxing University, Shaoxing, China
| | - Yingying Huang
- Department of Primary Education, Shaoxing University, Shaoxing, China
| | - Huimeng Sun
- Department of Primary Education, Shaoxing University, Shaoxing, China
| | - Xinyi Ru
- Department of Primary Education, Shaoxing University, Shaoxing, China
| | - Mingming Zhang
- Department of Psychology, College of Education, Shanghai Normal University, Shanghai, China.
| | - Wei Chen
- Center for Brain, Mind and Education, Shaoxing University, Shaoxing, China.
- Department of Psychology, Shaoxing University, Shaoxing, China.
- Interdisciplinary Center for Philosophy and Cognitive Sciences, Renmin University of China, Beijing, China.
| |
Collapse
|
27
|
Chen C, Dupré la Tour T, Gallant JL, Klein D, Deniz F. The Cortical Representation of Language Timescales is Shared between Reading and Listening. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.01.06.522601. [PMID: 37577530 PMCID: PMC10418083 DOI: 10.1101/2023.01.06.522601] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/15/2023]
Abstract
Language comprehension involves integrating low-level sensory inputs into a hierarchy of increasingly high-level features. Prior work studied brain representations of different levels of the language hierarchy, but has not determined whether these brain representations are shared between written and spoken language. To address this issue, we analyzed fMRI BOLD data recorded while participants read and listened to the same narratives in each modality. Levels of the language hierarchy were operationalized as timescales, where each timescale refers to a set of spectral components of a language stimulus. Voxelwise encoding models were used to determine where different timescales are represented across the cerebral cortex, for each modality separately. These models reveal that between the two modalities timescale representations are organized similarly across the cortical surface. Our results suggest that, after low-level sensory processing, language integration proceeds similarly regardless of stimulus modality.
Collapse
Affiliation(s)
- Catherine Chen
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, USA
| | - Tom Dupré la Tour
- Helen Wills Neuroscience Institute, University of California, Berkeley, USA
| | - Jack L. Gallant
- Helen Wills Neuroscience Institute, University of California, Berkeley, USA
| | - Dan Klein
- Department of Electrical Engineering and Computer Sciences, University of California, Berkeley, USA
| | - Fatma Deniz
- Helen Wills Neuroscience Institute, University of California, Berkeley, USA
- Institute of Software Engineering and Theoretical Computer Science, Technische Universität Berlin, Berlin, Germany
- Bernstein Center for Computational Neuroscience, Berlin, Germany
| |
Collapse
|
28
|
Vallet W, van Wassenhove V. Can cognitive neuroscience solve the lab-dilemma by going wild? Neurosci Biobehav Rev 2023; 155:105463. [PMID: 37967734 DOI: 10.1016/j.neubiorev.2023.105463] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/01/2023] [Revised: 10/18/2023] [Accepted: 11/08/2023] [Indexed: 11/17/2023]
Abstract
Reproducibility, measurability, and refutability are the foundation of the scientific method applied to empirical work. In the study of animal and human behavior, experimental protocols conducted in the lab are the most reliable means by which scientists can operationalize behaviors using controlled and parameterized setups. However, whether observations in the lab fully generalize in the real world remain legitimately disputed. The notion of "experimental design" was originally intended to ensure the generalizability of experimental findings to real-world situations. Experiments in the wild are more frequently explored and significant technological advances have been made allowing mobile neuroimaging. Yet some methodological limitations remain when testing scientific hypotheses in ecological conditions. Herein, we discuss the limitations of inferential processes derive from empirical observations in the wild. The multi-causal property of an ecological situation often lacks controls, and this major concern may prevent the replication and the reliability of behavioral observations. We discuss the epistemological and historical grounds of the induction process for behavioral and cognitive neurosciences and provide some possible heuristics for In situ experimental designs compatible with psychophysics in the wild.
Collapse
Affiliation(s)
- William Vallet
- CEA DRF/Joliot, NeuroSpin, INSERM, Cognitive Neuroimaging Unit, Université Paris Saclay, 91191 Gif-sur-Yvette, France; INSERM U1028, CNRS UMR 5292, PSYR2 Team, Centre de recherche en Neurosciences de Lyon (CRNL), Université Lyon 1, 69000 Lyon, France.
| | - Virginie van Wassenhove
- CEA DRF/Joliot, NeuroSpin, INSERM, Cognitive Neuroimaging Unit, Université Paris Saclay, 91191 Gif-sur-Yvette, France
| |
Collapse
|
29
|
Jangraw DC, Finn ES, Bandettini PA, Landi N, Sun H, Hoeft F, Chen G, Pugh KR, Molfese PJ. Inter-subject correlation during long narratives reveals widespread neural correlates of reading ability. Neuroimage 2023; 282:120390. [PMID: 37751811 PMCID: PMC10783814 DOI: 10.1016/j.neuroimage.2023.120390] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Revised: 09/19/2023] [Accepted: 09/23/2023] [Indexed: 09/28/2023] Open
Abstract
Recent work using fMRI inter-subject correlation analysis has provided new information about the brain's response to video and audio narratives, particularly in frontal regions not typically activated by single words. This approach is very well suited to the study of reading, where narrative is central to natural experience. But since past reading paradigms have primarily presented single words or phrases, the influence of narrative on semantic processing in the brain - and how that influence might change with reading ability - remains largely unexplored. In this study, we presented coherent stories to adolescents and young adults with a wide range of reading abilities. The stories were presented in alternating visual and auditory blocks. We used a dimensional inter-subject correlation analysis to identify regions in which better and worse readers had varying levels of consistency with other readers. This analysis identified a widespread set of brain regions in which activity timecourses were more similar among better readers than among worse readers. These differences were not detected with standard block activation analyses. Worse readers had higher correlation with better readers than with other worse readers, suggesting that the worse readers had "idiosyncratic" responses rather than using a single compensatory mechanism. Close inspection confirmed that these differences were not explained by differences in IQ or motion. These results suggest an expansion of the current view of where and how reading ability is reflected in the brain, and in doing so, they establish inter-subject correlation as a sensitive tool for future studies of reading disorders.
Collapse
Affiliation(s)
- David C Jangraw
- Section on Functional Imaging Methods, NIMH, Bethesda, MD, United States; Emotion and Development Branch, NIMH, Bethesda, MD, United States; Department of Electrical and Biomedical Engineering, University of Vermont, Burlington, VT, United States.
| | - Emily S Finn
- Section on Functional Imaging Methods, NIMH, Bethesda, MD, United States; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, United States
| | - Peter A Bandettini
- Section on Functional Imaging Methods, NIMH, Bethesda, MD, United States; Center for Multimodal Neuroimaging, NIMH, Bethesda, MD, United States
| | - Nicole Landi
- Haskins Laboratories, New Haven, CT, United States
| | - Haorui Sun
- Department of Electrical and Biomedical Engineering, University of Vermont, Burlington, VT, United States
| | - Fumiko Hoeft
- Haskins Laboratories, New Haven, CT, United States; Department of Psychological Sciences, University of Connecticut, Hartford, CT, United States
| | - Gang Chen
- Statistical Computing Core, NIMH, Bethesda, MD, United States
| | - Kenneth R Pugh
- Haskins Laboratories, New Haven, CT, United States; Department of Pediatrics, Yale University School of Medicine, New Haven, CT, United States
| | - Peter J Molfese
- Center for Multimodal Neuroimaging, NIMH, Bethesda, MD, United States; Haskins Laboratories, New Haven, CT, United States
| |
Collapse
|
30
|
Pan H, Chen Z, Jospe K, Gao Q, Sheng J, Gao Z, Perry A. Mood congruency affects physiological synchrony but not empathic accuracy in a naturalistic empathy task. Biol Psychol 2023; 184:108720. [PMID: 37952694 DOI: 10.1016/j.biopsycho.2023.108720] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 11/05/2023] [Accepted: 11/08/2023] [Indexed: 11/14/2023]
Abstract
Empathy is a crucial aspect of our daily lives, as it enhances our wellbeing and is a proxy for prosocial behavior. It encompasses two related but partially distinct components: cognitive and affective empathy. Both are susceptible to context, biases and an individual's physiological state. Few studies have explored the effects of a person's mood on these empathy components, and results are mixed. The current study takes advantage of an ecological, naturalistic empathy task - the empathic accuracy (EA) task - in combination with physiological measurements to examine and differentiate between the effects of one's mood on both empathy components. Participants were induced with positive or negative mood and presented videos of targets narrating autobiographical negative stories, selected from a Chinese empathy dataset that we developed (now publicly available). The stories were conveyed in audio-only, visual-only and full-video formats. Participants rated the target's emotional state while watching or listening to their stories, and physiological measures were taken throughout the process. Importantly, similar measures were taken from the targets when they narrated the stories, allowing a comparison between participants' and targets' measures. We found that in audio-only and visual-only conditions, participants whose moods were congruent with the target showed higher physiological synchrony than those with incongruent mood, implying a mood-congruency effect on affective empathy. However, there was no mood effect on empathic accuracy (reflecting cognitive empathy), suggesting a different influence of mood on the two empathy components.
Collapse
Affiliation(s)
- Hanxi Pan
- Department of Psychology and Behavioral Sciences, Zhejiang University, China
| | - Zhiyun Chen
- Department of Psychology and Behavioral Sciences, Zhejiang University, China
| | - Karine Jospe
- Department of Psychology, The Hebrew University of Jerusalem, Israel; Department of Psychology, Tel-Aviv University, Israel
| | - Qi Gao
- Department of Psychology and Behavioral Sciences, Zhejiang University, China.
| | - Jinyou Sheng
- Department of Psychology and Behavioral Sciences, Zhejiang University, China
| | - Zaifeng Gao
- Department of Psychology and Behavioral Sciences, Zhejiang University, China.
| | - Anat Perry
- Department of Psychology, The Hebrew University of Jerusalem, Israel
| |
Collapse
|
31
|
Tikka P, Kaipainen M, Salmi J. Narrative simulation of social experiences in naturalistic context - A neurocinematic approach. Neuropsychologia 2023; 188:108654. [PMID: 37507066 DOI: 10.1016/j.neuropsychologia.2023.108654] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2022] [Revised: 07/02/2023] [Accepted: 07/25/2023] [Indexed: 07/30/2023]
Abstract
Narratives may be regarded as simulations of everyday social situations. They are key to studying the human mind in socio-culturally determined contexts as they allow anchoring to the common ground of embodied and environmentally-engaged cognition. Here we review recent findings from naturalistic neuroscience on neural functions in conditions that mimic lifelike situations. We will focus particularly on neurocinematics, a research field that applies mediated narratives as stimuli for neuroimaging experiments. During the last two decades, this paradigm has contributed to an accumulation of insights about the neural underpinnings of behavior and sense-making in various narratively contextualized situations particularly pertaining to socio-emotional encounters. One of the key questions in neurocinematics is, how do intersubjectively synchronized brain activations relate to subjective experiences? Another question we address is how to bring natural contexts into experimental studies. Seeking to respond to both questions, we suggest neurocinematic studies to examine three manifestations of the same phenomenon side-by-side: subjective experiences of narrative situations, unfolding of narrative stimulus structure, and neural processes that co-constitute the experience. This approach facilitates identifying experientially meaningful activity patterns in the brain and points out what they may mean in relation to shared and communicable contents. Via rich-featured and temporally contextualized narrative stimuli, neurocinematics attempts to contribute to emerging holistic theories of neural dynamics and connectomics explaining typical and atypical interindividual variability.
Collapse
Affiliation(s)
- Pia Tikka
- Enactive Virtuality Lab, Baltic School of Film, Media and Arts, Tallinn University, Estonia.
| | | | - Juha Salmi
- Translational Cognitive Neuroscience Lab, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| |
Collapse
|
32
|
Wang Z, Yang J, Zheng Z, Cao W, Dong L, Li H, Wen X, Luo C, Cai Q, Jian W, Yao D. Trait- and State-Dependent Changes in Cortical-Subcortical Functional Networks Across the Adult Lifespan. J Magn Reson Imaging 2023; 58:720-731. [PMID: 36637029 DOI: 10.1002/jmri.28599] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2022] [Revised: 12/29/2022] [Accepted: 01/03/2023] [Indexed: 01/14/2023] Open
Abstract
BACKGROUND How the functional interactions of the basal ganglia/thalamus with the cerebral cortex and the cerebellum change over the adult lifespan in movie-watching and resting-state is less clear. PURPOSE To investigate the functional changes in the organization of the human cortical-subcortical functional networks over the adult lifespan using movie-watching and resting-state fMRI data. STUDY TYPE Cohort. SUBJECTS Healthy 467 adults (cross-sectional individuals aged 18-88 years) from the Cambridge Centre for Ageing and Neuroscience (www.cam-can.com). FIELD STRENGTH/SEQUENCE: fMRI using a gradient-echo echo-planar imaging (EPI) sequence at 3 T. ASSESSMENT Functional connectivities (FCs) of the subcortical subregions (i.e. the basal ganglia and thalamus) with both the cerebral cortex and cerebellum were examined in fMRI data acquired during resting state and movie-watching. And, fluid intelligence scores were also assessed. STATISTICAL TESTS Student's t-tests, false discovery rate (FDR) corrected. RESULTS As age increased, FCs that mainly within the basal ganglia and thalamus, and between the basal ganglia/thalamus and cortical networks (including the dorsal attention, ventral attention, and limbic networks) were both increased/decreased during movie-watching and resting states. However, FCs showed a state-dependent component with advancing age. During the movie-watching state, the FCs between the basal ganglia/thalamus and cerebellum/frontoparietal control networks were mainly increased with age, and the FCs in the somatomotor network were decreased with age. During the resting state, the FCs between the basal ganglia/thalamus and default mode/visual networks were mainly increased with age, and the FCs in the cerebellum were mainly decreased with age. Moreover, inverse relationships between FCs and fluid intelligence were mainly found in these network regions. DATA CONCLUSION Our study may suggest that changes in cortical-subcortical functional networks across the adult lifespan were both state-dependent and stable traits, and that aging fMRI studies should consider the effects of both physiological characteristics and individual situations. EVIDENCE LEVEL 2. TECHNICAL EFFICACY Stage 3.
Collapse
Affiliation(s)
- Ziqi Wang
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Jie Yang
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Zihao Zheng
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Weifang Cao
- Department of Radiology, Shandong First Medical University & Shandong Academy of Medical Sciences, Tai'an, China
| | - Li Dong
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
- Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu, China
- School of Electrical Engineering, Zhengzhou University, Zhengzhou, China
- Sichuan Institute for Brain Science and Brain-Inspired Intelligence, Chengdu, China
| | - Hechun Li
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Xin Wen
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Cheng Luo
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
- Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu, China
| | - Qingyan Cai
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Wei Jian
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
| | - Dezhong Yao
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China
- Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu, China
- School of Electrical Engineering, Zhengzhou University, Zhengzhou, China
- Sichuan Institute for Brain Science and Brain-Inspired Intelligence, Chengdu, China
| |
Collapse
|
33
|
Sheng J, Wang S, Zhang L, Liu C, Shi L, Zhou Y, Hu H, Chen C, Xue G. Intersubject similarity in neural representations underlies shared episodic memory content. Proc Natl Acad Sci U S A 2023; 120:e2308951120. [PMID: 37603733 PMCID: PMC10466090 DOI: 10.1073/pnas.2308951120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2023] [Accepted: 07/05/2023] [Indexed: 08/23/2023] Open
Abstract
Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.
Collapse
Affiliation(s)
- Jintao Sheng
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Sisi Wang
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Liang Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Chuqi Liu
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Liang Shi
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Yu Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Huinan Hu
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Chuansheng Chen
- Department of Psychological Science, University of California, Irvine, CA92697
| | - Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing102206, China
| |
Collapse
|
34
|
Mizrahi T, Axelrod V. Naturalistic auditory stimuli with fNIRS prefrontal cortex imaging: A potential paradigm for disorder of consciousness diagnostics (a study with healthy participants). Neuropsychologia 2023; 187:108604. [PMID: 37271305 DOI: 10.1016/j.neuropsychologia.2023.108604] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2023] [Revised: 06/01/2023] [Accepted: 06/02/2023] [Indexed: 06/06/2023]
Abstract
Disorder of consciousness (DOC) is a devastating condition due to brain damage. A patient in this condition is non-responsive, but nevertheless might be conscious at least at some level. Determining the conscious level of DOC patients is important for both medical and ethical reasons, but reliably achieving this has been a major challenge. Naturalistic stimuli in combination with neuroimaging have been proposed as a promising approach for DOC patient diagnosis. Capitalizing on and extending this proposal, the goal of the present study conducted with healthy participants was to develop a new paradigm with naturalistic auditory stimuli and functional near-infrared spectroscopy (fNIRS) - an approach that can be used at the bedside. Twenty-four healthy participants passively listened to 9 min of auditory story, scrambled auditory story, classical music, and scrambled classical music segments while their prefrontal cortex activity was recorded using fNIRS. We found much higher intersubject correlation (ISC) during story compared to scrambled story conditions both at the group level and in the majority of individual subjects, suggesting that fNIRS imaging of the prefrontal cortex might be a sensitive method to capture neural changes associated with narrative comprehension. In contrast, the ISC during the classical music segment did not differ reliably from scrambled classical music and was also much lower than the story condition. Our main result is that naturalistic auditory stories with fNIRS might be used in a clinical setup to identify high-level processing and potential consciousness in DOC patients.
Collapse
Affiliation(s)
- Tamar Mizrahi
- The Gonda Multidisciplinary Brain Research Center, Bar Ilan University, Ramat Gan, Israel; Head Injuries Rehabilitation Department, Sheba Medical Center, Ramat Gan, Israel
| | - Vadim Axelrod
- The Gonda Multidisciplinary Brain Research Center, Bar Ilan University, Ramat Gan, Israel.
| |
Collapse
|
35
|
Li X, Friedrich P, Patil KR, Eickhoff SB, Weis S. A topography-based predictive framework for naturalistic viewing fMRI. Neuroimage 2023:120245. [PMID: 37353099 DOI: 10.1016/j.neuroimage.2023.120245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 05/26/2023] [Accepted: 06/20/2023] [Indexed: 06/25/2023] Open
Abstract
Functional magnetic resonance imaging (fMRI) during naturalistic viewing (NV) provides exciting opportunities for studying brain functions in more ecologically valid settings. Understanding individual differences in brain functions during NV and their behavioural relevance has recently become an important goal. However, methods specifically designed for this purpose remain limited. Here, we propose a topography-based predictive framework (TOPF) to fill this methodological gap. TOPF identifies individual-specific evoked activity topographies in a data-driven manner and examines their behavioural relevance using a machine learning-based predictive framework. We validate TOPF on both NV and task-based fMRI data from multiple conditions. Our results show that TOPF effectively and stably captures individual differences in evoked brain activity and successfully predicts phenotypes across cognition, emotion and personality on unseen subjects from their activity topographies. Moreover, TOPF compares favourably with functional connectivity-based approaches in prediction performance, with the identified predictive brain regions being neurobiologically interpretable. Crucially, we highlight the importance of examining individual evoked brain activity topographies in advancing our understanding of the brain-behaviour relationship. We believe that the TOPF approach provides a simple but powerful tool for understanding brain-behaviour relationships on an individual level with a strong potential for clinical applications.
Collapse
Affiliation(s)
- Xuan Li
- Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research Centre Jülich, 52425 Jülich, Germany;; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany.
| | - Patrick Friedrich
- Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research Centre Jülich, 52425 Jülich, Germany;; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
| | - Kaustubh R Patil
- Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research Centre Jülich, 52425 Jülich, Germany;; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
| | - Simon B Eickhoff
- Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research Centre Jülich, 52425 Jülich, Germany;; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
| | - Susanne Weis
- Institute of Neuroscience and Medicine (INM-7: Brain and Behaviour), Research Centre Jülich, 52425 Jülich, Germany;; Institute of Systems Neuroscience, Heinrich Heine University Düsseldorf, 40225 Düsseldorf, Germany
| |
Collapse
|
36
|
Mahrukh R, Shakil S, Malik AS. Sentiments analysis of fMRI using automatically generated stimuli labels under naturalistic paradigm. Sci Rep 2023; 13:7267. [PMID: 37142654 PMCID: PMC10160115 DOI: 10.1038/s41598-023-33734-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 04/18/2023] [Indexed: 05/06/2023] Open
Abstract
Our emotions and sentiments are influenced by naturalistic stimuli such as the movies we watch and the songs we listen to, accompanied by changes in our brain activation. Comprehension of these brain-activation dynamics can assist in identification of any associated neurological condition such as stress and depression, leading towards making informed decision about suitable stimuli. A large number of open-access functional magnetic resonance imaging (fMRI) datasets collected under naturalistic conditions can be used for classification/prediction studies. However, these datasets do not provide emotion/sentiment labels, which limits their use in supervised learning studies. Manual labeling by subjects can generate these labels, however, this method is subjective and biased. In this study, we are proposing another approach of generating automatic labels from the naturalistic stimulus itself. We are using sentiment analyzers (VADER, TextBlob, and Flair) from natural language processing to generate labels using movie subtitles. Subtitles generated labels are used as the class labels for positive, negative, and neutral sentiments for classification of brain fMRI images. Support vector machine, random forest, decision tree, and deep neural network classifiers are used. We are getting reasonably good classification accuracy (42-84%) for imbalanced data, which is increased (55-99%) for balanced data.
Collapse
Affiliation(s)
| | - Sadia Shakil
- Institute of Space Technology, Islamabad, Pakistan.
- Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic.
- Turner Institute for Brain and Mental Health, Monash University, Melbourne, Australia.
| | - Aamir Saeed Malik
- Faculty of Information Technology, Brno University of Technology, Brno, Czech Republic.
| |
Collapse
|
37
|
Putkinen V, Nazari-Farsani S, Karjalainen T, Santavirta S, Hudson M, Seppälä K, Sun L, Karlsson HK, Hirvonen J, Nummenmaa L. Pattern recognition reveals sex-dependent neural substrates of sexual perception. Hum Brain Mapp 2023; 44:2543-2556. [PMID: 36773282 PMCID: PMC10028630 DOI: 10.1002/hbm.26229] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2022] [Revised: 12/13/2022] [Accepted: 01/16/2023] [Indexed: 02/12/2023] Open
Abstract
Sex differences in brain activity evoked by sexual stimuli remain elusive despite robust evidence for stronger enjoyment of and interest toward sexual stimuli in men than in women. To test whether visual sexual stimuli evoke different brain activity patterns in men and women, we measured hemodynamic brain activity induced by visual sexual stimuli in two experiments with 91 subjects (46 males). In one experiment, the subjects viewed sexual and nonsexual film clips, and dynamic annotations for nudity in the clips were used to predict hemodynamic activity. In the second experiment, the subjects viewed sexual and nonsexual pictures in an event-related design. Men showed stronger activation than women in the visual and prefrontal cortices and dorsal attention network in both experiments. Furthermore, using multivariate pattern classification we could accurately predict the sex of the subject on the basis of the brain activity elicited by the sexual stimuli. The classification generalized across the experiments indicating that the sex differences were task-independent. Eye tracking data obtained from an independent sample of subjects (N = 110) showed that men looked longer than women at the chest area of the nude female actors in the film clips. These results indicate that visual sexual stimuli evoke discernible brain activity patterns in men and women which may reflect stronger attentional engagement with sexual stimuli in men.
Collapse
Affiliation(s)
- Vesa Putkinen
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
| | - Sanaz Nazari-Farsani
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
| | - Tomi Karjalainen
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
| | - Severi Santavirta
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
| | - Matthew Hudson
- Turku PET Centre, University of Turku, Turku, Finland
- School of Psychology, University of Plymouth, Plymouth, UK
| | - Kerttu Seppälä
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
- Department of Medical Physics, Turku University Hospital, Turku, Finland
| | - Lihua Sun
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
| | - Henry K Karlsson
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
| | - Jussi Hirvonen
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
- Department of Radiology, Turku University Hospital, Turku, Finland
| | - Lauri Nummenmaa
- Turku PET Centre, University of Turku, Turku, Finland
- Turku University Hospital, Turku, Finland
- Department of Psychology, University of Turku, Turku, Finland
| |
Collapse
|
38
|
Kröll JP, Friedrich P, Li X, Patil KR, Mochalski L, Waite L, Qian X, Chee MW, Zhou JH, Eickhoff S, Weis S. Naturalistic viewing increases individual identifiability based on connectivity within functional brain networks. Neuroimage 2023; 273:120083. [PMID: 37015270 DOI: 10.1016/j.neuroimage.2023.120083] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2022] [Revised: 03/07/2023] [Accepted: 03/31/2023] [Indexed: 04/06/2023] Open
Abstract
Naturalistic viewing (NV) is currently considered a promising paradigm for studying individual differences in functional brain organization. While whole brain functional connectivity (FC) under NV has been relatively well characterized, so far little work has been done on a network level. Here, we extend current knowledge by characterizing the influence of NV on FC in fourteen meta-analytically derived brain networks considering three different movie stimuli in comparison to resting-state (RS). We show that NV increases identifiability of individuals over RS based on functional connectivity in certain, but not all networks. Furthermore, movie stimuli including a narrative appear more distinct from RS. In addition, we assess individual variability in network FC by comparing within- and between-subject similarity during NV and RS. We show that NV can evoke individually distinct NFC patterns by increasing inter-subject variability while retaining within-subject similarity. Crucially, our results highlight that this effect is not observable across all networks, but rather dependent on the network-stimulus combination. Our results confirm that NV can improve the detection of individual differences over RS and underline the importance of selecting the appropriate combination of movie and cognitive network for the research question at hand.
Collapse
Affiliation(s)
- Jean-Philippe Kröll
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Patrick Friedrich
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Xuan Li
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Kaustubh R Patil
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Lisa Mochalski
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Laura Waite
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany
| | - Xing Qian
- Centre for Sleep and Cognition & Centre for Translational MR Research, Yong Loo Lin School of Medicine, National University of Singapore
| | - Michael Wl Chee
- Centre for Sleep and Cognition & Centre for Translational MR Research, Yong Loo Lin School of Medicine, National University of Singapore; Integrative Sciences and Engineering Programme (ISEP), National University of Singapore, Singapore, Singapore
| | - Juan Helen Zhou
- Centre for Sleep and Cognition & Centre for Translational MR Research, Yong Loo Lin School of Medicine, National University of Singapore; Department of Electrical and Computer Engineering, National University of Singapore, Singapore, Singapore; Integrative Sciences and Engineering Programme (ISEP), National University of Singapore, Singapore, Singapore
| | - Simon Eickhoff
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| | - Susanne Weis
- Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Centre Jülich, Jülich 52428, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf 40225, Germany
| |
Collapse
|
39
|
Addeh A, Vega F, Medi PR, Williams RJ, Pike GB, MacDonald ME. Direct machine learning reconstruction of respiratory variation waveforms from resting state fMRI data in a pediatric population. Neuroimage 2023; 269:119904. [PMID: 36709788 DOI: 10.1016/j.neuroimage.2023.119904] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2022] [Revised: 01/20/2023] [Accepted: 01/25/2023] [Indexed: 01/27/2023] Open
Abstract
In many functional magnetic resonance imaging (fMRI) studies, respiratory signals are unavailable or do not have acceptable quality due to issues with subject compliance, equipment failure or signal error. In large databases, such as the Human Connectome Projects, over half of the respiratory recordings may be unusable. As a result, the direct removal of low frequency respiratory variations from the blood oxygen level-dependent (BOLD) signal time series is not possible. This study proposes a deep learning-based method for reconstruction of respiratory variation (RV) waveforms directly from BOLD fMRI data in pediatric participants (aged 5 to 21 years old), and does not require any respiratory measurement device. To do this, the Lifespan Human Connectome Project in Development (HCP-D) dataset, which includes respiratory measurements, was used to both train a convolutional neural network (CNN) and evaluate its performance. Results show that a CNN can capture informative features from the BOLD signal time course and reconstruct accurate RV timeseries, especially when the subject has a prominent respiratory event. This work advances the use of direct estimation of physiological parameters from fMRI, which will eventually lead to reduced complexity and decrease the burden on participants because they may not be required to wear a respiratory bellows.
Collapse
Affiliation(s)
- Abdoljalil Addeh
- Department of Biomedical Engineering, Schulich School of Engineering, University of Calgary, Canada; Department of Electrical & Software Engineering, Schulich School of Engineering, University of Calgary, Canada; Department of Radiology, Cumming School of Medicine, University of Calgary, Canada; Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Canada
| | - Fernando Vega
- Department of Biomedical Engineering, Schulich School of Engineering, University of Calgary, Canada; Department of Electrical & Software Engineering, Schulich School of Engineering, University of Calgary, Canada; Department of Radiology, Cumming School of Medicine, University of Calgary, Canada; Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Canada
| | - Prathistith Raj Medi
- Data Science and Artificial Intelligence, International Institute of Information Technology, Naya Raipur, India
| | - Rebecca J Williams
- Department of Radiology, Cumming School of Medicine, University of Calgary, Canada; Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Canada; Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Canada
| | - G Bruce Pike
- Department of Radiology, Cumming School of Medicine, University of Calgary, Canada; Department of Clinical Neurosciences, Cumming School of Medicine, University of Calgary, Canada; Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Canada
| | - M Ethan MacDonald
- Department of Biomedical Engineering, Schulich School of Engineering, University of Calgary, Canada; Department of Electrical & Software Engineering, Schulich School of Engineering, University of Calgary, Canada; Department of Radiology, Cumming School of Medicine, University of Calgary, Canada; Hotchkiss Brain Institute, Cumming School of Medicine, University of Calgary, Canada
| |
Collapse
|
40
|
Hudson M, Santavirta S, Putkinen V, Seppälä K, Sun L, Karjalainen T, Karlsson HK, Hirvonen J, Nummenmaa L. Neural responses to biological motion distinguish autistic and schizotypal traits. Soc Cogn Affect Neurosci 2023; 18:nsad011. [PMID: 36847146 PMCID: PMC10032360 DOI: 10.1093/scan/nsad011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2022] [Revised: 01/26/2023] [Accepted: 02/24/2023] [Indexed: 03/01/2023] Open
Abstract
Difficulties in social interactions characterize both autism and schizophrenia and are correlated in the neurotypical population. It is unknown whether this represents a shared etiology or superficial phenotypic overlap. Both conditions exhibit atypical neural activity in response to the perception of social stimuli and decreased neural synchronization between individuals. This study investigated if neural activity and neural synchronization associated with biological motion perception are differentially associated with autistic and schizotypal traits in the neurotypical population. Participants viewed naturalistic social interactions while hemodynamic brain activity was measured with fMRI, which was modeled against a continuous measure of the extent of biological motion. General linear model analysis revealed that biological motion perception was associated with neural activity across the action observation network. However, intersubject phase synchronization analysis revealed neural activity to be synchronized between individuals in occipital and parietal areas but desynchronized in temporal and frontal regions. Autistic traits were associated with decreased neural activity (precuneus and middle cingulate gyrus), and schizotypal traits were associated with decreased neural synchronization (middle and inferior frontal gyri). Biological motion perception elicits divergent patterns of neural activity and synchronization, which dissociate autistic and schizotypal traits in the general population, suggesting that they originate from different neural mechanisms.
Collapse
Affiliation(s)
- Matthew Hudson
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Turku University Hospital, Turku 20520, Finland
- School of Psychology, University of Plymouth, Plymouth PL4 8AA, UK
- Brain Research & Imaging Centre, Faculty of Health, University of Plymouth, Research Way, Plymouth PL6 8BU, UK
| | - Severi Santavirta
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Turku University Hospital, Turku 20520, Finland
| | - Vesa Putkinen
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Turku University Hospital, Turku 20520, Finland
| | - Kerttu Seppälä
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Turku University Hospital, Turku 20520, Finland
- Department of Medical Physics, Turku University Hospital, Turku 20520, Finland
| | - Lihua Sun
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Turku University Hospital, Turku 20520, Finland
- Department of Nuclear Medicine, Huashan Hospital, Fudan University, Shanghai 200040, China
| | - Tomi Karjalainen
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Turku University Hospital, Turku 20520, Finland
| | - Henry K Karlsson
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Turku University Hospital, Turku 20520, Finland
| | - Jussi Hirvonen
- Department of Radiology, University of Turku and Turku University Hospital, Turku 20520, Finland
- Medical Imaging Centre, Department of Radiology, Tampere University and Tampere University Hospital, Tampere 33100, Finland
| | - Lauri Nummenmaa
- Turku PET Centre, University of Turku, Turku 20520, Finland
- Department of Psychology, University of Turku, Turku 20520, Finland
| |
Collapse
|
41
|
Santavirta S, Karjalainen T, Nazari-Farsani S, Hudson M, Putkinen V, Seppälä K, Sun L, Glerean E, Hirvonen J, Karlsson HK, Nummenmaa L. Functional organization of social perception in the human brain. Neuroimage 2023; 272:120025. [PMID: 36958619 PMCID: PMC10112277 DOI: 10.1016/j.neuroimage.2023.120025] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2022] [Revised: 03/07/2023] [Accepted: 03/11/2023] [Indexed: 03/25/2023] Open
Abstract
Humans rapidly extract diverse and complex information from ongoing social interactions, but the perceptual and neural organization of the different aspects of social perception remains unresolved. We showed short movie clips with rich social content to 97 healthy participants while their haemodynamic brain activity was measured with fMRI. The clips were annotated moment-to-moment for a large set of social features and 45 of the features were evaluated reliably between annotators. Cluster analysis of the social features revealed that 13 dimensions were sufficient for describing the social perceptual space. Three different analysis methods were used to map the social perceptual processes in the human brain. Regression analysis mapped regional neural response profiles for different social dimensions. Multivariate pattern analysis then established the spatial specificity of the responses and intersubject correlation analysis connected social perceptual processing with neural synchronization. The results revealed a gradient in the processing of social information in the brain. Posterior temporal and occipital regions were broadly tuned to most social dimensions and the classifier revealed that these responses showed spatial specificity for social dimensions; in contrast Heschl gyri and parietal areas were also broadly associated with different social signals, yet the spatial patterns of responses did not differentiate social dimensions. Frontal and subcortical regions responded only to a limited number of social dimensions and the spatial response patterns did not differentiate social dimension. Altogether these results highlight the distributed nature of social processing in the brain.
Collapse
Affiliation(s)
- Severi Santavirta
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland.
| | - Tomi Karjalainen
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland
| | - Sanaz Nazari-Farsani
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland
| | - Matthew Hudson
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland; School of Psychology, University of Plymouth, Plymouth, United Kingdom
| | - Vesa Putkinen
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland
| | - Kerttu Seppälä
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland; Department of Medical Physics, Turku University Hospital, Turku, Finland
| | - Lihua Sun
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland; Department of Nuclear Medicine, Pudong Hospital, Fudan University, Shanghai, China
| | - Enrico Glerean
- Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Jussi Hirvonen
- Department of Radiology, University of Turku and Turku University Hospital, Turku, Finland; Medical Imaging Center, Department of Radiology, Tampere University and Tampere University Hospital, Tampere, Finland
| | - Henry K Karlsson
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland
| | - Lauri Nummenmaa
- Turku PET Centre, University of Turku, Turku, Finland; Turku PET Centre, Turku University Hospital, Turku, Finland; Department of Psychology, University of Turku, Turku, Finland
| |
Collapse
|
42
|
Russ BE, Koyano KW, Day-Cooney J, Perwez N, Leopold DA. Temporal continuity shapes visual responses of macaque face patch neurons. Neuron 2023; 111:903-914.e3. [PMID: 36630962 PMCID: PMC10023462 DOI: 10.1016/j.neuron.2022.12.021] [Citation(s) in RCA: 9] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/22/2022] [Revised: 09/09/2022] [Accepted: 12/13/2022] [Indexed: 01/12/2023]
Abstract
Macaque inferior temporal cortex neurons respond selectively to complex visual images, with recent work showing that they are also entrained reliably by the evolving content of natural movies. To what extent does temporal continuity itself shape the responses of high-level visual neurons? We addressed this question by measuring how cells in face-selective regions of the macaque visual cortex were affected by the manipulation of a movie's temporal structure. Sampling a 5-min movie at 1 s intervals, we measured neural responses to randomized, brief stimuli of different lengths, ranging from 800 ms dynamic movie snippets to 100 ms static frames. We found that the disruption of temporal continuity strongly altered neural response profiles, particularly in the early response period after stimulus onset. The results suggest that models of visual system function based on discrete and randomized visual presentations may not translate well to the brain's natural modes of operation.
Collapse
Affiliation(s)
- Brian E Russ
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA; Center for Biomedical Imaging and Neuromodulation, Nathan Kline Institute, Orangeburg, NY 10962, USA; Nash Family Department of Neuroscience and Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, New York, NY 10029, USA; Department of Psychiatry, New York University at Langone, New York City, NY 10016, USA.
| | - Kenji W Koyano
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA
| | - Julian Day-Cooney
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA
| | - Neda Perwez
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA
| | - David A Leopold
- Section on Cognitive Neurophysiology and Imaging, National Institute of Mental Health, Bethesda, MD 20814, USA; Neurophysiology Imaging Facility, National Institute of Mental Health, National Institute of Neurological Disorders and Stroke, National Eye Institute, Bethesda, MD 20814, USA
| |
Collapse
|
43
|
Zhou S, Xu X, He X, Zhou F, Zhai Y, Chen J, Long Y, Zheng L, Lu C. Biasing the neurocognitive processing of videos with the presence of a real cultural other. Cereb Cortex 2023; 33:1090-1103. [PMID: 35348645 DOI: 10.1093/cercor/bhac122] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2021] [Revised: 03/02/2022] [Accepted: 03/03/2022] [Indexed: 11/12/2022] Open
Abstract
In the digital age, while short videos present vital events with powerful information, the presence of cultural cues may bias our processing of videos of foreign cultures. However, the underlying neurocognitive processes remain unclear. In this study, we hypothesized that cultural cues might bias video processing by either enhancing cultural perspective-taking or shifting cultural self-schema. To test these hypotheses, we used a novel paradigm in which the cultural cue was a real cultural other (the priming participants) who watched American/Chinese videos together with the primed participants. The results showed that when the cue was present, the right temporoparietal junction (rTPJ) response to videos with other cultural content was shifted, showing a priming effect. Moreover, the activity pattern in the rTPJ was more congruent with the primed culture than with the original culture, reflecting a neural biasing effect. Finally, intersubject representational similarity analysis indicated that the neural biasing effect in the rTPJ was more closely associated with cultural perspective-taking than with cultural self-schema. In summary, these findings support the perspective-taking hypothesis, suggesting that cultural cues can significantly bias our cultural mindset by altering cultural perspective-taking when we are exposed to culture-relevant naturalistic stimuli.
Collapse
Affiliation(s)
- Siyuan Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China
| | - Xinran Xu
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China
| | - Xiangyu He
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China
| | - Faxin Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China
| | - Yu Zhai
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China
| | - Jinglu Chen
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China
| | - Yuhang Long
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China.,Institute of Developmental Psychology, Faculty of Psychology, Beijing Normal University, No19. Xinjiekouwai Street, Beijing 100875, PR China
| | - Lifen Zheng
- Center for Teacher Education Research, Faculty of Education, Beijing Normal University, No19. Xinjiekouwai Street, Beijing 100875, PR China
| | - Chunming Lu
- State Key Laboratory of Cognitive Neuroscience and Learning and IDG/McGovern Institute for Brain Research, Faculty of Psychology, Beijing Normal University, No. 19 Xinjiekouwai Street, Beijing 100875, PR China
| |
Collapse
|
44
|
Saalasti S, Alho J, Lahnakoski JM, Bacha-Trams M, Glerean E, Jääskeläinen IP, Hasson U, Sams M. Lipreading a naturalistic narrative in a female population: Neural characteristics shared with listening and reading. Brain Behav 2023; 13:e2869. [PMID: 36579557 PMCID: PMC9927859 DOI: 10.1002/brb3.2869] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 11/29/2022] [Accepted: 12/06/2022] [Indexed: 12/30/2022] Open
Abstract
INTRODUCTION Few of us are skilled lipreaders while most struggle with the task. Neural substrates that enable comprehension of connected natural speech via lipreading are not yet well understood. METHODS We used a data-driven approach to identify brain areas underlying the lipreading of an 8-min narrative with participants whose lipreading skills varied extensively (range 6-100%, mean = 50.7%). The participants also listened to and read the same narrative. The similarity between individual participants' brain activity during the whole narrative, within and between conditions, was estimated by a voxel-wise comparison of the Blood Oxygenation Level Dependent (BOLD) signal time courses. RESULTS Inter-subject correlation (ISC) of the time courses revealed that lipreading, listening to, and reading the narrative were largely supported by the same brain areas in the temporal, parietal and frontal cortices, precuneus, and cerebellum. Additionally, listening to and reading connected naturalistic speech particularly activated higher-level linguistic processing in the parietal and frontal cortices more consistently than lipreading, probably paralleling the limited understanding obtained via lip-reading. Importantly, higher lipreading test score and subjective estimate of comprehension of the lipread narrative was associated with activity in the superior and middle temporal cortex. CONCLUSIONS Our new data illustrates that findings from prior studies using well-controlled repetitive speech stimuli and stimulus-driven data analyses are also valid for naturalistic connected speech. Our results might suggest an efficient use of brain areas dealing with phonological processing in skilled lipreaders.
Collapse
Affiliation(s)
- Satu Saalasti
- Department of Psychology and Logopedics, University of Helsinki, Helsinki, Finland.,Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland.,Advanced Magnetic Imaging (AMI) Centre, Aalto NeuroImaging, School of Science, Aalto University, Espoo, Finland
| | - Jussi Alho
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Juha M Lahnakoski
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland.,Independent Max Planck Research Group for Social Neuroscience, Max Planck Institute of Psychiatry, Munich, Germany.,Institute of Neuroscience and Medicine, Brain & Behaviour (INM-7), Research Center Jülich, Jülich, Germany.,Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Mareike Bacha-Trams
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Enrico Glerean
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland.,Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, USA
| | - Iiro P Jääskeläinen
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland
| | - Uri Hasson
- Department of Psychology and the Neuroscience Institute, Princeton University, Princeton, USA
| | - Mikko Sams
- Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland.,Aalto Studios - MAGICS, Aalto University, Espoo, Finland
| |
Collapse
|
45
|
Jääskeläinen IP, Kosonogov V. Perspective taking in the human brain: complementary evidence from neuroimaging studies with media-based naturalistic stimuli and artificial controlled paradigms. Front Hum Neurosci 2023; 17:1051934. [PMID: 36875238 PMCID: PMC9975546 DOI: 10.3389/fnhum.2023.1051934] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Accepted: 02/27/2023] [Indexed: 02/17/2023] Open
Abstract
Perception and interpretation of various types of events and information in life crucially depend on one's perspective. A specific perspective can be explicitly adopted, for example, via instructing an experimental subject, implicitly via a priori information given to subjects, and by subjects' personality traits or cultural background. The neural basis of perspective taking has been addressed in a number of recent neuroimaging studies, some of which have used movies and narratives as media-based stimuli to pursue a holistic understanding of the phenomenon under ecologically valid conditions. Results across these studies suggest that the human brain flexibly adapts to support the information-processing needs of different perspectives, however, also that inferior temporal-occipital areas and posterior-medial parietal areas are engaged across different perspectives. These findings are complemented by studies that have investigated specific aspects of perspective taking with highly controlled experimental designs. They have disclosed involvement of the temporoparietal junction in visual perspective taking and the importance of the affective component of the pain matrix when empathizing with others' pain. Identification with the protagonists also seems to matter, as dorsomedial vs. ventromedial prefrontal areas are recruited when the protagonist is dissimilar vs. similar to self. Finally, as a translational aspect, perspective taking can, under certain conditions, serve as an effective emotion regulation technique, wherein lateral and medial regions of the prefrontal cortex seem to support reappraisal processes. Together, findings from studies with media-based stimuli and more traditional paradigms complement each other to gain a comprehensive understanding of the neural basis of perspective taking.
Collapse
Affiliation(s)
- Iiro P Jääskeläinen
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Vladimir Kosonogov
- International Laboratory of Social Neurobiology, Institute of Cognitive Neuroscience, HSE University, Moscow, Russia
| |
Collapse
|
46
|
Li H, Jiang S, Dong D, Hu J, He C, Hou C, He H, Huang H, Shen D, Pei H, Zhao G, Dong L, Yao D, Luo C. Vascular feature as a modulator of the aging brain. Cereb Cortex 2022; 32:5609-5621. [PMID: 35174854 DOI: 10.1093/cercor/bhac039] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2021] [Revised: 01/17/2022] [Accepted: 01/18/2022] [Indexed: 01/25/2023] Open
Abstract
The cerebral functional reorganization and declined cognitive function of aging might associate with altered vascular features. Here, we explored the altered cerebral hierarchical functional network of 2 conditions (task-free and naturalistic stimuli) in older adults and its relationship with vascular features (systemic microvascular and perfusion features, measured by magnetic resonance imaging) and behavior. Using cerebral gradient analysis, we found that compressive gradient of resting-state mainly located on the primary sensory-motor system and transmodal regions in aging, and further compress in these regions under the continuous naturalistic stimuli. Combining cerebral functional gradient, vascular features, and cognitive performance, the more compressive gradient in the resting-state, the worse vascular state, the lower cognitive function in older adults. Further modulation analysis demonstrated that both vascular features can regulate the relationship between gradient scores in the insula and behavior. Interestingly, systemic microvascular oxygenation also can modulate the relationship between cerebral gradient and cerebral perfusion. Furthermore, the less alteration of the compressive gradient with naturalistic stimuli came with lower cognitive function. Our findings demonstrated that the altered cerebral hierarchical functional structure in aging was linked with changed vascular features and behavior, offering a new framework for studying the physiological mechanism of functional connectivity in aging.
Collapse
Affiliation(s)
- Hechun Li
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu 610054, P. R. China
| | - Sisi Jiang
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu 610054, P. R. China
- Radiology Department, Chengdu Mental Health Center, Chengdu 610036, P. R. China
- Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu 2019RU035, P. R. China
| | - Debo Dong
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- Faculty of Psychology, Southwest University, Chongqing 400715, China
| | - Jian Hu
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
| | - Chuan He
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
| | - Changyue Hou
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu 610054, P. R. China
| | - Hui He
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- Radiology Department, Chengdu Mental Health Center, Chengdu 610036, P. R. China
| | - Huan Huang
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
| | - Dai Shen
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
| | - Haonan Pei
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
| | - Guocheng Zhao
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- Radiology Department, Chengdu Mental Health Center, Chengdu 610036, P. R. China
| | - Li Dong
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu 2019RU035, P. R. China
| | - Dezhong Yao
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu 610054, P. R. China
- Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu 2019RU035, P. R. China
- Radiation Oncology Key Laboratory of Sichuan Province, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Cancer Hospital affiliate to School of Medicine, University of Electronic Science and Technology of China, Chengdu 610042, P. R. China
| | - Cheng Luo
- The Clinical Hospital of Chengdu Brain Science Institute, MOE Key Lab for Neuroinformation, Center for Information in Medicine, School of Life Science and Technology, University of Electronic Science and Technology of China, No.4, Section 2, North Jianshe Road, Chengdu 610054, P. R. China
- High-Field Magnetic Resonance Brain Imaging Key Laboratory of Sichuan Province, University of Electronic Science and Technology of China, Chengdu 610054, P. R. China
- Research Unit of NeuroInformation, Chinese Academy of Medical Sciences, Chengdu 2019RU035, P. R. China
- Radiation Oncology Key Laboratory of Sichuan Province, Sichuan Cancer Hospital & Institute, Sichuan Cancer Center, Cancer Hospital affiliate to School of Medicine, University of Electronic Science and Technology of China, Chengdu 610042, P. R. China
| |
Collapse
|
47
|
Wilf M, Dupuis C, Nardo D, Huber D, Sander S, Al-Kaar J, Haroud M, Perrin H, Fornari E, Crottaz-Herbette S, Serino A. Virtual reality-based sensorimotor adaptation shapes subsequent spontaneous and naturalistic stimulus-driven brain activity. Cereb Cortex 2022; 33:5163-5180. [PMID: 36288926 PMCID: PMC10152055 DOI: 10.1093/cercor/bhac407] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2022] [Revised: 09/15/2022] [Accepted: 09/16/2022] [Indexed: 11/12/2022] Open
Abstract
Our everyday life summons numerous novel sensorimotor experiences, to which our brain needs to adapt in order to function properly. However, tracking plasticity of naturalistic behavior and associated brain modulations is challenging. Here, we tackled this question implementing a prism adaptation-like training in virtual reality (VRPA) in combination with functional neuroimaging. Three groups of healthy participants (N = 45) underwent VRPA (with a shift either to the left/right side, or with no shift), and performed functional magnetic resonance imaging (fMRI) sessions before and after training. To capture modulations in free-flowing, task-free brain activity, the fMRI sessions included resting-state and free-viewing of naturalistic videos. We found significant decreases in spontaneous functional connectivity between attentional and default mode (DMN)/fronto-parietal networks, only for the adaptation groups, more pronouncedly in the hemisphere contralateral to the induced shift. In addition, VRPA was found to bias visual responses to naturalistic videos: Following rightward adaptation, we found upregulation of visual response in an area in the parieto-occipital sulcus (POS) only in the right hemisphere. Notably, the extent of POS upregulation correlated with the size of the VRPA-induced after-effect measured in behavioral tests. This study demonstrates that a brief VRPA exposure can change large-scale cortical connectivity and correspondingly bias visual responses to naturalistic sensory inputs.
Collapse
Affiliation(s)
- Meytal Wilf
- MySpace Lab, Department of Clinical Neurosciences, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland.,Center of Advanced Technologies in Rehabilitation (CATR), Sheba Medical Center, Tel Hashomer 52621, Israel
| | - Celine Dupuis
- MindMaze SA, Chemin de Roseneck 5, 1006 Lausanne, Switzerland
| | - Davide Nardo
- MRC Cognition and Brain Sciences Unit, University of Cambridge, 15 Chaucer Rd, Cambridge CB2 7EF, United Kingdom.,Department of Education, University of Roma Tre, Rome, Italy
| | - Diana Huber
- MySpace Lab, Department of Clinical Neurosciences, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland
| | - Sibilla Sander
- MySpace Lab, Department of Clinical Neurosciences, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland
| | - Joud Al-Kaar
- Neuropsychology and Neurorehabilitation Service, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland
| | - Meriem Haroud
- MySpace Lab, Department of Clinical Neurosciences, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland.,Neuropsychology and Neurorehabilitation Service, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland
| | - Henri Perrin
- MySpace Lab, Department of Clinical Neurosciences, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland
| | - Eleonora Fornari
- Biomedical Imaging Center (CIBM), Department of Radiology, Lausanne University Hospital (CHUV) and University of Lausanne, Rue du Bugnon 46, 1011 Lausanne, Switzerland
| | - Sonia Crottaz-Herbette
- MindMaze SA, Chemin de Roseneck 5, 1006 Lausanne, Switzerland.,Neuropsychology and Neurorehabilitation Service, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland
| | - Andrea Serino
- MySpace Lab, Department of Clinical Neurosciences, Lausanne University Hospital (CHUV) and University of Lausanne, Avenue Pierre Decker 5, 1011 Lausanne, Switzerland.,MindMaze SA, Chemin de Roseneck 5, 1006 Lausanne, Switzerland
| |
Collapse
|
48
|
Niu J, Zheng Z, Wang Z, Xu L, Meng Q, Zhang X, Kuang L, Wang S, Dong L, Qiu J, Jiao Q, Cao W. Thalamo-cortical inter-subject functional correlation during movie watching across the adult lifespan. Front Neurosci 2022; 16:984571. [PMID: 36213738 PMCID: PMC9534554 DOI: 10.3389/fnins.2022.984571] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/02/2022] [Accepted: 09/05/2022] [Indexed: 11/13/2022] Open
Abstract
An increasing number of studies have shown that the functional interactions between the thalamus and cerebral cortices play an important role in cognitive function and are influenced by age. Previous studies have revealed age-related changes in the thalamo-cortical system within individuals, while neglecting differences between individuals. Here, we characterized inter-subject functional correlation (ISFC) between the thalamus and several cortical brain networks in 500 healthy participants aged 18–87 years old from the Cambridge Centre for Aging and Neuroscience (Cam-CAN) cohort using movie-watching state fMRI data. General linear models (GLM) were performed to assess age-related changes in ISFC of thalamo-cortical networks and the relationship between ISFC and fluid intelligence. We found significant age-related decreases in ISFC between the posterior thalamus (e.g., ventral posterior nucleus and pulvinar) and the attentional network, sensorimotor network, and visual network (FDR correction with p < 0.05). Meanwhile, the ISFC between the thalamus (mainly the mediodorsal nucleus and ventral thalamic nuclei) and higher-order cortical networks, including the default mode network, salience network and control network, showed complex changes with age. Furthermore, the altered ISFC of thalamo-cortical networks was positively correlated with decreased fluid intelligence (FDR correction with p < 0.05). Overall, our results provide further evidence that alterations in the functional integrity of the thalamo-cortical system might play an important role in cognitive decline during aging.
Collapse
Affiliation(s)
- Jinpeng Niu
- Department of Radiology, The Second Affiliated Hospital of Shandong First Medical University, Tai’an, China
- Department of Radiology, Shandong First Medical University and Shandong Academy of Medical Science, Tai’an, China
| | - Zihao Zheng
- Ministry of Education (MOE) Key Laboratory for Neuroinformation, School of Life Sciences and Technology, The Clinical Hospital of Chengdu Brain Science Institute, University of Electronic Science and Technology of China, Chengdu, China
| | - Ziqi Wang
- Ministry of Education (MOE) Key Laboratory for Neuroinformation, School of Life Sciences and Technology, The Clinical Hospital of Chengdu Brain Science Institute, University of Electronic Science and Technology of China, Chengdu, China
| | - Longchun Xu
- Department of Radiology, The Second Affiliated Hospital of Shandong First Medical University, Tai’an, China
| | - Qingmin Meng
- Department of Interventional Radiology, Taian Central Hospital, Tai’an, China
| | - Xiaotong Zhang
- Department of Radiology, Shandong First Medical University and Shandong Academy of Medical Science, Tai’an, China
| | - Liangfeng Kuang
- Department of Radiology, Shandong First Medical University and Shandong Academy of Medical Science, Tai’an, China
| | - Shigang Wang
- Department of Radiology, Shandong First Medical University and Shandong Academy of Medical Science, Tai’an, China
| | - Li Dong
- Ministry of Education (MOE) Key Laboratory for Neuroinformation, School of Life Sciences and Technology, The Clinical Hospital of Chengdu Brain Science Institute, University of Electronic Science and Technology of China, Chengdu, China
| | - Jianfeng Qiu
- Department of Radiology, Shandong First Medical University and Shandong Academy of Medical Science, Tai’an, China
| | - Qing Jiao
- Department of Radiology, The Second Affiliated Hospital of Shandong First Medical University, Tai’an, China
- Department of Radiology, Shandong First Medical University and Shandong Academy of Medical Science, Tai’an, China
| | - Weifang Cao
- Department of Radiology, The Second Affiliated Hospital of Shandong First Medical University, Tai’an, China
- Department of Radiology, Shandong First Medical University and Shandong Academy of Medical Science, Tai’an, China
- *Correspondence: Weifang Cao,
| |
Collapse
|
49
|
Kim SG. On the encoding of natural music in computational models and human brains. Front Neurosci 2022; 16:928841. [PMID: 36203808 PMCID: PMC9531138 DOI: 10.3389/fnins.2022.928841] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2022] [Accepted: 08/15/2022] [Indexed: 11/13/2022] Open
Abstract
This article discusses recent developments and advances in the neuroscience of music to understand the nature of musical emotion. In particular, it highlights how system identification techniques and computational models of music have advanced our understanding of how the human brain processes the textures and structures of music and how the processed information evokes emotions. Musical models relate physical properties of stimuli to internal representations called features, and predictive models relate features to neural or behavioral responses and test their predictions against independent unseen data. The new frameworks do not require orthogonalized stimuli in controlled experiments to establish reproducible knowledge, which has opened up a new wave of naturalistic neuroscience. The current review focuses on how this trend has transformed the domain of the neuroscience of music.
Collapse
|
50
|
Yu X, Li Y, Zhu K, Wang W, Wen W. Strong displayed passion and preparedness of broadcaster in live streaming e-commerce increases consumers' neural engagement. Front Psychol 2022; 13:674011. [PMID: 36176806 PMCID: PMC9514034 DOI: 10.3389/fpsyg.2022.674011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Accepted: 07/05/2022] [Indexed: 11/16/2022] Open
Abstract
Live streaming shopping, the streaming of real-time videos promoting products that consumers can purchase online, has recently been a booming area of e-commerce, especially during the COVID-19 pandemic. The success of live streaming e-commerce largely relies on the extent to which the broadcaster can get consumers engaged by the live stream. Thus, it is important to discover the antecedents of consumer engagement in such a context. Drawing on consumer engagement and neuroscience literature, this study used electroencephalography inter-subject correlation (EEG-ISC) to explore how broadcasters' entrepreneurial passion during live streaming videos influenced consumers' neural engagement as they watched the live streaming videos. We used the framework of displayed passion and preparedness from the entrepreneurial passion literature to predict consumer engagement. We found significant ISC for strong displayed passion, while preparedness had partially significant effects on the first, second, and summed components of ISC. The interaction effects of these two factors on the first and summed components of ISC were partially significant. Strong displayed passion and preparedness activated the left and right prefrontal regions of the consumers' brains. These findings indicate that broadcasters' displayed passion and preparedness can influence consumer engagement in live streaming e-commerce settings. Our findings suggest that a scientific approach could be used to improve a broadcaster's performance by testing ISC during rehearsals before live streaming.
Collapse
Affiliation(s)
- Xiaoyu Yu
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| | - Yajie Li
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
- *Correspondence: Yajie Li
| | - Kexin Zhu
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| | - Wenhao Wang
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| | - Wen Wen
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| |
Collapse
|