1
|
Morgenroth E, Vilaclara L, Muszynski M, Gaviria J, Vuilleumier P, Van De Ville D. Probing neurodynamics of experienced emotions-a Hitchhiker's guide to film fMRI. Soc Cogn Affect Neurosci 2023; 18:nsad063. [PMID: 37930850 PMCID: PMC10656947 DOI: 10.1093/scan/nsad063] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Revised: 08/04/2023] [Accepted: 11/01/2023] [Indexed: 11/08/2023] Open
Abstract
Film functional magnetic resonance imaging (fMRI) has gained tremendous popularity in many areas of neuroscience. However, affective neuroscience remains somewhat behind in embracing this approach, even though films lend themselves to study how brain function gives rise to complex, dynamic and multivariate emotions. Here, we discuss the unique capabilities of film fMRI for emotion research, while providing a general guide of conducting such research. We first give a brief overview of emotion theories as these inform important design choices. Next, we discuss films as experimental paradigms for emotion elicitation and address the process of annotating them. We then situate film fMRI in the context of other fMRI approaches, and present an overview of results from extant studies so far with regard to advantages of film fMRI. We also give an overview of state-of-the-art analysis techniques including methods that probe neurodynamics. Finally, we convey limitations of using film fMRI to study emotion. In sum, this review offers a practitioners' guide to the emerging field of film fMRI and underscores how it can advance affective neuroscience.
Collapse
Affiliation(s)
- Elenor Morgenroth
- Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Geneva 1202, Switzerland
- Department of Radiology and Medical Informatics, University of Geneva, Geneva 1202, Switzerland
- Swiss Center for Affective Sciences, University of Geneva, Geneva 1202, Switzerland
| | - Laura Vilaclara
- Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Geneva 1202, Switzerland
- Department of Radiology and Medical Informatics, University of Geneva, Geneva 1202, Switzerland
| | - Michal Muszynski
- Department of Basic Neurosciences, University of Geneva, Geneva 1202, Switzerland
| | - Julian Gaviria
- Swiss Center for Affective Sciences, University of Geneva, Geneva 1202, Switzerland
- Department of Basic Neurosciences, University of Geneva, Geneva 1202, Switzerland
- Department of Psychiatry, University of Geneva, Geneva 1202, Switzerland
| | - Patrik Vuilleumier
- Swiss Center for Affective Sciences, University of Geneva, Geneva 1202, Switzerland
- Department of Basic Neurosciences, University of Geneva, Geneva 1202, Switzerland
- CIBM Center for Biomedical Imaging, Geneva 1202, Switzerland
| | - Dimitri Van De Ville
- Neuro-X Institute, École Polytechnique Fédérale de Lausanne, Geneva 1202, Switzerland
- Department of Radiology and Medical Informatics, University of Geneva, Geneva 1202, Switzerland
- CIBM Center for Biomedical Imaging, Geneva 1202, Switzerland
| |
Collapse
|
2
|
Meier K, Tarczy-Hornoch K, Boynton GM, Fine I. Characterizing amblyopic perception under non-rivalrous viewing conditions. Sci Rep 2023; 13:7993. [PMID: 37198211 PMCID: PMC10189719 DOI: 10.1038/s41598-023-31301-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2022] [Accepted: 03/09/2023] [Indexed: 05/19/2023] Open
Abstract
Current assessments of interocular interactions in amblyopia often use rivalrous stimuli, with conflicting stimuli in each eye, which does not reflect vision under typical circumstances. Here we measure interocular interactions in observers with amblyopia, strabismus with equal vision, and controls using a non-rivalrous stimulus. Observers used a joystick to continuously report the perceived binocular contrast of dichoptic grating stimuli, identical except that the stimulus was contrast-modulated independently in each eye over time. Consistent with previous studies, a model predicting the time-course of perceived contrast found increased amblyopic eye attenuation, and reduced contrast normalization of the fellow eye by the amblyopic eye, in amblyopic participants compared to controls. However, these suppressive interocular effects were weaker than those found in previous studies, suggesting that rivalrous stimuli may overestimate the effects of amblyopia on interocular interactions during naturalistic viewing conditions.
Collapse
Affiliation(s)
- Kimberly Meier
- Department of Psychology, University of Washington, Seattle, WA, USA.
| | | | | | - Ione Fine
- Department of Psychology, University of Washington, Seattle, WA, USA
| |
Collapse
|
3
|
Xu S, Zhang Z, Li L, Zhou Y, Lin D, Zhang M, Zhang L, Huang G, Liu X, Becker B, Liang Z. Functional connectivity profiles of the default mode and visual networks reflect temporal accumulative effects of sustained naturalistic emotional experience. Neuroimage 2023; 269:119941. [PMID: 36791897 DOI: 10.1016/j.neuroimage.2023.119941] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2022] [Revised: 01/30/2023] [Accepted: 02/11/2023] [Indexed: 02/15/2023] Open
Abstract
Determining and decoding emotional brain processes under ecologically valid conditions remains a key challenge in affective neuroscience. The current functional Magnetic Resonance Imaging (fMRI) based emotion decoding studies are mainly based on brief and isolated episodes of emotion induction, while sustained emotional experience in naturalistic environments that mirror daily life experiences are scarce. Here we used 12 different 10-minute movie clips as ecologically valid emotion-evoking procedures in n = 52 individuals to explore emotion-specific fMRI functional connectivity (FC) profiles on the whole-brain level at high spatial resolution (432 parcellations including cortical and subcortical structures). Employing machine-learning based decoding and cross validation procedures allowed to investigate FC profiles contributing to classification that can accurately distinguish sustained happiness and sadness and that generalize across subjects, movie clips, and parcellations. Both functional brain network-based and subnetwork-based emotion classification results suggested that emotion manifests as distributed representation of multiple networks, rather than a single functional network or subnetwork. Further, the results showed that the Visual Network (VN) and Default Mode Network (DMN) associated functional networks, especially VN-DMN, exhibited a strong contribution to emotion classification. To further estimate the temporal accumulative effect of naturalistic long-term movie-based video-evoking emotions, we divided the 10-min episode into three stages: early stimulation (1∼200 s), middle stimulation (201∼400 s), and late stimulation (401∼600 s) and examined the emotion classification performance at different stimulation stages. We found that the late stimulation contributes most to the classification (accuracy=85.32%, F1-score=85.62%) compared to early and middle stimulation stages, implying that continuous exposure to emotional stimulation can lead to more intense emotions and further enhance emotion-specific distinguishable representations. The present work demonstrated that sustained happiness and sadness under naturalistic conditions are presented in emotion-specific network profiles and these expressions may play different roles in the generation and modulation of emotions. These findings elucidated the importance of network level adaptations for sustained emotional experiences during naturalistic contexts and open new venues for imaging network level contributions under naturalistic conditions.
Collapse
Affiliation(s)
- Shuyue Xu
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China; Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen 518060, China
| | - Zhiguo Zhang
- Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China; Peng Cheng Laboratory, Shenzhen 518055, China; Marshall Laboratory of Biomedical Engineering, Shenzhen 518060, China
| | - Linling Li
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China; Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen 518060, China
| | - Yongjie Zhou
- Department of Psychiatric Rehabilitation, Shenzhen Kangning Hospital, Shenzhen, China
| | - Danyi Lin
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China; Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen 518060, China
| | - Min Zhang
- Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China
| | - Li Zhang
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China; Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen 518060, China
| | - Gan Huang
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China; Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen 518060, China
| | - Xiqin Liu
- Center of Psychosomatic Medicine, Sichuan Provincial Center for Mental Health, MOE Key Laboratory for Neuroinformation, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu 611731, China
| | - Benjamin Becker
- Center of Psychosomatic Medicine, Sichuan Provincial Center for Mental Health, MOE Key Laboratory for Neuroinformation, Sichuan Provincial People's Hospital, University of Electronic Science and Technology of China, Chengdu 611731, China.
| | - Zhen Liang
- School of Biomedical Engineering, Medical School, Shenzhen University, Shenzhen 518060, China; Guangdong Provincial Key Laboratory of Biomedical Measurements and Ultrasound Imaging, Shenzhen 518060, China; Marshall Laboratory of Biomedical Engineering, Shenzhen 518060, China.
| |
Collapse
|
4
|
Grall C, Finn ES. Leveraging the Power of Media to Drive Cognition: A Media-Informed Approach to Naturalistic Neuroscience. Soc Cogn Affect Neurosci 2022; 17:598-608. [PMID: 35257180 PMCID: PMC9164202 DOI: 10.1093/scan/nsac019] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2021] [Revised: 02/01/2022] [Accepted: 03/07/2022] [Indexed: 11/18/2022] Open
Abstract
So-called ‘naturalistic’ stimuli have risen in popularity in cognitive, social and affective neuroscience over the last 15 years. However, a critical property of these stimuli is frequently overlooked: Media—like film, television, books and podcasts—are ‘fundamentally not natural’. They are deliberately crafted products meant to elicit particular human thought, emotion and behavior. Here, we argue for a more informed approach to adopting media stimuli in experimental paradigms. We discuss the pitfalls of combining stimuli that are designed for research with those that are designed for other purposes (e.g. entertainment) under the umbrella term of ‘naturalistic’ and present strategies to improve rigor in the stimulus selection process. We assert that experiencing media should be considered a task akin to any other experimental task(s) and explain how this shift in perspective will compel more nuanced and generalizable research using these stimuli. Throughout, we offer theoretical and practical knowledge from multidisciplinary media research to raise the standard for the treatment of media stimuli in neuroscience research.
Collapse
|
5
|
Bowren J, Sanchez-Giraldo L, Schwartz O. Inference via sparse coding in a hierarchical vision model. J Vis 2022; 22:19. [PMID: 35212744 PMCID: PMC8883180 DOI: 10.1167/jov.22.2.19] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022] Open
Abstract
Sparse coding has been incorporated in models of the visual cortex for its computational advantages and connection to biology. But how the level of sparsity contributes to performance on visual tasks is not well understood. In this work, sparse coding has been integrated into an existing hierarchical V2 model (Hosoya & Hyvärinen, 2015), but replacing its independent component analysis (ICA) with an explicit sparse coding in which the degree of sparsity can be controlled. After training, the sparse coding basis functions with a higher degree of sparsity resembled qualitatively different structures, such as curves and corners. The contributions of the models were assessed with image classification tasks, specifically tasks associated with mid-level vision including figure–ground classification, texture classification, and angle prediction between two line stimuli. In addition, the models were assessed in comparison with a texture sensitivity measure that has been reported in V2 (Freeman et al., 2013) and a deleted-region inference task. The results from the experiments show that although sparse coding performed worse than ICA at classifying images, only sparse coding was able to better match the texture sensitivity level of V2 and infer deleted image regions, both by increasing the degree of sparsity in sparse coding. Greater degrees of sparsity allowed for inference over larger deleted image regions. The mechanism that allows for this inference capability in sparse coding is described in this article.
Collapse
Affiliation(s)
- Joshua Bowren
- Department of Computer Science, University of Miami, Coral Gables, FL, USA.,
| | - Luis Sanchez-Giraldo
- Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, USA.,
| | - Odelia Schwartz
- Department of Computer Science, University of Miami, Coral Gables, FL, USA.,
| |
Collapse
|
6
|
Finn ES, Glerean E, Hasson U, Vanderwal T. Naturalistic imaging: The use of ecologically valid conditions to study brain function. Neuroimage 2021; 247:118776. [PMID: 34864153 DOI: 10.1016/j.neuroimage.2021.118776] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
7
|
Meer JNVD, Breakspear M, Chang LJ, Sonkusare S, Cocchi L. Movie viewing elicits rich and reliable brain state dynamics. Nat Commun 2020; 11:5004. [PMID: 33020473 PMCID: PMC7536385 DOI: 10.1038/s41467-020-18717-w] [Citation(s) in RCA: 66] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2020] [Accepted: 09/03/2020] [Indexed: 12/20/2022] Open
Abstract
Adaptive brain function requires that sensory impressions of the social and natural milieu are dynamically incorporated into intrinsic brain activity. While dynamic switches between brain states have been well characterised in resting state acquisitions, the remodelling of these state transitions by engagement in naturalistic stimuli remains poorly understood. Here, we show that the temporal dynamics of brain states, as measured in fMRI, are reshaped from predominantly bistable transitions between two relatively indistinct states at rest, toward a sequence of well-defined functional states during movie viewing whose transitions are temporally aligned to specific features of the movie. The expression of these brain states covaries with different physiological states and reflects subjectively rated engagement in the movie. In sum, a data-driven decoding of brain states reveals the distinct reshaping of functional network expression and reliable state transitions that accompany the switch from resting state to perceptual immersion in an ecologically valid sensory experience.
Collapse
Affiliation(s)
- Johan N van der Meer
- Program of Mental Health, QIMR Berghofer Medical Research Institute, Brisbane, 300 Herston Road, Brisbane, 4006, QLD, Australia.
| | - Michael Breakspear
- School of Psychology, Faculty of Science, University of Newcastle, University Drive, Callaghan, NSW, Australia
- Discipline of Psychiatry, Faculty of Health and Medicine, University of Newcastle, University Drive, Callaghan, NSW, Australia
| | - Luke J Chang
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755, NH, USA
| | - Saurabh Sonkusare
- Program of Mental Health, QIMR Berghofer Medical Research Institute, Brisbane, 300 Herston Road, Brisbane, 4006, QLD, Australia
| | - Luca Cocchi
- Program of Mental Health, QIMR Berghofer Medical Research Institute, Brisbane, 300 Herston Road, Brisbane, 4006, QLD, Australia.
| |
Collapse
|