Manipulating the structure of natural scenes using wavelets to study the functional architecture of perceptual hierarchies in the brain

Probing neurodynamics of experienced emotions-a Hitchhiker's guide to film fMRI

Film functional magnetic resonance imaging (fMRI) has gained tremendous popularity in many areas of neuroscience. However, affective neuroscience remains somewhat behind in embracing this approach, even though films lend themselves to study how brain function gives rise to complex, dynamic and multivariate emotions. Here, we discuss the unique capabilities of film fMRI for emotion research, while providing a general guide of conducting such research. We first give a brief overview of emotion theories as these inform important design choices. Next, we discuss films as experimental paradigms for emotion elicitation and address the process of annotating them. We then situate film fMRI in the context of other fMRI approaches, and present an overview of results from extant studies so far with regard to advantages of film fMRI. We also give an overview of state-of-the-art analysis techniques including methods that probe neurodynamics. Finally, we convey limitations of using film fMRI to study emotion. In sum, this review offers a practitioners' guide to the emerging field of film fMRI and underscores how it can advance affective neuroscience.

Characterizing amblyopic perception under non-rivalrous viewing conditions

Current assessments of interocular interactions in amblyopia often use rivalrous stimuli, with conflicting stimuli in each eye, which does not reflect vision under typical circumstances. Here we measure interocular interactions in observers with amblyopia, strabismus with equal vision, and controls using a non-rivalrous stimulus. Observers used a joystick to continuously report the perceived binocular contrast of dichoptic grating stimuli, identical except that the stimulus was contrast-modulated independently in each eye over time. Consistent with previous studies, a model predicting the time-course of perceived contrast found increased amblyopic eye attenuation, and reduced contrast normalization of the fellow eye by the amblyopic eye, in amblyopic participants compared to controls. However, these suppressive interocular effects were weaker than those found in previous studies, suggesting that rivalrous stimuli may overestimate the effects of amblyopia on interocular interactions during naturalistic viewing conditions.

Functional connectivity profiles of the default mode and visual networks reflect temporal accumulative effects of sustained naturalistic emotional experience

Determining and decoding emotional brain processes under ecologically valid conditions remains a key challenge in affective neuroscience. The current functional Magnetic Resonance Imaging (fMRI) based emotion decoding studies are mainly based on brief and isolated episodes of emotion induction, while sustained emotional experience in naturalistic environments that mirror daily life experiences are scarce. Here we used 12 different 10-minute movie clips as ecologically valid emotion-evoking procedures in n = 52 individuals to explore emotion-specific fMRI functional connectivity (FC) profiles on the whole-brain level at high spatial resolution (432 parcellations including cortical and subcortical structures). Employing machine-learning based decoding and cross validation procedures allowed to investigate FC profiles contributing to classification that can accurately distinguish sustained happiness and sadness and that generalize across subjects, movie clips, and parcellations. Both functional brain network-based and subnetwork-based emotion classification results suggested that emotion manifests as distributed representation of multiple networks, rather than a single functional network or subnetwork. Further, the results showed that the Visual Network (VN) and Default Mode Network (DMN) associated functional networks, especially VN-DMN, exhibited a strong contribution to emotion classification. To further estimate the temporal accumulative effect of naturalistic long-term movie-based video-evoking emotions, we divided the 10-min episode into three stages: early stimulation (1∼200 s), middle stimulation (201∼400 s), and late stimulation (401∼600 s) and examined the emotion classification performance at different stimulation stages. We found that the late stimulation contributes most to the classification (accuracy=85.32%, F1-score=85.62%) compared to early and middle stimulation stages, implying that continuous exposure to emotional stimulation can lead to more intense emotions and further enhance emotion-specific distinguishable representations. The present work demonstrated that sustained happiness and sadness under naturalistic conditions are presented in emotion-specific network profiles and these expressions may play different roles in the generation and modulation of emotions. These findings elucidated the importance of network level adaptations for sustained emotional experiences during naturalistic contexts and open new venues for imaging network level contributions under naturalistic conditions.

Leveraging the Power of Media to Drive Cognition: A Media-Informed Approach to Naturalistic Neuroscience

So-called ‘naturalistic’ stimuli have risen in popularity in cognitive, social and affective neuroscience over the last 15 years. However, a critical property of these stimuli is frequently overlooked: Media—like film, television, books and podcasts—are ‘fundamentally not natural’. They are deliberately crafted products meant to elicit particular human thought, emotion and behavior. Here, we argue for a more informed approach to adopting media stimuli in experimental paradigms. We discuss the pitfalls of combining stimuli that are designed for research with those that are designed for other purposes (e.g. entertainment) under the umbrella term of ‘naturalistic’ and present strategies to improve rigor in the stimulus selection process. We assert that experiencing media should be considered a task akin to any other experimental task(s) and explain how this shift in perspective will compel more nuanced and generalizable research using these stimuli. Throughout, we offer theoretical and practical knowledge from multidisciplinary media research to raise the standard for the treatment of media stimuli in neuroscience research.

Inference via sparse coding in a hierarchical vision model

Sparse coding has been incorporated in models of the visual cortex for its computational advantages and connection to biology. But how the level of sparsity contributes to performance on visual tasks is not well understood. In this work, sparse coding has been integrated into an existing hierarchical V2 model (Hosoya & Hyvärinen, 2015), but replacing its independent component analysis (ICA) with an explicit sparse coding in which the degree of sparsity can be controlled. After training, the sparse coding basis functions with a higher degree of sparsity resembled qualitatively different structures, such as curves and corners. The contributions of the models were assessed with image classification tasks, specifically tasks associated with mid-level vision including figure–ground classification, texture classification, and angle prediction between two line stimuli. In addition, the models were assessed in comparison with a texture sensitivity measure that has been reported in V2 (Freeman et al., 2013) and a deleted-region inference task. The results from the experiments show that although sparse coding performed worse than ICA at classifying images, only sparse coding was able to better match the texture sensitivity level of V2 and infer deleted image regions, both by increasing the degree of sparsity in sparse coding. Greater degrees of sparsity allowed for inference over larger deleted image regions. The mechanism that allows for this inference capability in sparse coding is described in this article.

Movie viewing elicits rich and reliable brain state dynamics

Adaptive brain function requires that sensory impressions of the social and natural milieu are dynamically incorporated into intrinsic brain activity. While dynamic switches between brain states have been well characterised in resting state acquisitions, the remodelling of these state transitions by engagement in naturalistic stimuli remains poorly understood. Here, we show that the temporal dynamics of brain states, as measured in fMRI, are reshaped from predominantly bistable transitions between two relatively indistinct states at rest, toward a sequence of well-defined functional states during movie viewing whose transitions are temporally aligned to specific features of the movie. The expression of these brain states covaries with different physiological states and reflects subjectively rated engagement in the movie. In sum, a data-driven decoding of brain states reveals the distinct reshaping of functional network expression and reliable state transitions that accompany the switch from resting state to perceptual immersion in an ecologically valid sensory experience.