From naturalistic neuroscience to modeling radical embodiment with narrative enactive systems

An embodiment of the cinematographer: emotional and perceptual responses to different camera movement techniques

We investigate the relationship between camera movement techniques and cognitive responses in audiences, reporting on an experiment exploring the effects of different camera movement methods on viewers' degree of immersion and emotional response. This follows directly from preceding experimental literature and is further motivated by accounts and experiences of practicing cinematographers (authors included), which indicates a correspondence between the two. We designed three different cinematic scenes with indifferent moods, and shot each one time with Steadicam, dolly, handheld, and static camera, resulting in 12 different clips. A total of 44 non-professional participants watched the clips and rated their reactions in terms of arousal and degree of involvement. Experimental results are mixed: movement affects the sense of involvement but not necessarily emotional response. We present and discuss some further explorative results and possible future directions to improve the design. We argue in this contribution that there is value in experimental approaches to cinematography, enabling the systematic study of creative intuitions and audience responses in controlled settings.

The Dual-Task Cost Is Due to Neural Interferences Disrupting the Optimal Spatio-Temporal Dynamics of the Competing Tasks

Dual-tasking is extremely prominent nowadays, despite ample evidence that it comes with a performance cost: the Dual-Task (DT) cost. Neuroimaging studies have established that tasks are more likely to interfere if they rely on common brain regions, but the precise neural origin of the DT cost has proven elusive so far, mostly because fMRI does not record neural activity directly and cannot reveal the key effect of timing, and how the spatio-temporal neural dynamics of the tasks coincide. Recently, DT electrophysiological studies in monkeys have recorded neural populations shared by the two tasks with millisecond precision to provide a much finer understanding of the origin of the DT cost. We used a similar approach in humans, with intracranial EEG, to assess the neural origin of the DT cost in a particularly challenging naturalistic paradigm which required accurate motor responses to frequent visual stimuli (task T1) and the retrieval of information from long-term memory (task T2), as when answering passengers’ questions while driving. We found that T2 elicited neuroelectric interferences in the gamma-band (>40 Hz), in key regions of the T1 network including the Multiple Demand Network. They reproduced the effect of disruptive electrocortical stimulations to create a situation of dynamical incompatibility, which might explain the DT cost. Yet, participants were able to flexibly adapt their strategy to minimize interference, and most surprisingly, reduce the reliance of T1 on key regions of the executive control network-the anterior insula and the dorsal anterior cingulate cortex-with no performance decrement.

Perceived Conventionality in Co-speech Gestures Involves the Fronto-Temporal Language Network

Face-to-face communication is multimodal; it encompasses spoken words, facial expressions, gaze, and co-speech gestures. In contrast to linguistic symbols (e.g., spoken words or signs in sign language) relying on mostly explicit conventions, gestures vary in their degree of conventionality. Bodily signs may have a general accepted or conventionalized meaning (e.g., a head shake) or less so (e.g., self-grooming). We hypothesized that subjective perception of conventionality in co-speech gestures relies on the classical language network, i.e., the left hemispheric inferior frontal gyrus (IFG, Broca's area) and the posterior superior temporal gyrus (pSTG, Wernicke's area) and studied 36 subjects watching video-recorded story retellings during a behavioral and an functional magnetic resonance imaging (fMRI) experiment. It is well documented that neural correlates of such naturalistic videos emerge as intersubject covariance (ISC) in fMRI even without involving a stimulus (model-free analysis). The subjects attended either to perceived conventionality or to a control condition (any hand movements or gesture-speech relations). Such tasks modulate ISC in contributing neural structures and thus we studied ISC changes to task demands in language networks. Indeed, the conventionality task significantly increased covariance of the button press time series and neuronal synchronization in the left IFG over the comparison with other tasks. In the left IFG, synchronous activity was observed during the conventionality task only. In contrast, the left pSTG exhibited correlated activation patterns during all conditions with an increase in the conventionality task at the trend level only. Conceivably, the left IFG can be considered a core region for the processing of perceived conventionality in co-speech gestures similar to spoken language. In general, the interpretation of conventionalized signs may rely on neural mechanisms that engage during language comprehension.