1
|
Yang MT, Fan HC, Lee HJ, Woudsma KJ, Lin KS, Liang JS, Lin FH. Inter-subject gamma oscillation synchronization as a biomarker of abnormal processing of social interaction in ADHD. Sci Rep 2024; 14:17924. [PMID: 39095651 PMCID: PMC11297305 DOI: 10.1038/s41598-024-68905-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2024] [Accepted: 07/29/2024] [Indexed: 08/04/2024] Open
Abstract
Children with attention-deficit hyperactivity disorder (ADHD) have difficulties in social interactions. Studying brain activity during social interactions is difficult with conventional artificial stimuli. This pioneering study examined the neural correlates of social perception in children with ADHD and matched controls using naturalistic stimuli. We presented 20 children with ADHD and 20 age-and-sex-matched controls with tailored movies featuring high- or low-level social interactions while recording electroencephalographic signals. Both groups exhibited synchronized gamma-band oscillations, but controls demonstrated greater inter-subject correlations. Additionally, the difference in inter-subject correlations between high- and low-interaction movies was significantly larger in controls compared to ADHD patients. Between 55 and 75 Hz comparing viewing high interaction movies with low interaction moves, controls had a significantly larger weighting in the right parietal lobe, while ADHD patients had a significantly smaller weighting in the left occipital lobe. These findings reveal distinct spatiotemporal neural signatures in social interaction processing among children with ADHD and controls using naturalistic stimuli. These neural markers offer potential for group differentiation and assessing intervention efficacy, advancing our understanding ADHD-related social interaction mechanisms.
Collapse
Affiliation(s)
- Ming-Tao Yang
- Department of Pediatrics, Far Eastern Memorial Hospital, No. 21, Sec. 2, Nanya S. Rd., Banciao Dist., New Taipei City, 220, Taiwan.
- Department of Chemical Engineering and Materials Science, Yuan Ze University, Taoyuan, Taiwan.
| | - Hueng-Chuen Fan
- Department of Pediatrics, Tungs' Taichung Metroharbor Hospital, Wuchi, Taichung, Taiwan
- Department of Rehabilitation, Jen-Teh Junior College of Medicine, Nursing and Management, Miaoli, Taiwan
- Department of Life Sciences, Agricultural Biotechnology Center, National Chung Hsing University, Taichung, Taiwan
| | - Hsin-Ju Lee
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, Canada
| | - K J Woudsma
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, Canada
| | - Kuen-Song Lin
- Department of Chemical Engineering and Materials Science, Yuan Ze University, Taoyuan, Taiwan
| | - Jao-Shwann Liang
- Department of Pediatrics, Far Eastern Memorial Hospital, No. 21, Sec. 2, Nanya S. Rd., Banciao Dist., New Taipei City, 220, Taiwan
| | - Fa-Hsuan Lin
- Physical Sciences Platform, Sunnybrook Research Institute, Toronto, Canada
- Department of Medical Biophysics, University of Toronto, Toronto, Canada
- Institute of Medical Science, University of Toronto, Toronto, Canada
| |
Collapse
|
2
|
Kaneshiro B, Nguyen DT, Norcia AM, Dmochowski JP, Berger J. Inter-subject correlation of electroencephalographic and behavioural responses reflects time-varying engagement with natural music. Eur J Neurosci 2024; 59:3162-3183. [PMID: 38626924 DOI: 10.1111/ejn.16324] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2021] [Revised: 02/28/2024] [Accepted: 03/07/2024] [Indexed: 06/15/2024]
Abstract
Musical engagement can be conceptualized through various activities, modes of listening and listener states. Recent research has reported that a state of focused engagement can be indexed by the inter-subject correlation (ISC) of audience responses to a shared naturalistic stimulus. While statistically significant ISC has been reported during music listening, we lack insight into the temporal dynamics of engagement over the course of musical works-such as those composed in the Western classical style-which involve the formulation of expectations that are realized or derailed at subsequent points of arrival. Here, we use the ISC of electroencephalographic (EEG) and continuous behavioural (CB) responses to investigate the time-varying dynamics of engagement with functional tonal music. From a sample of adult musicians who listened to a complete cello concerto movement, we found that ISC varied throughout the excerpt for both measures. In particular, significant EEG ISC was observed during periods of musical tension that built to climactic highpoints, while significant CB ISC corresponded more to declarative entrances and points of arrival. Moreover, we found that a control stimulus retaining envelope characteristics of the intact music, but little other temporal structure, also elicited significantly correlated EEG and CB responses, though to lesser extents than the original version. In sum, these findings shed light on the temporal dynamics of engagement during music listening and clarify specific aspects of musical engagement that may be indexed by each measure.
Collapse
Affiliation(s)
- Blair Kaneshiro
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, California, USA
- Center for the Study of Language and Information, Stanford University, Stanford, California, USA
- Graduate School of Education, Stanford University, Stanford, California, USA
| | - Duc T Nguyen
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, California, USA
- Center for the Study of Language and Information, Stanford University, Stanford, California, USA
| | - Anthony M Norcia
- Department of Psychology, Stanford University, Stanford, California, USA
- Wu Tsai Neurosciences Institute, Stanford University, Stanford, California, USA
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of New York, New York, New York, USA
| | - Jonathan Berger
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, California, USA
| |
Collapse
|
3
|
Madsen J, Parra LC. Bidirectional brain-body interactions during natural story listening. Cell Rep 2024; 43:114081. [PMID: 38581682 DOI: 10.1016/j.celrep.2024.114081] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/01/2023] [Revised: 11/25/2023] [Accepted: 03/24/2024] [Indexed: 04/08/2024] Open
Abstract
Narratives can synchronize neural and physiological signals between individuals, but the relationship between these signals, and the underlying mechanism, is unclear. We hypothesized a top-down effect of cognition on arousal and predicted that auditory narratives will drive not only brain signals but also peripheral physiological signals. We find that auditory narratives entrained gaze variation, saccade initiation, pupil size, and heart rate. This is consistent with a top-down effect of cognition on autonomic function. We also hypothesized a bottom-up effect, whereby autonomic physiology affects arousal. Controlled breathing affected pupil size, and heart rate was entrained by controlled saccades. Additionally, fluctuations in heart rate preceded fluctuations of pupil size and brain signals. Gaze variation, pupil size, and heart rate were all associated with anterior-central brain signals. Together, these results suggest bidirectional causal effects between peripheral autonomic function and central brain circuits involved in the control of arousal.
Collapse
Affiliation(s)
- Jens Madsen
- Department of Biomedical Engineering, City College of New York, 85 St. Nicholas Terrace, New York, NY 10031, USA.
| | - Lucas C Parra
- Department of Biomedical Engineering, City College of New York, 85 St. Nicholas Terrace, New York, NY 10031, USA
| |
Collapse
|
4
|
Virk T, Letendre T, Pathman T. The convergence of naturalistic paradigms and cognitive neuroscience methods to investigate memory and its development. Neuropsychologia 2024; 196:108779. [PMID: 38154592 DOI: 10.1016/j.neuropsychologia.2023.108779] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2023] [Revised: 12/12/2023] [Accepted: 12/23/2023] [Indexed: 12/30/2023]
Abstract
Studies that involve lab-based stimuli (e.g., words, pictures) are fundamental in the memory literature. At the same time, there is growing acknowledgment that memory processes assessed in the lab may not be analogous to how memory operates in the real world. Naturalistic paradigms can bridge this gap and over the decades a growing proportion of memory research has involved more naturalistic events. However, there is significant variation in the types of naturalistic studies used to study memory and its development, each with various advantages and limitations. Further, there are notable gaps in how often different types of naturalistic approaches have been combined with cognitive neuroscience methods (e.g., fMRI, EEG) to elucidate the neural processes and substrates involved in memory encoding and retrieval in the real world. Here we summarize and discuss what we identify as progressively more naturalistic methodologies used in the memory literature (movie, virtual reality, staged-events inside and outside of the lab, photo-taking, and naturally occurring event studies). Our goal is to describe each approach's benefits (e.g., naturalistic quality, feasibility), limitations (e.g., viability of neuroimaging method for event encoding versus event retrieval), and discuss possible future directions with each approach. We focus on child studies, when available, but also highlight past adult studies. Although there is a growing body of child memory research, naturalistic approaches combined with cognitive neuroscience methodologies in this domain remain sparse. Overall, this viewpoint article reviews how we can study memory through the lens of developmental cognitive neuroscience, while utilizing naturalistic and real-world events.
Collapse
|
5
|
Panela RA, Copelli F, Herrmann B. Reliability and generalizability of neural speech tracking in younger and older adults. Neurobiol Aging 2024; 134:165-180. [PMID: 38103477 DOI: 10.1016/j.neurobiolaging.2023.11.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/27/2023] [Revised: 11/09/2023] [Accepted: 11/16/2023] [Indexed: 12/19/2023]
Abstract
Neural tracking of spoken speech is considered a potential clinical biomarker for speech-processing difficulties, but the reliability of neural speech tracking is unclear. Here, younger and older adults listened to stories in two sessions while electroencephalography was recorded to investigate the reliability and generalizability of neural speech tracking. Speech tracking amplitude was larger for older than younger adults, consistent with an age-related loss of inhibition. The reliability of neural speech tracking was moderate (ICC ∼0.5-0.75) and tended to be higher for older adults. However, reliability was lower for speech tracking than for neural responses to noise bursts (ICC >0.8), which we used as a benchmark for maximum reliability. Neural speech tracking generalized moderately across different stories (ICC ∼0.5-0.6), which appeared greatest for audiobook-like stories spoken by the same person. Hence, a variety of stories could possibly be used for clinical assessments. Overall, the current data are important for developing a biomarker of speech processing but suggest that further work is needed to increase the reliability to meet clinical standards.
Collapse
Affiliation(s)
- Ryan A Panela
- Rotman Research Institute, Baycrest Academy for Research and Education, M6A 2E1 North York, ON, Canada; Department of Psychology, University of Toronto, M5S 1A1 Toronto, ON, Canada
| | - Francesca Copelli
- Rotman Research Institute, Baycrest Academy for Research and Education, M6A 2E1 North York, ON, Canada; Department of Psychology, University of Toronto, M5S 1A1 Toronto, ON, Canada
| | - Björn Herrmann
- Rotman Research Institute, Baycrest Academy for Research and Education, M6A 2E1 North York, ON, Canada; Department of Psychology, University of Toronto, M5S 1A1 Toronto, ON, Canada.
| |
Collapse
|
6
|
Kosonogov V, Ntoumanis I, Hajiyeva G, Jääskeläinen I. The role of engagement and arousal in emotion regulation: an EEG study. Exp Brain Res 2024; 242:179-193. [PMID: 37994917 DOI: 10.1007/s00221-023-06741-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Accepted: 11/02/2023] [Indexed: 11/24/2023]
Abstract
Cognitive reappraisal and expressive suppression are well-studied strategies of emotion regulation (ER). However, the results on their physiological basis are controversial. While in some studies, ER was accompanied by the inhibition of the nervous system, others suggested that ER even might increase arousal and engagement. We calculated the inter-subject correlation (ISC) and indices of engagement, valence and arousal of EEG during suppression, reappraisal, or natural watching of neutral and negative videos. First, both suppression and reappraisal provoked a higher ISC in comparison with watching negative or neutral videos. We consider this as a marker of engagement to the task and feedback processing required for ER. Second, the engagement index was lower during ER compared to watching negative videos in central electrodes, whereas both strategies provoked a higher engagement in frontal electrodes. Third, the arousal index of EEG was higher during all negative conditions; therefore, regulation required a certain level of arousal. In summary, different EEG measures seem to be sensitive to different aspects of ER.
Collapse
Affiliation(s)
- Vladimir Kosonogov
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Krivokolenny 3, Moscow, Russia, 101000.
| | - Ioannis Ntoumanis
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Krivokolenny 3, Moscow, Russia, 101000
| | - Gullu Hajiyeva
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Krivokolenny 3, Moscow, Russia, 101000
| | - Iiro Jääskeläinen
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Krivokolenny 3, Moscow, Russia, 101000
| |
Collapse
|
7
|
Zadina JN. The Synergy Zone: Connecting the Mind, Brain, and Heart for the Ideal Classroom Learning Environment. Brain Sci 2023; 13:1314. [PMID: 37759915 PMCID: PMC10526388 DOI: 10.3390/brainsci13091314] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2023] [Revised: 09/08/2023] [Accepted: 09/09/2023] [Indexed: 09/29/2023] Open
Abstract
This paper proposes a new perspective on implementing neuroeducation in the classroom. The pandemic exacerbated the mental health issues of faculty and students, creating a mental health crisis that impairs learning. It is important to get our students back in "the zone", both cognitively and emotionally, by creating an ideal learning environment for capturing our students and keeping them-the Synergy Zone. Research that examines the classroom environment often focuses on the foreground-instructors' organizational and instructional aspects and content. However, the emotional climate of the classroom affects student well-being. This emotional climate would ideally exhibit the brain states of engagement, attention, connection, and enjoyment by addressing the mind, brain, and heart. This ideal learning environment would be achieved by combining proposed practices derived from three areas of research: flow theory, brain synchronization, and positive emotion with heart engagement. Each of these enhances the desired brain states in a way that the whole is greater than the sum of the individual parts. I call this the Synergy Zone. A limitation of this proposed model is that implementation of some aspects may be challenging, and professional development resources might be needed. This essay presenting this perspective provides the relevant scientific research and the educational implications of implementation.
Collapse
Affiliation(s)
- Janet N Zadina
- Brain Research and Instruction, New Orleans, LA 70002, USA
| |
Collapse
|
8
|
Sheng J, Wang S, Zhang L, Liu C, Shi L, Zhou Y, Hu H, Chen C, Xue G. Intersubject similarity in neural representations underlies shared episodic memory content. Proc Natl Acad Sci U S A 2023; 120:e2308951120. [PMID: 37603733 PMCID: PMC10466090 DOI: 10.1073/pnas.2308951120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2023] [Accepted: 07/05/2023] [Indexed: 08/23/2023] Open
Abstract
Individuals generally form their unique memories from shared experiences, yet the neural representational mechanisms underlying this subjectiveness of memory are poorly understood. The current study addressed this important question from the cross-subject neural representational perspective, leveraging a large functional magnetic resonance imaging dataset (n = 415) of a face-name associative memory task. We found that individuals' memory abilities were predicted by their synchronization to the group-averaged, canonical trial-by-trial activation level and, to a lesser degree, by their similarity to the group-averaged representational patterns during encoding. More importantly, the memory content shared between pairs of participants could be predicted by their shared local neural activation pattern, particularly in the angular gyrus and ventromedial prefrontal cortex, even after controlling for differences in memory abilities. These results uncover neural representational mechanisms for individualized memory and underscore the constructive nature of episodic memory.
Collapse
Affiliation(s)
- Jintao Sheng
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Sisi Wang
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Liang Zhang
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Chuqi Liu
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Liang Shi
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Yu Zhou
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Huinan Hu
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| | - Chuansheng Chen
- Department of Psychological Science, University of California, Irvine, CA92697
| | - Gui Xue
- State Key Laboratory of Cognitive Neuroscience and Learning and International Data Group/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing102206, China
| |
Collapse
|
9
|
Farrell S, Valdes AL. 'The Mind' promotes brain synchronization: an ecological evaluation of brain synchronization in co-operative tasks. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023; 2023:1-4. [PMID: 38082864 DOI: 10.1109/embc40787.2023.10340212] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/18/2023]
Abstract
This paper presents an ecologically valid approach for using EEG hyperscanning methods to assess levels of interbrain synchrony (IBS) in teams during co-operative tasks. We employ a card-based task in an out-of-the-lab setting to evaluate levels of neural synchrony between team members completing a co-operative task. We also examine the interplay between the recorded synchronization levels and the collective performance of the team.Clinical Relevance- This study provides a simplistic and ecologically valid setup with potential to bring a better understanding of brain synchronization in clinical settings where co-operation would improve outcomes, such as home care facilities and memory clinics.
Collapse
|
10
|
Stuldreher IV, Maasland E, Bottenheft C, van Erp JBF, Brouwer AM. Physiological synchrony in electrodermal activity predicts decreased vigilant attention induced by sleep deprivation. FRONTIERS IN NEUROERGONOMICS 2023; 4:1199347. [PMID: 38234480 PMCID: PMC10790929 DOI: 10.3389/fnrgo.2023.1199347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 04/03/2023] [Accepted: 06/15/2023] [Indexed: 01/19/2024]
Abstract
Introduction When multiple individuals are presented with narrative movie or audio clips, their electrodermal activity (EDA) and heart rate show significant similarities. Higher levels of such inter-subject physiological synchrony are related with higher levels of attention toward the narrative, as for instance expressed by more correctly answered questions about the narrative. We here investigate whether physiological synchrony in EDA and heart rate during watching of movie clips predicts performance on a subsequent vigilant attention task among participants exposed to a night of total sleep deprivation. Methods We recorded EDA and heart rate of 54 participants during a night of total sleep deprivation. Every hour from 22:00 to 07:00 participants watched a 10-min movie clip during which we computed inter-subject physiological synchrony. Afterwards, they answered questions about the movie and performed the psychomotor vigilance task (PVT) to capture attentional performance. Results We replicated findings that inter-subject correlations in EDA and heart rate predicted the number of correct answers on questions about the movie clips. Furthermore, we found that inter-subject correlations in EDA, but not in heart rate, predicted PVT performance. Individuals' mean EDA and heart rate also predicted their PVT performance. For EDA, inter-subject correlations explained more variance of PVT performance than individuals' mean EDA. Discussion Together, these findings confirm the association between physiological synchrony and attention. Physiological synchrony in EDA does not only capture the attentional processing during the time that it is determined, but also proves valuable for capturing more general changes in the attentional state of monitored individuals.
Collapse
Affiliation(s)
- Ivo V. Stuldreher
- Human Performance, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
| | - Emma Maasland
- Human Performance, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
| | - Charelle Bottenheft
- Human Performance, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
| | - Jan B. F. van Erp
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
- Human Machine Teaming, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
| | - Anne-Marie Brouwer
- Human Performance, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, Netherlands
| |
Collapse
|
11
|
Dini H, Simonetti A, Bigne E, Bruni LE. Higher levels of narrativity lead to similar patterns of posterior EEG activity across individuals. Front Hum Neurosci 2023; 17:1160981. [PMID: 37234601 PMCID: PMC10206039 DOI: 10.3389/fnhum.2023.1160981] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Accepted: 04/26/2023] [Indexed: 05/28/2023] Open
Abstract
Introduction The focus of cognitive and psychological approaches to narrative has not so much been on the elucidation of important aspects of narrative, but rather on using narratives as tools for the investigation of higher order cognitive processes elicited by narratives (e.g., understanding, empathy, etc.). In this study, we work toward a scalar model of narrativity, which can provide testable criteria for selecting and classifying communication forms in their level of narrativity. We investigated whether being exposed to videos with different levels of narrativity modulates shared neural responses, measured by inter-subject correlation, and engagement levels. Methods Thirty-two participants watched video advertisements with high-level and low-level of narrativity while their neural responses were measured through electroencephalogram. Additionally, participants' engagement levels were calculated based on the composite of their self-reported attention and immersion scores. Results Results demonstrated that both calculated inter-subject correlation and engagement scores for high-level video ads were significantly higher than those for low-level, suggesting that narrativity levels modulate inter-subject correlation and engagement. Discussion We believe that these findings are a step toward the elucidation of the viewers' way of processing and understanding a given communication artifact as a function of the narrative qualities expressed by the level of narrativity.
Collapse
Affiliation(s)
- Hossein Dini
- The Augmented Cognition Lab, Aalborg University, Copenhagen, Denmark
| | - Aline Simonetti
- Department of Marketing and Market Research, University of Valencia, Valencia, Spain
| | - Enrique Bigne
- Department of Marketing and Market Research, University of Valencia, Valencia, Spain
| | - Luis Emilio Bruni
- The Augmented Cognition Lab, Aalborg University, Copenhagen, Denmark
| |
Collapse
|
12
|
Hammond H, Armstrong M, Thomas GA, Gilchrist ID. Audience immersion: validating attentional and physiological measures against self-report. Cogn Res Princ Implic 2023; 8:22. [PMID: 37074525 PMCID: PMC10113978 DOI: 10.1186/s41235-023-00475-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2022] [Accepted: 04/04/2023] [Indexed: 04/20/2023] Open
Abstract
When an audience member becomes immersed, their attention shifts towards the media and story, and they allocate cognitive resources to represent events and characters. Here, we investigate whether it is possible to measure immersion using continuous behavioural and physiological measures. Using television and film clips, we validated dual-task reaction times, heart rate, and skin conductance against self-reported narrative engagement. We find that reaction times to a secondary task were strongly positively correlated with self-reported immersion: slower reaction times were indicative of greater immersion, particularly emotional engagement. Synchrony in heart rate across participants was associated with self-reported attentional and emotional engagement with the story, although we found no such relationship with skin conductance. These results establish both dual-task reaction times and heart rate as candidate measures for the real-time, continuous, assessment of audience immersion.
Collapse
Affiliation(s)
- Hugo Hammond
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol, BS8 1TU, UK.
| | - Michael Armstrong
- British Broadcasting Corporation (BBC) Research and Development, Saltford, UK
| | - Graham A Thomas
- British Broadcasting Corporation (BBC) Research and Development, Saltford, UK
| | - Iain D Gilchrist
- School of Psychological Science, University of Bristol, 12a Priory Road, Bristol, BS8 1TU, UK
| |
Collapse
|
13
|
Davidesco I, Laurent E, Valk H, West T, Milne C, Poeppel D, Dikker S. The Temporal Dynamics of Brain-to-Brain Synchrony Between Students and Teachers Predict Learning Outcomes. Psychol Sci 2023; 34:633-643. [PMID: 37053267 DOI: 10.1177/09567976231163872] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/15/2023] Open
Abstract
Much of human learning happens through interaction with other people, but little is known about how this process is reflected in the brains of students and teachers. Here, we concurrently recorded electroencephalography (EEG) data from nine groups, each of which contained four students and a teacher. All participants were young adults from the northeast United States. Alpha-band (8-12 Hz) brain-to-brain synchrony between students predicted both immediate and delayed posttest performance. Further, brain-to-brain synchrony was higher in specific lecture segments associated with questions that students answered correctly. Brain-to-brain synchrony between students and teachers predicted learning outcomes at an approximately 300-ms lag in the students' brain activity relative to the teacher's brain activity, which is consistent with the time course of spoken-language comprehension. These findings provide key new evidence for the importance of collecting brain data simultaneously from groups of learners in ecologically valid settings.
Collapse
Affiliation(s)
- Ido Davidesco
- Department of Educational Psychology, University of Connecticut
| | | | | | - Tessa West
- Department of Psychology, New York University
| | | | - David Poeppel
- Department of Psychology, New York University
- Ernst Strüngmann Institute for Neuroscience, Frankfurt, Germany
| | | |
Collapse
|
14
|
Ntoumanis I, Davydova A, Sheronova J, Panidi K, Kosonogov V, Shestakova AN, Jääskeläinen IP, Klucharev V. Neural mechanisms of expert persuasion on willingness to pay for sugar. Front Behav Neurosci 2023; 17:1147140. [PMID: 36992860 PMCID: PMC10040640 DOI: 10.3389/fnbeh.2023.1147140] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Accepted: 02/20/2023] [Indexed: 03/15/2023] Open
Abstract
Introduction: Sugar consumption is associated with many negative health consequences. It is, therefore, important to understand what can effectively influence individuals to consume less sugar. We recently showed that a healthy eating call by a health expert can significantly decrease the willingness to pay (WTP) for sugar-containing food. Here, we investigate which aspects of neural responses to the same healthy eating call can predict the efficacy of expert persuasion.Methods: Forty-five healthy participants performed two blocks of a bidding task, in which they had to bid on sugar-containing, sugar-free and non-edible products, while their electroencephalography (EEG) was recorded. In between the two blocks, they listened to a healthy eating call by a nutritionist emphasizing the risks of sugar consumption.Results: We found that after listening to the healthy eating call, participants significantly decreased their WTP for sugar-containing products. Moreover, a higher intersubject correlation of EEG (a measure of engagement) during listening to the healthy eating call resulted in a larger decrease in WTP for sugar-containing food. Whether or not a participant’s valuation of a product was highly influenced by the healthy eating call could also be predicted by spatiotemporal patterns of EEG responses to the healthy eating call, using a machine learning classification model. Finally, the healthy eating call increased the amplitude of the P300 component of the visual event-related potential in response to sugar-containing food.Disussion: Overall, our results shed light on the neural basis of expert persuasion and demonstrate that EEG is a powerful tool to design and assess health-related advertisements before they are released to the public.
Collapse
Affiliation(s)
- Ioannis Ntoumanis
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
- *Correspondence: Ioannis Ntoumanis
| | - Alina Davydova
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Julia Sheronova
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Ksenia Panidi
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Vladimir Kosonogov
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Anna N. Shestakova
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Iiro P. Jääskeläinen
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| | - Vasily Klucharev
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| |
Collapse
|
15
|
Stuldreher IV, van Erp JBF, Brouwer AM. Robustness of Physiological Synchrony in Wearable Electrodermal Activity and Heart Rate as a Measure of Attentional Engagement to Movie Clips. SENSORS (BASEL, SWITZERLAND) 2023; 23:3006. [PMID: 36991720 PMCID: PMC10058467 DOI: 10.3390/s23063006] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Revised: 03/01/2023] [Accepted: 03/06/2023] [Indexed: 06/19/2023]
Abstract
Individuals that pay attention to narrative stimuli show synchronized heart rate (HR) and electrodermal activity (EDA) responses. The degree to which this physiological synchrony occurs is related to attentional engagement. Factors that can influence attention, such as instructions, salience of the narrative stimulus and characteristics of the individual, affect physiological synchrony. The demonstrability of synchrony depends on the amount of data used in the analysis. We investigated how demonstrability of physiological synchrony varies with varying group size and stimulus duration. Thirty participants watched six 10 min movie clips while their HR and EDA were monitored using wearable sensors (Movisens EdaMove 4 and Wahoo Tickr, respectively). We calculated inter-subject correlations as a measure of synchrony. Group size and stimulus duration were varied by using data from subsets of the participants and movie clips in the analysis. We found that for HR, higher synchrony correlated significantly with the number of answers correct for questions about the movie, confirming that physiological synchrony is associated with attention. For both HR and EDA, with increasing amounts of data used, the percentage of participants with significant synchrony increased. Importantly, we found that it did not matter how the amount of data was increased. Increasing the group size or increasing the stimulus duration led to the same results. Initial comparisons with results from other studies suggest that our results do not only apply to our specific set of stimuli and participants. All in all, the current work can act as a guideline for future research, indicating the amount of data minimally needed for robust analysis of synchrony based on inter-subject correlations.
Collapse
Affiliation(s)
- Ivo V. Stuldreher
- Human Performance, Netherlands Organisation for Applied Scientific Research (TNO), 3769 DE Soesterberg, The Netherlands
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7522 NB Enschede, The Netherlands
| | - Jan B. F. van Erp
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, 7522 NB Enschede, The Netherlands
- Human Machine Teaming, Netherlands Organisation for Applied Scientific Research (TNO), 3769 DE Soesterberg, The Netherlands
| | - Anne-Marie Brouwer
- Human Performance, Netherlands Organisation for Applied Scientific Research (TNO), 3769 DE Soesterberg, The Netherlands
- Artificial Intelligence, Faculty of Social Sciences, Radboud University, Thomas van Aquinostraat 4, 6525 GD Nijmegen, The Netherlands
| |
Collapse
|
16
|
Ntoumanis I, Shestakova A, Koriakina M, Kadieva D, Kopytin G, Jääskeläinen IP. Developmental differences in the perception of naturalistic human movements. Front Hum Neurosci 2023; 16:1046277. [PMID: 36704095 PMCID: PMC9872020 DOI: 10.3389/fnhum.2022.1046277] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/16/2022] [Accepted: 12/19/2022] [Indexed: 01/11/2023] Open
Abstract
Introduction It is widely believed that we are more attentive towards moving versus static stimuli. However, the neural correlates underlying the perception of human movements have not been extensively investigated in ecologically valid settings, nor has the developmental aspect of this phenomenon. Here, we set forth to investigate how human limb movements displayed in naturalistic videos influence the attentional engagement of children and young adults. Methods Thirty-nine healthy participants (4-26 years old) were presented with naturalistic videos featuring human goal-directed movements, while neural activity was recorded using electroencephalography (EEG). Video scenes were automatically annotated as containing arm, leg or no movement, using a machine learning algorithm. The viewers' attentional engagement was quantified by the intersubject correlation of EEG responses evoked by the videos. Results Our results demonstrate that scenes featuring limb movements, especially simultaneous arm and leg movements, elicit higher attentional engagement than scenes with no limb movement. Interestingly, this effect was found to diminish with age. Discussion Overall, our findings extend previous work on the perception of human motion by implementing naturalistic stimuli in the experimental design and extend the list of factors influencing the viewer's engagement exerted by naturalistic videos.
Collapse
Affiliation(s)
- Ioannis Ntoumanis
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Anna Shestakova
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Maria Koriakina
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
- Federal State Budgetary Institution the Turner Scientific Research Institute for Children’s Orthopedics Under the Ministry of Health of the Russian Federation, Saint-Petersburg, Russia
| | - Dzerassa Kadieva
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Grigory Kopytin
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
| | - Iiro P. Jääskeläinen
- International Laboratory of Social Neurobiology, Institute for Cognitive Neuroscience, HSE University, Moscow, Russia
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
| |
Collapse
|
17
|
EEG measures of attention toward food-related stimuli vary with food neophobia. Food Qual Prefer 2022. [DOI: 10.1016/j.foodqual.2022.104805] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
|
18
|
Ntoumanis I, Agranovich O, Shestakova AN, Blagovechtchenski E, Koriakina M, Kadieva D, Kopytin G, Jääskeläinen IP. Altered Cerebral Processing of Videos in Children with Motor Dysfunction Suggests Broad Embodiment of Perceptual Cognitive Functions. J Pers Med 2022; 12:jpm12111841. [PMID: 36579567 PMCID: PMC9697218 DOI: 10.3390/jpm12111841] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2022] [Revised: 10/31/2022] [Accepted: 11/03/2022] [Indexed: 11/10/2022] Open
Abstract
Embodied cognition theory suggests that motor dysfunctions affect cognition. We examined this hypothesis by inspecting whether cerebral processing of movies, featuring both goal-directed movements and content without humans, differ between children with congenital motor dysfunction and healthy controls. Electroencephalography was recorded from 23 healthy children and 23 children with limited or absent arm movement due to either arthrogryposis multiplex congenita or obstetric brachial plexus palsy. Each individual patient exhibited divergent neural responses, disclosed by significantly lower inter-subject correlation (ISC) of brain activity, during the videos compared to the healthy children. We failed to observe associations between this finding and the motor-related content of the various video scenes, suggesting that differences between the patients and controls reflect modulation of perceptual-cognitive processing of videos by upper-limb motor dysfunctions not limited to the watching-mirroring of motor actions. Thus, perceptual-cognitive processes in the brain seem to be more robustly embodied than has previously been thought.
Collapse
Affiliation(s)
- Ioannis Ntoumanis
- Institute for Cognitive Neuroscience, HSE University, Myasnitskaya Ulitsa, 20, 101000 Moscow, Russia
- Correspondence: or ; Tel.: +7-9-256988509
| | - Olga Agranovich
- Federal State Budgetary Institution the Turner Scientific Research Institute for Children’s Orthopedics under the Ministry of Health of the Russian Federation, 196603 St. Petersburg, Russia
| | - Anna N. Shestakova
- Institute for Cognitive Neuroscience, HSE University, Myasnitskaya Ulitsa, 20, 101000 Moscow, Russia
| | - Evgeny Blagovechtchenski
- Institute for Cognitive Neuroscience, HSE University, Myasnitskaya Ulitsa, 20, 101000 Moscow, Russia
- Federal State Budgetary Institution the Turner Scientific Research Institute for Children’s Orthopedics under the Ministry of Health of the Russian Federation, 196603 St. Petersburg, Russia
| | - Maria Koriakina
- Institute for Cognitive Neuroscience, HSE University, Myasnitskaya Ulitsa, 20, 101000 Moscow, Russia
- Federal State Budgetary Institution the Turner Scientific Research Institute for Children’s Orthopedics under the Ministry of Health of the Russian Federation, 196603 St. Petersburg, Russia
| | - Dzerassa Kadieva
- Institute for Cognitive Neuroscience, HSE University, Myasnitskaya Ulitsa, 20, 101000 Moscow, Russia
| | - Grigory Kopytin
- Institute for Cognitive Neuroscience, HSE University, Myasnitskaya Ulitsa, 20, 101000 Moscow, Russia
| | - Iiro P. Jääskeläinen
- Institute for Cognitive Neuroscience, HSE University, Myasnitskaya Ulitsa, 20, 101000 Moscow, Russia
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, School of Science, Aalto University, 02150 Espoo, Finland
| |
Collapse
|
19
|
Yu X, Li Y, Zhu K, Wang W, Wen W. Strong displayed passion and preparedness of broadcaster in live streaming e-commerce increases consumers' neural engagement. Front Psychol 2022; 13:674011. [PMID: 36176806 PMCID: PMC9514034 DOI: 10.3389/fpsyg.2022.674011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2021] [Accepted: 07/05/2022] [Indexed: 11/16/2022] Open
Abstract
Live streaming shopping, the streaming of real-time videos promoting products that consumers can purchase online, has recently been a booming area of e-commerce, especially during the COVID-19 pandemic. The success of live streaming e-commerce largely relies on the extent to which the broadcaster can get consumers engaged by the live stream. Thus, it is important to discover the antecedents of consumer engagement in such a context. Drawing on consumer engagement and neuroscience literature, this study used electroencephalography inter-subject correlation (EEG-ISC) to explore how broadcasters' entrepreneurial passion during live streaming videos influenced consumers' neural engagement as they watched the live streaming videos. We used the framework of displayed passion and preparedness from the entrepreneurial passion literature to predict consumer engagement. We found significant ISC for strong displayed passion, while preparedness had partially significant effects on the first, second, and summed components of ISC. The interaction effects of these two factors on the first and summed components of ISC were partially significant. Strong displayed passion and preparedness activated the left and right prefrontal regions of the consumers' brains. These findings indicate that broadcasters' displayed passion and preparedness can influence consumer engagement in live streaming e-commerce settings. Our findings suggest that a scientific approach could be used to improve a broadcaster's performance by testing ISC during rehearsals before live streaming.
Collapse
Affiliation(s)
- Xiaoyu Yu
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| | - Yajie Li
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
- *Correspondence: Yajie Li
| | - Kexin Zhu
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| | - Wenhao Wang
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| | - Wen Wen
- Laboratory Neuro-Entrepreneurship, Shanghai University, Shanghai, China
- School of Management, Shanghai University, Shanghai, China
| |
Collapse
|
20
|
Personality similarity predicts synchronous neural responses in fMRI and EEG data. Sci Rep 2022; 12:14325. [PMID: 35995958 PMCID: PMC9395427 DOI: 10.1038/s41598-022-18237-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Accepted: 08/08/2022] [Indexed: 11/27/2022] Open
Abstract
Successful communication and cooperation among different members of society depends, in part, on a consistent understanding of the physical and social world. What drives this alignment in perspectives? We present evidence from two neuroimaging studies using functional magnetic resonance imaging (fMRI; N = 66 with 2145 dyadic comparisons) and electroencephalography (EEG; N = 225 with 25,200 dyadic comparisons) to show that: (1) the extent to which people’s neural responses are synchronized when viewing naturalistic stimuli is related to their personality profiles, and (2) that this effect is stronger than that of similarity in gender, ethnicity and political affiliation. The localization of the fMRI results in combination with the additional eye tracking analyses suggest that the relationship between personality similarity and neural synchrony likely reflects alignment in the interpretation of stimuli and not alignment in overt visual attention. Together, the findings suggest that similarity in psychological dispositions aligns people’s reality via shared interpretations of the external world.
Collapse
|
21
|
Truong NCD, Wang X, Wanniarachchi H, Lang Y, Nerur S, Chen KY, Liu H. Mapping and understanding of correlated electroencephalogram (EEG) responses to the newsvendor problem. Sci Rep 2022; 12:13800. [PMID: 35963934 PMCID: PMC9376113 DOI: 10.1038/s41598-022-17970-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Accepted: 08/03/2022] [Indexed: 12/03/2022] Open
Abstract
Decision-making is one of the most critical activities of human beings. To better understand the underlying neurocognitive mechanism while making decisions under an economic context, we designed a decision-making paradigm based on the newsvendor problem (NP) with two scenarios: low-profit margins as the more challenging scenario and high-profit margins as the less difficult one. The EEG signals were acquired from healthy humans while subjects were performing the task. We adopted the Correlated Component Analysis (CorrCA) method to identify linear combinations of EEG channels that maximize the correlation across subjects ([Formula: see text]) or trials ([Formula: see text]). The inter-subject or inter-trial correlation values (ISC or ITC) of the first three components were estimated to investigate the modulation of the task difficulty on subjects' EEG signals and respective correlations. We also calculated the alpha- and beta-band power of the projection components obtained by the CorrCA to assess the brain responses across multiple task periods. Finally, the CorrCA forward models, which represent the scalp projections of the brain activities by the maximally correlated components, were further translated into source distributions of underlying cortical activity using the exact Low Resolution Electromagnetic Tomography Algorithm (eLORETA). Our results revealed strong and significant correlations in EEG signals among multiple subjects and trials during the more difficult decision-making task than the easier one. We also observed that the NP decision-making and feedback tasks desynchronized the normalized alpha and beta powers of the CorrCA components, reflecting the engagement state of subjects. Source localization results furthermore suggested several sources of neural activities during the NP decision-making process, including the dorsolateral prefrontal cortex, anterior PFC, orbitofrontal cortex, posterior cingulate cortex, and somatosensory association cortex.
Collapse
Affiliation(s)
- Nghi Cong Dung Truong
- Department of Bioengineering, University of Texas at Arlington, 500 UTA Blvd, Arlington, TX, 76019, USA
| | - Xinlong Wang
- Department of Bioengineering, University of Texas at Arlington, 500 UTA Blvd, Arlington, TX, 76019, USA
| | - Hashini Wanniarachchi
- Department of Bioengineering, University of Texas at Arlington, 500 UTA Blvd, Arlington, TX, 76019, USA
| | - Yan Lang
- Information Systems and Operations Management, University of Texas at Arlington, 701 S. Nedderman Drive, Arlington, TX, 76019, USA
- Department of Business, State University of New York at Oneonta, 108 Ravine Parkway Oneonta, New York, NY, 13820, USA
| | - Sridhar Nerur
- Information Systems and Operations Management, University of Texas at Arlington, 701 S. Nedderman Drive, Arlington, TX, 76019, USA
| | - Kay-Yut Chen
- Information Systems and Operations Management, University of Texas at Arlington, 701 S. Nedderman Drive, Arlington, TX, 76019, USA
| | - Hanli Liu
- Department of Bioengineering, University of Texas at Arlington, 500 UTA Blvd, Arlington, TX, 76019, USA.
| |
Collapse
|
22
|
Holtze B, Rosenkranz M, Jaeger M, Debener S, Mirkovic B. Ear-EEG Measures of Auditory Attention to Continuous Speech. Front Neurosci 2022; 16:869426. [PMID: 35592265 PMCID: PMC9111016 DOI: 10.3389/fnins.2022.869426] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/04/2022] [Accepted: 03/25/2022] [Indexed: 11/13/2022] Open
Abstract
Auditory attention is an important cognitive function used to separate relevant from irrelevant auditory information. However, most findings on attentional selection have been obtained in highly controlled laboratory settings using bulky recording setups and unnaturalistic stimuli. Recent advances in electroencephalography (EEG) facilitate the measurement of brain activity outside the laboratory, and around-the-ear sensors such as the cEEGrid promise unobtrusive acquisition. In parallel, methods such as speech envelope tracking, intersubject correlations and spectral entropy measures emerged which allow us to study attentional effects in the neural processing of natural, continuous auditory scenes. In the current study, we investigated whether these three attentional measures can be reliably obtained when using around-the-ear EEG. To this end, we analyzed the cEEGrid data of 36 participants who attended to one of two simultaneously presented speech streams. Speech envelope tracking results confirmed a reliable identification of the attended speaker from cEEGrid data. The accuracies in identifying the attended speaker increased when fitting the classification model to the individual. Artifact correction of the cEEGrid data with artifact subspace reconstruction did not increase the classification accuracy. Intersubject correlations were higher for those participants attending to the same speech stream than for those attending to different speech streams, replicating previously obtained results with high-density cap-EEG. We also found that spectral entropy decreased over time, possibly reflecting the decrease in the listener's level of attention. Overall, these results support the idea of using ear-EEG measurements to unobtrusively monitor auditory attention to continuous speech. This knowledge may help to develop assistive devices that support listeners separating relevant from irrelevant information in complex auditory environments.
Collapse
Affiliation(s)
- Björn Holtze
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
| | - Marc Rosenkranz
- Neurophysiology of Everyday Life Group, Department of Psychology, University of Oldenburg, Oldenburg, Germany
| | - Manuela Jaeger
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
- Division Hearing, Speech and Audio Technology, Fraunhofer Institute for Digital Media Technology IDMT, Oldenburg, Germany
| | - Stefan Debener
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
- Research Center for Neurosensory Science, University of Oldenburg, Oldenburg, Germany
- Cluster of Excellence Hearing4all, University of Oldenburg, Oldenburg, Germany
| | - Bojana Mirkovic
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
| |
Collapse
|
23
|
Nguyen M, Chang A, Micciche E, Meshulam M, Nastase SA, Hasson U. Teacher-student neural coupling during teaching and learning. Soc Cogn Affect Neurosci 2022; 17:367-376. [PMID: 34450637 PMCID: PMC8972247 DOI: 10.1093/scan/nsab103] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/05/2021] [Revised: 08/10/2021] [Accepted: 08/26/2021] [Indexed: 11/17/2022] Open
Abstract
Human communication is remarkably versatile, enabling teachers to share highly abstracted and novel information with their students. What neural processes enable such transfer of information across brains during naturalistic teaching and learning? Here, a teacher was scanned in functional magnetic resonance imaging while giving an oral lecture with slides on a scientific topic followed by a review lecture. Students were then scanned while watching either the intact Lecture and Review (N = 20) or a temporally scrambled version of the lecture (N = 20). Using intersubject correlation, we observed widespread Teacher-Student neural coupling spanning sensory cortex and language regions along the superior temporal sulcus as well as higher-level regions including posterior medial cortex (PMC), superior parietal lobule, and dorsolateral and dorsomedial prefrontal cortex. Teacher-student alignment in higher-level areas was not observed when learning was disrupted by temporally scrambling the lecture. Moreover, teacher-student coupling in PMC was significantly correlated with learning: the more closely the student's brain mirrored the teacher's brain, the more the student improved their learning score. Together, these results suggest that the alignment of neural responses between teacher and students may reflect effective communication of complex information across brains in classroom settings.
Collapse
Affiliation(s)
- Mai Nguyen
- Department of Psychology, Princeton University, Princeton, NJ 08540, USA
| | - Ashley Chang
- Department of Psychology, Princeton University, Princeton, NJ 08540, USA
| | - Emily Micciche
- Department of Psychology, Vanderbilt University, Nashville, TN 37240, USA
| | - Meir Meshulam
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08540, USA
| | - Samuel A Nastase
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08540, USA
| | - Uri Hasson
- Department of Psychology, Princeton University, Princeton, NJ 08540, USA
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08540, USA
| |
Collapse
|
24
|
Irsik VC, Johnsrude IS, Herrmann B. Neural Activity during Story Listening Is Synchronized across Individuals Despite Acoustic Masking. J Cogn Neurosci 2022; 34:933-950. [PMID: 35258555 DOI: 10.1162/jocn_a_01842] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Older people with hearing problems often experience difficulties understanding speech in the presence of background sound. As a result, they may disengage in social situations, which has been associated with negative psychosocial health outcomes. Measuring listening (dis)engagement during challenging listening situations has received little attention thus far. We recruit young, normal-hearing human adults (both sexes) and investigate how speech intelligibility and engagement during naturalistic story listening is affected by the level of acoustic masking (12-talker babble) at different signal-to-noise ratios (SNRs). In Experiment 1, we observed that word-report scores were above 80% for all but the lowest SNR (-3 dB SNR) we tested, at which performance dropped to 54%. In Experiment 2, we calculated intersubject correlation (ISC) using EEG data to identify dynamic spatial patterns of shared neural activity evoked by the stories. ISC has been used as a neural measure of participants' engagement with naturalistic materials. Our results show that ISC was stable across all but the lowest SNRs, despite reduced speech intelligibility. Comparing ISC and intelligibility demonstrated that word-report performance declined more strongly with decreasing SNR compared to ISC. Our measure of neural engagement suggests that individuals remain engaged in story listening despite missing words because of background noise. Our work provides a potentially fruitful approach to investigate listener engagement with naturalistic, spoken stories that may be used to investigate (dis)engagement in older adults with hearing impairment.
Collapse
Affiliation(s)
| | | | - Björn Herrmann
- The University of Western Ontario.,Rotman Research Institute, Toronto, ON, Canada.,University of Toronto
| |
Collapse
|
25
|
Madsen J, Parra LC. Cognitive processing of a common stimulus synchronizes brains, hearts, and eyes. PNAS NEXUS 2022; 1:pgac020. [PMID: 36712806 PMCID: PMC9802497 DOI: 10.1093/pnasnexus/pgac020] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 01/14/2022] [Accepted: 03/01/2022] [Indexed: 04/21/2023]
Abstract
Neural, physiological, and behavioral signals synchronize between human subjects in a variety of settings. Multiple hypotheses have been proposed to explain this interpersonal synchrony, but there is no clarity under which conditions it arises, for which signals, or whether there is a common underlying mechanism. We hypothesized that cognitive processing of a shared stimulus is the source of synchrony between subjects, measured here as intersubject correlation (ISC). To test this, we presented informative videos to participants in an attentive and distracted condition and subsequently measured information recall. ISC was observed for electro-encephalography, gaze position, pupil size, and heart rate, but not respiration and head movements. The strength of correlation was co-modulated in the different signals, changed with attentional state, and predicted subsequent recall of information presented in the videos. There was robust within-subject coupling between brain, heart, and eyes, but not respiration or head movements. The results suggest that ISC is the result of effective cognitive processing, and thus emerges only for those signals that exhibit a robust brain-body connection. While physiological and behavioral fluctuations may be driven by multiple features of the stimulus, correlation with other individuals is co-modulated by the level of attentional engagement with the stimulus.
Collapse
Affiliation(s)
| | - Lucas C Parra
- Department of Biomedical Engineering, City College of New York, 160 Convent Avenue, New York, NY 10031, USA
| |
Collapse
|
26
|
Jääskeläinen IP, Ahveninen J, Klucharev V, Shestakova AN, Levy J. Behavioral Experience-Sampling Methods in Neuroimaging Studies With Movie and Narrative Stimuli. Front Hum Neurosci 2022; 16:813684. [PMID: 35153706 PMCID: PMC8828971 DOI: 10.3389/fnhum.2022.813684] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/12/2021] [Accepted: 01/06/2022] [Indexed: 01/22/2023] Open
Abstract
Movies and narratives are increasingly utilized as stimuli in functional magnetic resonance imaging (fMRI), magnetoencephalography (MEG), and electroencephalography (EEG) studies. Emotional reactions of subjects, what they pay attention to, what they memorize, and their cognitive interpretations are all examples of inner experiences that can differ between subjects during watching of movies and listening to narratives inside the scanner. Here, we review literature indicating that behavioral measures of inner experiences play an integral role in this new research paradigm via guiding neuroimaging analysis. We review behavioral methods that have been developed to sample inner experiences during watching of movies and listening to narratives. We also review approaches that allow for joint analyses of the behaviorally sampled inner experiences and neuroimaging data. We suggest that building neurophenomenological frameworks holds potential for solving the interrelationships between inner experiences and their neural underpinnings. Finally, we tentatively suggest that recent developments in machine learning approaches may pave way for inferring different classes of inner experiences directly from the neuroimaging data, thus potentially complementing the behavioral self-reports.
Collapse
Affiliation(s)
- Iiro P. Jääskeläinen
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
- International Laboratory of Social Neurobiology, Institute of Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
- *Correspondence: Iiro P. Jääskeläinen,
| | - Jyrki Ahveninen
- Massachusetts General Hospital, Massachusetts Institute of Technology and Harvard Medical School, Athinoula A. Martinos Center for Biomedical Imaging, Charlestown, MA, United States
| | - Vasily Klucharev
- International Laboratory of Social Neurobiology, Institute of Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
| | - Anna N. Shestakova
- International Laboratory of Social Neurobiology, Institute of Cognitive Neuroscience, National Research University Higher School of Economics, Moscow, Russia
| | - Jonathan Levy
- Brain and Mind Laboratory, Department of Neuroscience and Biomedical Engineering, Aalto University School of Science, Espoo, Finland
- Baruch Ivcher School of Psychology, Interdisciplinary Center Herzliya, Reichman University, Herzliya, Israel
| |
Collapse
|
27
|
Stuldreher IV, Merasli A, Thammasan N, van Erp JBF, Brouwer AM. Unsupervised Clustering of Individuals Sharing Selective Attentional Focus Using Physiological Synchrony. FRONTIERS IN NEUROERGONOMICS 2022; 2:750248. [PMID: 38235215 PMCID: PMC10790845 DOI: 10.3389/fnrgo.2021.750248] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Accepted: 12/09/2021] [Indexed: 01/19/2024]
Abstract
Research on brain signals as indicators of a certain attentional state is moving from laboratory environments to everyday settings. Uncovering the attentional focus of individuals in such settings is challenging because there is usually limited information about real-world events, as well as a lack of data from the real-world context at hand that is correctly labeled with respect to individuals' attentional state. In most approaches, such data is needed to train attention monitoring models. We here investigate whether unsupervised clustering can be combined with physiological synchrony in the electroencephalogram (EEG), electrodermal activity (EDA), and heart rate to automatically identify groups of individuals sharing attentional focus without using knowledge of the sensory stimuli or attentional focus of any of the individuals. We used data from an experiment in which 26 participants listened to an audiobook interspersed with emotional sounds and beeps. Thirteen participants were instructed to focus on the narrative of the audiobook and 13 participants were instructed to focus on the interspersed emotional sounds and beeps. We used a broad range of commonly applied dimensionality reduction ordination techniques-further referred to as mappings-in combination with unsupervised clustering algorithms to identify the two groups of individuals sharing attentional focus based on physiological synchrony. Analyses were performed using the three modalities EEG, EDA, and heart rate separately, and using all possible combinations of these modalities. The best unimodal results were obtained when applying clustering algorithms on physiological synchrony data in EEG, yielding a maximum clustering accuracy of 85%. Even though the use of EDA or heart rate by itself did not lead to accuracies significantly higher than chance level, combining EEG with these measures in a multimodal approach generally resulted in higher classification accuracies than when using only EEG. Additionally, classification results of multimodal data were found to be more consistent across algorithms than unimodal data, making algorithm choice less important. Our finding that unsupervised classification into attentional groups is possible is important to support studies on attentional engagement in everyday settings.
Collapse
Affiliation(s)
- Ivo V. Stuldreher
- TNO Human Factors, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
| | - Alexandre Merasli
- TNO Human Factors, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
| | - Nattapong Thammasan
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
| | - Jan B. F. van Erp
- TNO Human Factors, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
| | - Anne-Marie Brouwer
- TNO Human Factors, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
| |
Collapse
|
28
|
Dauer T, Nguyen DT, Gang N, Dmochowski JP, Berger J, Kaneshiro B. Inter-subject Correlation While Listening to Minimalist Music: A Study of Electrophysiological and Behavioral Responses to Steve Reich's Piano Phase. Front Neurosci 2021; 15:702067. [PMID: 34955706 PMCID: PMC8695499 DOI: 10.3389/fnins.2021.702067] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2021] [Accepted: 11/18/2021] [Indexed: 12/03/2022] Open
Abstract
Musical minimalism utilizes the temporal manipulation of restricted collections of rhythmic, melodic, and/or harmonic materials. One example, Steve Reich's Piano Phase, offers listeners readily audible formal structure with unpredictable events at the local level. For example, pattern recurrences may generate strong expectations which are violated by small temporal and pitch deviations. A hyper-detailed listening strategy prompted by these minute deviations stands in contrast to the type of listening engagement typically cultivated around functional tonal Western music. Recent research has suggested that the inter-subject correlation (ISC) of electroencephalographic (EEG) responses to natural audio-visual stimuli objectively indexes a state of “engagement,” demonstrating the potential of this approach for analyzing music listening. But can ISCs capture engagement with minimalist music, which features less obvious expectation formation and has historically received a wide range of reactions? To approach this question, we collected EEG and continuous behavioral (CB) data while 30 adults listened to an excerpt from Steve Reich's Piano Phase, as well as three controlled manipulations and a popular-music remix of the work. Our analyses reveal that EEG and CB ISC are highest for the remix stimulus and lowest for our most repetitive manipulation, no statistical differences in overall EEG ISC between our most musically meaningful manipulations and Reich's original piece, and evidence that compositional features drove engagement in time-resolved ISC analyses. We also found that aesthetic evaluations corresponded well with overall EEG ISC. Finally we highlight co-occurrences between stimulus events and time-resolved EEG and CB ISC. We offer the CB paradigm as a useful analysis measure and note the value of minimalist compositions as a limit case for the neuroscientific study of music listening. Overall, our participants' neural, continuous behavioral, and question responses showed strong similarities that may help refine our understanding of the type of engagement indexed by ISC for musical stimuli.
Collapse
Affiliation(s)
- Tysen Dauer
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, United States.,Rotman Research Institute, Baycrest Centre, Toronto, ON, Canada
| | - Duc T Nguyen
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, United States.,Center for the Study of Language and Information, Stanford University, Stanford, CA, United States
| | - Nick Gang
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, United States
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of New York, New York, NY, United States
| | - Jonathan Berger
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, United States
| | - Blair Kaneshiro
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, United States.,Center for the Study of Language and Information, Stanford University, Stanford, CA, United States
| |
Collapse
|
29
|
Janssen TW, Grammer JK, Bleichner MG, Bulgarelli C, Davidesco I, Dikker S, Jasińska KK, Siugzdaite R, Vassena E, Vatakis A, Zion‐Golumbic E, van Atteveldt N. Opportunities and Limitations of Mobile Neuroimaging Technologies in Educational Neuroscience. MIND, BRAIN AND EDUCATION : THE OFFICIAL JOURNAL OF THE INTERNATIONAL MIND, BRAIN, AND EDUCATION SOCIETY 2021; 15:354-370. [PMID: 35875415 PMCID: PMC9292610 DOI: 10.1111/mbe.12302] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Revised: 06/18/2021] [Accepted: 09/01/2021] [Indexed: 05/20/2023]
Abstract
As the field of educational neuroscience continues to grow, questions have emerged regarding the ecological validity and applicability of this research to educational practice. Recent advances in mobile neuroimaging technologies have made it possible to conduct neuroscientific studies directly in naturalistic learning environments. We propose that embedding mobile neuroimaging research in a cycle (Matusz, Dikker, Huth, & Perrodin, 2019), involving lab-based, seminaturalistic, and fully naturalistic experiments, is well suited for addressing educational questions. With this review, we take a cautious approach, by discussing the valuable insights that can be gained from mobile neuroimaging technology, including electroencephalography and functional near-infrared spectroscopy, as well as the challenges posed by bringing neuroscientific methods into the classroom. Research paradigms used alongside mobile neuroimaging technology vary considerably. To illustrate this point, studies are discussed with increasingly naturalistic designs. We conclude with several ethical considerations that should be taken into account in this unique area of research.
Collapse
Affiliation(s)
- Tieme W.P. Janssen
- Department of Clinical, Neuro‐ & Developmental Psychology, Vrije Universiteit
| | - Jennie K. Grammer
- Graduate School of Education and Information Studies, University of California Los Angeles
| | | | - Chiara Bulgarelli
- Centre for Brain and Cognitive Development, Birkbeck University of London
| | - Ido Davidesco
- Department of Educational Psychology, University of Connecticut
| | | | - Kaja K. Jasińska
- Department of Applied Psychology and Human Development, University of Toronto
| | | | - Eliana Vassena
- Donders Institute for Brain, Cognition and Behaviour, Radboud University
| | | | | | | |
Collapse
|
30
|
Pérez P, Madsen J, Banellis L, Türker B, Raimondo F, Perlbarg V, Valente M, Niérat MC, Puybasset L, Naccache L, Similowski T, Cruse D, Parra LC, Sitt JD. Conscious processing of narrative stimuli synchronizes heart rate between individuals. Cell Rep 2021; 36:109692. [PMID: 34525363 DOI: 10.1016/j.celrep.2021.109692] [Citation(s) in RCA: 33] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2020] [Revised: 06/15/2021] [Accepted: 08/19/2021] [Indexed: 10/20/2022] Open
Abstract
Heart rate has natural fluctuations that are typically ascribed to autonomic function. Recent evidence suggests that conscious processing can affect the timing of the heartbeat. We hypothesized that heart rate is modulated by conscious processing and therefore dependent on attentional focus. To test this, we leverage the observation that neural processes synchronize between subjects by presenting an identical narrative stimulus. As predicted, we find significant inter-subject correlation of heart rate (ISC-HR) when subjects are presented with an auditory or audiovisual narrative. Consistent with our hypothesis, we find that ISC-HR is reduced when subjects are distracted from the narrative, and higher ISC-HR predicts better recall of the narrative. Finally, patients with disorders of consciousness have lower ISC-HR, as compared to healthy individuals. We conclude that heart rate fluctuations are partially driven by conscious processing, depend on attentional state, and may represent a simple metric to assess conscious state in unresponsive patients.
Collapse
Affiliation(s)
- Pauline Pérez
- Paris Brain Institute, ICM, 75013 Paris, France; Inserm U 1127, 75013 Paris, France; CNRS UMR 7225, 75013 Paris, France
| | - Jens Madsen
- Department of Biomedical Engineering, City College of New York, New York, NY, USA
| | - Leah Banellis
- School of Psychology, University of Birmingham, Birmingham B15 2TT, England; Centre for Human Brain Health, University of Birmingham, Birmingham B15 2TT, England
| | - Bașak Türker
- Paris Brain Institute, ICM, 75013 Paris, France; Inserm U 1127, 75013 Paris, France; CNRS UMR 7225, 75013 Paris, France
| | | | - Vincent Perlbarg
- Bioinformatics and Biostatistics Core Facility, iCONICS, IHU-A-ICM, Institut du Cerveau et de la Moelle épinière, Paris, France
| | | | - Marie-Cécile Niérat
- Sorbonne Université, INSERM, UMRS1158 Neurophysiologie Respiratoire Expérimentale et Clinique, 75005 Paris, France
| | - Louis Puybasset
- Department of Anesthesia and Intensive Care, Pitié-Salpetrière Hospital, Paris, France; Sorbonne Université, CNRS, INSERM, Laboratoire d'Imagerie Biomédicale, LIB, Paris, France
| | - Lionel Naccache
- Paris Brain Institute, ICM, 75013 Paris, France; Inserm U 1127, 75013 Paris, France; CNRS UMR 7225, 75013 Paris, France; Department of Neurophysiology, Hôpital de la Pitié-Salpetrière, AP-HP, 75013 Paris, France; Faculté de Médecine Pitié-Salpêtrière, Sorbonne Université, 75013 Paris, France
| | - Thomas Similowski
- Sorbonne Université, INSERM, UMRS1158 Neurophysiologie Respiratoire Expérimentale et Clinique, 75005 Paris, France; AP-HP, Groupe Hospitalier Universitaire APHP-Sorbonne Université, site Pitié-Salpêtrière, Service de Pneumologie, Médecine Intensive et Réanimation (Département R3S), 75013 Paris, France
| | - Damian Cruse
- School of Psychology, University of Birmingham, Birmingham B15 2TT, England; Centre for Human Brain Health, University of Birmingham, Birmingham B15 2TT, England
| | - Lucas C Parra
- Department of Biomedical Engineering, City College of New York, New York, NY, USA.
| | - Jacobo D Sitt
- Paris Brain Institute, ICM, 75013 Paris, France; Inserm U 1127, 75013 Paris, France; CNRS UMR 7225, 75013 Paris, France.
| |
Collapse
|
31
|
Song H, Finn ES, Rosenberg MD. Neural signatures of attentional engagement during narratives and its consequences for event memory. Proc Natl Acad Sci U S A 2021; 118:e2021905118. [PMID: 34385312 PMCID: PMC8379980 DOI: 10.1073/pnas.2021905118] [Citation(s) in RCA: 30] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
As we comprehend narratives, our attentional engagement fluctuates over time. Despite theoretical conceptions of narrative engagement as emotion-laden attention, little empirical work has characterized the cognitive and neural processes that comprise subjective engagement in naturalistic contexts or its consequences for memory. Here, we relate fluctuations in narrative engagement to patterns of brain coactivation and test whether neural signatures of engagement predict subsequent memory. In behavioral studies, participants continuously rated how engaged they were as they watched a television episode or listened to a story. Self-reported engagement was synchronized across individuals and driven by the emotional content of the narratives. In functional MRI datasets collected as different individuals watched the same show or listened to the same story, engagement drove neural synchrony, such that default mode network activity was more synchronized across individuals during more engaging moments of the narratives. Furthermore, models based on time-varying functional brain connectivity predicted evolving states of engagement across participants and independent datasets. The functional connections that predicted engagement overlapped with a validated neuromarker of sustained attention and predicted recall of narrative events. Together, our findings characterize the neural signatures of attentional engagement in naturalistic contexts and elucidate relationships among narrative engagement, sustained attention, and event memory.
Collapse
Affiliation(s)
- Hayoung Song
- Department of Psychology, University of Chicago, Chicago, IL 60637;
| | - Emily S Finn
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH 03755
| | - Monica D Rosenberg
- Department of Psychology, University of Chicago, Chicago, IL 60637;
- Neuroscience Institute, University of Chicago, Chicago, IL 60637
| |
Collapse
|
32
|
Rosenkranz M, Holtze B, Jaeger M, Debener S. EEG-Based Intersubject Correlations Reflect Selective Attention in a Competing Speaker Scenario. Front Neurosci 2021; 15:685774. [PMID: 34194296 PMCID: PMC8236636 DOI: 10.3389/fnins.2021.685774] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/25/2021] [Accepted: 05/18/2021] [Indexed: 11/13/2022] Open
Abstract
Several solutions have been proposed to study the relationship between ongoing brain activity and natural sensory stimuli, such as running speech. Computing the intersubject correlation (ISC) has been proposed as one possible approach. Previous evidence suggests that ISCs between the participants' electroencephalogram (EEG) may be modulated by attention. The current study addressed this question in a competing-speaker paradigm, where participants (N = 41) had to attend to one of two concurrently presented speech streams. ISCs between participants' EEG were higher for participants attending to the same story compared to participants attending to different stories. Furthermore, we found that ISCs between individual and group data predicted whether an individual attended to the left or right speech stream. Interestingly, the magnitude of the shared neural response with others attending to the same story was related to the individual neural representation of the attended and ignored speech envelope. Overall, our findings indicate that ISC differences reflect the magnitude of selective attentional engagement to speech.
Collapse
Affiliation(s)
- Marc Rosenkranz
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
| | - Björn Holtze
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany
| | - Manuela Jaeger
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany.,Division Hearing, Fraunhofer Institute for Digital Media Technology IDMT, Speech and Audio Technology, Oldenburg, Germany
| | - Stefan Debener
- Neuropsychology Lab, Department of Psychology, University of Oldenburg, Oldenburg, Germany.,Cluster of Excellence Hearing4all, University of Oldenburg, Oldenburg, Germany.,Research Center for Neurosensory Science, University of Oldenburg, Oldenburg, Germany
| |
Collapse
|
33
|
Irsik VC, Almanaseer A, Johnsrude IS, Herrmann B. Cortical Responses to the Amplitude Envelopes of Sounds Change with Age. J Neurosci 2021; 41:5045-5055. [PMID: 33903222 PMCID: PMC8197634 DOI: 10.1523/jneurosci.2715-20.2021] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2020] [Revised: 02/16/2021] [Accepted: 03/26/2021] [Indexed: 11/21/2022] Open
Abstract
Many older listeners have difficulty understanding speech in noise, when cues to speech-sound identity are less redundant. The amplitude envelope of speech fluctuates dramatically over time, and features such as the rate of amplitude change at onsets (attack) and offsets (decay), signal critical information about the identity of speech sounds. Aging is also thought to be accompanied by increases in cortical excitability, which may differentially alter sensitivity to envelope dynamics. Here, we recorded electroencephalography in younger and older human adults (of both sexes) to investigate how aging affects neural synchronization to 4 Hz amplitude-modulated noises with different envelope shapes (ramped: slow attack and sharp decay; damped: sharp attack and slow decay). We observed that subcortical responses did not differ between age groups, whereas older compared with younger adults exhibited larger cortical responses to sound onsets, consistent with an increase in auditory cortical excitability. Neural activity in older adults synchronized more strongly to rapid-onset, slow-offset (damped) envelopes, was less sinusoidal, and was more peaked. Younger adults demonstrated the opposite pattern, showing stronger synchronization to slow-onset, rapid-offset (ramped) envelopes, as well as a more sinusoidal neural response shape. The current results suggest that age-related changes in the excitability of auditory cortex alter responses to envelope dynamics. This may be part of the reason why older adults experience difficulty understanding speech in noise.SIGNIFICANCE STATEMENT Many middle-aged and older adults report difficulty understanding speech when there is background noise, which can trigger social withdrawal and negative psychosocial health outcomes. The difficulty may be related to age-related changes in how the brain processes temporal sound features. We tested younger and older people on their sensitivity to different envelope shapes, using EEG. Our results demonstrate that aging is associated with heightened sensitivity to sounds with a sharp attack and gradual decay, and sharper neural responses that deviate from the sinusoidal features of the stimulus, perhaps reflecting increased excitability in the aged auditory cortex. Altered responses to temporal sound features may be part of the reason why older adults often experience difficulty understanding speech in social situations.
Collapse
Affiliation(s)
- Vanessa C Irsik
- Department of Psychology & the Brain and Mind Institute, University of Western Ontario, London, Ontario N6A 3K7, Canada
| | - Ala Almanaseer
- Department of Psychology & the Brain and Mind Institute, University of Western Ontario, London, Ontario N6A 3K7, Canada
| | - Ingrid S Johnsrude
- Department of Psychology & the Brain and Mind Institute, University of Western Ontario, London, Ontario N6A 3K7, Canada
- School of Communication and Speech Disorders, University of Western Ontario, London, Ontario N6A 5B7, Canada
| | - Björn Herrmann
- Department of Psychology & the Brain and Mind Institute, University of Western Ontario, London, Ontario N6A 3K7, Canada
- Rotman Research Institute Baycrest, Toronto, Ontario M6A 2E1, Canada
- Department of Psychology, University of Toronto, Toronto, Ontario M5S 1A1, Canada
| |
Collapse
|
34
|
Yeshurun Y, Nguyen M, Hasson U. The default mode network: where the idiosyncratic self meets the shared social world. Nat Rev Neurosci 2021; 22:181-192. [PMID: 33483717 PMCID: PMC7959111 DOI: 10.1038/s41583-020-00420-w] [Citation(s) in RCA: 228] [Impact Index Per Article: 76.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 12/10/2020] [Indexed: 01/29/2023]
Abstract
The default mode network (DMN) is classically considered an 'intrinsic' system, specializing in internally oriented cognitive processes such as daydreaming, reminiscing and future planning. In this Perspective, we suggest that the DMN is an active and dynamic 'sense-making' network that integrates incoming extrinsic information with prior intrinsic information to form rich, context-dependent models of situations as they unfold over time. We review studies that relied on naturalistic stimuli, such as stories and movies, to demonstrate how an individual's DMN neural responses are influenced both by external information accumulated as events unfold over time and by the individual's idiosyncratic past memories and knowledge. The integration of extrinsic and intrinsic information over long timescales provides a space for negotiating a shared neural code, which is necessary for establishing shared meaning, shared communication tools, shared narratives and, above all, shared communities and social networks.
Collapse
Affiliation(s)
- Yaara Yeshurun
- School of Psychological Sciences, Tel-Aviv University, Tel-Aviv, Israel.
- Sagol School of Neuroscience, Tel-Aviv University, Tel-Aviv, Israel.
| | - Mai Nguyen
- School of Psychological Sciences, Tel-Aviv University, Tel-Aviv, Israel
- Sagol School of Neuroscience, Tel-Aviv University, Tel-Aviv, Israel
- Department of Psychology, Princeton University, Princeton, NJ, USA
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Uri Hasson
- School of Psychological Sciences, Tel-Aviv University, Tel-Aviv, Israel.
- Sagol School of Neuroscience, Tel-Aviv University, Tel-Aviv, Israel.
- Department of Psychology, Princeton University, Princeton, NJ, USA.
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA.
| |
Collapse
|
35
|
Wisdom of crowds benefits perceptual decision making across difficulty levels. Sci Rep 2021; 11:538. [PMID: 33436921 PMCID: PMC7804123 DOI: 10.1038/s41598-020-80500-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/15/2020] [Accepted: 12/11/2020] [Indexed: 11/29/2022] Open
Abstract
Decades of research on collective decision making has claimed that aggregated judgment of multiple individuals is more accurate than expert individual judgement. A longstanding problem in this regard has been to determine how decisions of individuals can be combined to form intelligent group decisions. Our study consisted of a random target detection task in natural scenes, where human subjects (18 subjects, 7 female) detected the presence or absence of a random target as indicated by the cue word displayed prior to stimulus display. Concurrently the neural activities (EEG signals) were recorded. A separate behavioural experiment was performed by different subjects (20 subjects, 11 female) on the same set of images to categorize the tasks according to their difficulty levels. We demonstrate that the weighted average of individual decision confidence/neural decision variables produces significantly better performance than the frequently used majority pooling algorithm. Further, the classification error rates from individual judgement were found to increase with increasing task difficulty. This error could be significantly reduced upon combining the individual decisions using group aggregation rules. Using statistical tests, we show that combining all available participants is unnecessary to achieve minimum classification error rate. We also try to explore if group aggregation benefits depend on the correlation between the individual judgements of the group and our results seem to suggest that reduced inter-subject correlation can improve collective decision making for a fixed difficulty level.
Collapse
|
36
|
Dikker S, Haegens S, Bevilacqua D, Davidesco I, Wan L, Kaggen L, McClintock J, Chaloner K, Ding M, West T, Poeppel D. Morning brain: real-world neural evidence that high school class times matter. Soc Cogn Affect Neurosci 2020; 15:1193-1202. [PMID: 33068110 PMCID: PMC7745151 DOI: 10.1093/scan/nsaa142] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Revised: 08/14/2020] [Accepted: 10/16/2020] [Indexed: 12/20/2022] Open
Abstract
Researchers, parents and educators consistently observe a stark mismatch between biologically preferred and socially imposed sleep-wake hours in adolescents, fueling debate about high school start times. We contribute neural evidence to this debate with electroencephalogram data collected from high school students during their regular morning, mid-morning and afternoon classes. Overall, student alpha power was lower when class content was taught via videos than through lectures. Students' resting state alpha brain activity decreased as the day progressed, consistent with adolescents being least attentive early in the morning. During the lessons, students showed consistently worse performance and higher alpha power for early morning classes than for mid-morning classes, while afternoon quiz scores and alpha levels varied. Together, our findings demonstrate that both class activity and class time are reflected in adolescents' brain states in a real-world setting, and corroborate educational research suggesting that mid-morning may be the best time to learn.
Collapse
Affiliation(s)
- Suzanne Dikker
- Max Planck-NYU Center for Language, Music and Emotion, New York, NY, USA
- Department of Psychology, New York University, New York, NY, USA
| | - Saskia Haegens
- Department of Psychiatry, Division of Systems Neuroscience, Columbia University and the Research Foundation for Mental Hygiene, New York State Psychiatric Institute, New York, NY, USA
- Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, The Netherlands
| | - Dana Bevilacqua
- Max Planck-NYU Center for Language, Music and Emotion, New York, NY, USA
- Department of Psychology, New York University, New York, NY, USA
| | - Ido Davidesco
- Department of Psychology, New York University, New York, NY, USA
| | - Lu Wan
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA
| | - Lisa Kaggen
- Department of Psychology, New York University, New York, NY, USA
| | | | | | - Mingzhou Ding
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA
| | - Tessa West
- Department of Psychology, New York University, New York, NY, USA
| | - David Poeppel
- Max Planck-NYU Center for Language, Music and Emotion, New York, NY, USA
- Department of Psychology, New York University, New York, NY, USA
- Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany
| |
Collapse
|
37
|
Stuldreher IV, Thammasan N, van Erp JBF, Brouwer AM. Physiological Synchrony in EEG, Electrodermal Activity and Heart Rate Detects Attentionally Relevant Events in Time. Front Neurosci 2020; 14:575521. [PMID: 33343277 PMCID: PMC7744592 DOI: 10.3389/fnins.2020.575521] [Citation(s) in RCA: 12] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/23/2020] [Accepted: 11/10/2020] [Indexed: 11/13/2022] Open
Abstract
Interpersonal physiological synchrony (PS), or the similarity of physiological signals between individuals over time, may be used to detect attentionally engaging moments in time. We here investigated whether PS in the electroencephalogram (EEG), electrodermal activity (EDA), heart rate and a multimodal metric signals the occurrence of attentionally relevant events in time in two groups of participants. Both groups were presented with the same auditory stimulus, but were instructed to attend either to the narrative of an audiobook (audiobook-attending: AA group) or to interspersed emotional sounds and beeps (stimulus-attending: SA group). We hypothesized that emotional sounds could be detected in both groups as they are expected to draw attention involuntarily, in a bottom-up fashion. Indeed, we found this to be the case for PS in EDA or the multimodal metric. Beeps, that are expected to be only relevant due to specific "top-down" attentional instructions, could indeed only be detected using PS among SA participants, for EDA, EEG and the multimodal metric. We further hypothesized that moments in the audiobook accompanied by high PS in either EEG, EDA, heart rate or the multimodal metric for AA participants would be rated as more engaging by an independent group of participants compared to moments corresponding to low PS. This hypothesis was not supported. Our results show that PS can support the detection of attentionally engaging events over time. Currently, the relation between PS and engagement is only established for well-defined, interspersed stimuli, whereas the relation between PS and a more abstract self-reported metric of engagement over time has not been established. As the relation between PS and engagement is dependent on event type and physiological measure, we suggest to choose a measure matching with the stimulus of interest. When the stimulus type is unknown, a multimodal metric is most robust.
Collapse
Affiliation(s)
- Ivo V. Stuldreher
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
| | - Nattapong Thammasan
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
| | - Jan B. F. van Erp
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
- Human Media Interaction, Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, Netherlands
| | - Anne-Marie Brouwer
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, Netherlands
| |
Collapse
|
38
|
Duclos C, Norton L, Laforge G, Frantz A, Maschke C, Badawy M, Letourneau J, Slessarev M, Gofton T, Debicki D, Owen AM, Blain-Moraes S. Protocol for the Prognostication of Consciousness Recovery Following a Brain Injury. Front Hum Neurosci 2020; 14:582125. [PMID: 33281582 PMCID: PMC7690215 DOI: 10.3389/fnhum.2020.582125] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/10/2020] [Accepted: 10/20/2020] [Indexed: 11/13/2022] Open
Abstract
Individuals who have suffered a severe brain injury typically require extensive hospitalization in intensive care units (ICUs), where critical treatment decisions are made to maximize their likelihood of recovering consciousness and cognitive function. These treatment decisions can be difficult when the neurological assessment of the patient is limited by unreliable behavioral responses. Reliable objective and quantifiable markers are lacking and there is both (1) a poor understanding of the mechanisms underlying the brain's ability to reconstitute consciousness and cognition after an injury and (2) the absence of a reliable and clinically feasible method of tracking cognitive recovery in ICU survivors. Our goal is to develop and validate a clinically relevant EEG paradigm that can inform the prognosis of unresponsive, brain-injured patients in the ICU. This protocol describes a study to develop a point-of-care system intended to accurately predict outcomes of unresponsive, brain-injured patients in the ICU. We will recruit 200 continuously-sedated brain-injured patients across five ICUs. Between 24 h and 7 days post-ICU admission, high-density EEG will be recorded from behaviorally unresponsive patients before, during and after a brief cessation of pharmacological sedation. Once patients have reached the waking stage, they will be asked to complete an abridged Cambridge Brain Sciences battery, a web-based series of neurocognitive tests. The test series will be repeated every day during acute admission (ICU, ward), or as often as possible given the constraints of ICU and ward care. Following discharge, patients will continue to complete the same test series on weekly, and then monthly basis, for up to 12 months following injury. Functional outcomes will also be assessed up to 12 months post-injury. We anticipate our findings will lead to an increased ability to identify patients, as soon as possible after their brain injury, who are most likely to survive, and to make accurate predictions about their long-term cognitive and functional outcome. In addition to providing critically needed support for clinical decision-making, this study has the potential to transform our understanding of key functional EEG networks associated with consciousness and cognition.
Collapse
Affiliation(s)
- Catherine Duclos
- School of Physical and Occupational Therapy, McGill University, Montreal, QC, Canada
- Montreal General Hospital, McGill University Health Centre, Montreal, QC, Canada
| | - Loretta Norton
- Department of Psychology, King’s University College at Western University, London, ON, Canada
- The Brain and Mind Institute, Western University, London, ON, Canada
| | - Geoffrey Laforge
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
| | - Allison Frantz
- Montreal General Hospital, McGill University Health Centre, Montreal, QC, Canada
- Integrated Program in Neuroscience, McGill University, Montreal, QC, Canada
| | - Charlotte Maschke
- Montreal General Hospital, McGill University Health Centre, Montreal, QC, Canada
- Integrated Program in Neuroscience, McGill University, Montreal, QC, Canada
| | - Mohamed Badawy
- Department of Anesthesia, McGill University, Montreal, QC, Canada
- Department of Neurology and Neurosurgery, Division of Neurocritical Care, McGill University Health Center, Montreal, QC, Canada
- Montreal Neurological Hospital, McGill University Health Center, Montreal, QC, Canada
| | - Justin Letourneau
- Department of Anesthesia, McGill University, Montreal, QC, Canada
- Department of Neurology and Neurosurgery, Division of Neurocritical Care, McGill University Health Center, Montreal, QC, Canada
- Montreal Neurological Hospital, McGill University Health Center, Montreal, QC, Canada
| | - Marat Slessarev
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Teneille Gofton
- Department of Clinical Neurological Sciences, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Derek Debicki
- The Brain and Mind Institute, Western University, London, ON, Canada
- Neurocritical Care Program, Division of Neurology, Department of Clinical Neurological Sciences, Western University, London, ON, Canada
| | - Adrian M. Owen
- The Brain and Mind Institute, Western University, London, ON, Canada
- Department of Psychology, Western University, London, ON, Canada
- Department of Physiology and Pharmacology, Western University, London, ON, Canada
| | - Stefanie Blain-Moraes
- School of Physical and Occupational Therapy, McGill University, Montreal, QC, Canada
- Montreal General Hospital, McGill University Health Centre, Montreal, QC, Canada
| |
Collapse
|
39
|
Herrmann B, Johnsrude IS. A model of listening engagement (MoLE). Hear Res 2020; 397:108016. [DOI: 10.1016/j.heares.2020.108016] [Citation(s) in RCA: 28] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/21/2019] [Revised: 04/28/2020] [Accepted: 06/02/2020] [Indexed: 12/30/2022]
|
40
|
Laforge G, Gonzalez-Lara LE, Owen AM, Stojanoski B. Individualized assessment of residual cognition in patients with disorders of consciousness. Neuroimage Clin 2020; 28:102472. [PMID: 33395966 PMCID: PMC7652775 DOI: 10.1016/j.nicl.2020.102472] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Revised: 10/01/2020] [Accepted: 10/13/2020] [Indexed: 12/02/2022]
Abstract
Patients diagnosed with disorders of consciousness show minimal or inconsistent behavioural evidence of conscious awareness. However, using functional neuroimaging, recent research in clinical neuroscience has identified a subpopulation of these patients who reliably produce neural markers indicative of awareness. In this study, we recorded electroencephalograms during a response-free movie task to assess narrative processing in patients with disorders of consciousness. Thirteen patients diagnosed with a disorder of consciousness and 28 healthy controls participated in this study. We designed a movie-watching/listening paradigm involving two suspenseful movie clips, one audiovisual and one audio-only, and used electroencephalography to extract patterns of brain activity that were maximally correlated between subjects. These activity patterns served as electrophysiological indices of narrative processing, which were compared to the neural responses of patients during the same movies. Our analysis revealed two patterns of neural activity, one for each movie condition, that were significantly and reliably correlated between healthy participants. Of the twelve patients who watched the audiovisual movie, 25% produced a pattern of activity that was significantly correlated with the healthy group, while of the ten who listened to the audio narrative, 30% produced electrophysiological patterns similar to controls (one patient responded appropriately to both). The method presented here allows for rapid bedside assessment of higher-order cognitive processing in patients with disorders of consciousness. By leveraging the common neural response to movie stimuli, we were able to identify comparable patterns of brain activity in individual, behaviourally non-responsive patients, reflecting a capacity for narrative processing.
Collapse
Affiliation(s)
- Geoffrey Laforge
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada.
| | - Laura E Gonzalez-Lara
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada
| | - Adrian M Owen
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada; The Department of Psychology, The University of Western Ontario, London ON N6A 5B7, Canada; The Department of Physiology and Pharmacology, The University of Western Ontario, London ON N6A 5B7, Canada
| | - Bobby Stojanoski
- The Brain and Mind Institute, The University of Western Ontario, London ON N6A 5B7, Canada; The Department of Psychology, The University of Western Ontario, London ON N6A 5B7, Canada
| |
Collapse
|
41
|
Movies and narratives as naturalistic stimuli in neuroimaging. Neuroimage 2020; 224:117445. [PMID: 33059053 PMCID: PMC7805386 DOI: 10.1016/j.neuroimage.2020.117445] [Citation(s) in RCA: 54] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2020] [Revised: 10/06/2020] [Accepted: 10/09/2020] [Indexed: 01/06/2023] Open
Abstract
Using movies and narratives as naturalistic stimuli in human neuroimaging studies has yielded significant advances in understanding of cognitive and emotional functions. The relevant literature was reviewed, with emphasis on how the use of naturalistic stimuli has helped advance scientific understanding of human memory, attention, language, emotions, and social cognition in ways that would have been difficult otherwise. These advances include discovering a cortical hierarchy of temporal receptive windows, which supports processing of dynamic information that accumulates over several time scales, such as immediate reactions vs. slowly emerging patterns in social interactions. Naturalistic stimuli have also helped elucidate how the hippocampus supports segmentation and memorization of events in day-to-day life and have afforded insights into attentional brain mechanisms underlying our ability to adopt specific perspectives during natural viewing. Further, neuroimaging studies with naturalistic stimuli have revealed the role of the default-mode network in narrative-processing and in social cognition. Finally, by robustly eliciting genuine emotions, these stimuli have helped elucidate the brain basis of both basic and social emotions apparently manifested as highly overlapping yet distinguishable patterns of brain activity.
Collapse
|
42
|
Stuldreher IV, Thammasan N, van Erp JBF, Brouwer AM. Physiological synchrony in EEG, electrodermal activity and heart rate reflects shared selective auditory attention. J Neural Eng 2020; 17:046028. [PMID: 32698177 DOI: 10.1088/1741-2552/aba87d] [Citation(s) in RCA: 23] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/28/2022]
Abstract
OBJECTIVE Concurrent changes in physiological signals across multiple listeners (physiological synchrony-PS), as caused by shared affective or cognitive processes, may be a suitable marker of selective attentional focus. We aimed to identify the selective attention of participants based on PS with individuals sharing attention with respect to different stimulus aspects. APPROACH We determined PS in electroencephalography (EEG), electrodermal activity (EDA) and electrocardiographic inter-beat interval (IBI) of participants who all heard the exact same audio track, but were instructed to either attend to the audiobook or to interspersed auditory events such as affective sounds and beeps that attending participants needed to keep track of. MAIN RESULTS PS in all three measures reflected the selective attentional focus of participants. In EEG and EDA, PS was higher for participants when linked to participants with the same attentional instructions than when linked to participants instructed to focus on different stimulus aspects, but in IBI this effect did not reach significance. Comparing PS between a participant and members from the same or the different attentional group allowed for the correct identification of the participant's attentional instruction in 96%, 73% and 73% of the cases, for EEG, EDA and IBI, respectively, all well above chance level. PS with respect to the attentional groups also predicted performance on post-audio questions about the groups' stimulus content. SIGNIFICANCE Our results show that selective attention of participants can be monitored using PS, not only in EEG, but also in EDA and IBI. These results are promising for real-world applications, where wearables measuring peripheral signals like EDA and IBI may be preferred over EEG sensors.
Collapse
Affiliation(s)
- Ivo V Stuldreher
- Perceptual and Cognitive Systems, Netherlands Organisation for Applied Scientific Research (TNO), Soesterberg, The Netherlands. Human Media Interaction, University of Twente, Enschede, The Netherlands
| | | | | | | |
Collapse
|
43
|
Maffei A. Spectrally resolved EEG intersubject correlation reveals distinct cortical oscillatory patterns during free‐viewing of affective scenes. Psychophysiology 2020; 57:e13652. [DOI: 10.1111/psyp.13652] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2020] [Revised: 06/08/2020] [Accepted: 07/06/2020] [Indexed: 01/10/2023]
Affiliation(s)
- Antonio Maffei
- Department of General Psychology University of Padua Padua Italy
| |
Collapse
|
44
|
DeGuzman P, Jain A, Tabert MH, Parra LC. Olfaction Modulates Inter-Subject Correlation of Neural Responses. Front Neurosci 2020; 14:702. [PMID: 32754012 PMCID: PMC7366795 DOI: 10.3389/fnins.2020.00702] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2020] [Accepted: 06/09/2020] [Indexed: 11/13/2022] Open
Abstract
Odors can be powerful stimulants. It is well-established that odors provide strong cues for recall of locations, people and events. The effects of specific scents on other cognitive functions are less well-established. We hypothesized that scents with different odor qualities will have a different effect on attention. To assess attention, we used Inter-Subject Correlation of the EEG because this metric is strongly modulated by attentional engagement with natural audiovisual stimuli. We predicted that scents known to be "energizing" would increase Inter-Subject Correlation during watching of videos as compared to "calming" scents. In a first experiment, we confirmed this for eucalyptol and linalool while participants watched animated autobiographical narratives. The result was replicated in a second experiment, but did not generalize to limonene, also considered an "energizing" odorant. In a third, double-blind experiment, we tested a battery of scents including single molecules, as well as mixtures, as participants watched various short video clips. We found a varying effect of odor on Inter-Subject Correlation across the various scents. This study provides a basis for reliably and reproducibly assessing effects of odors on brain activity. Future research is needed to further explore the effect of scent-based up-modulation in engagement on learning and memory performance. Educators, product developers and fragrance brands might also benefit from such objective neurophysiological measures.
Collapse
Affiliation(s)
| | - Anshul Jain
- International Flavors & Fragrances, Inc., R&D, Union Beach, NJ, United States
| | - Matthias H. Tabert
- International Flavors & Fragrances, Inc., R&D, Union Beach, NJ, United States
| | - Lucas C. Parra
- Department of Biomedical Engineering, City College of New York, New York, NY, United States
| |
Collapse
|
45
|
Espenhahn S, Yan T, Beltrano W, Kaur S, Godfrey K, Cortese F, Bray S, Harris AD. The effect of movie-watching on electroencephalographic responses to tactile stimulation. Neuroimage 2020; 220:117130. [PMID: 32622982 DOI: 10.1016/j.neuroimage.2020.117130] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2020] [Revised: 05/28/2020] [Accepted: 06/29/2020] [Indexed: 12/22/2022] Open
Abstract
Movie-watching is becoming a popular acquisition method to increase compliance and enable neuroimaging data collection in challenging populations such as children, with potential to facilitate studying the somatosensory system. However, relatively little is known about the possible crossmodal (audiovisual) influence of movies on cortical somatosensory processing. In this study, we examined the impact of dynamic audiovisual movies on concurrent cortical somatosensory processing using electroencephalography (EEG). Forty healthy young adults (18-25 years) received passive tactile fingertip stimulation while watching an "entertaining" movie and a novel "low-demand" movie called 'Inscapes' compared to eyes-open rest. Watching a movie did not modulate properties of early or late somatosensory-evoked potentials (SEPs). Similarly, no crossmodal influence on somatosensory adaptation, denoted by a reduction in SEP amplitude with repetitive tactile stimulation, was found. The prominent oscillatory responses in the alpha and beta frequency bands following tactile stimulation differed as a function of viewing condition, with stronger alpha/beta event-related desynchronization (ERD) during movie-watching compared to rest. These findings highlight that movie-watching is a valid acquisition method during which SEPs can be measured in basic research and clinical studies, but that the attentional demands of movies need to be taken into account when performing oscillatory analyses.
Collapse
Affiliation(s)
- Svenja Espenhahn
- Department of Radiology, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; Child and Adolescent Imaging Research (CAIR) Program, University of Calgary, Calgary, Alberta, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada.
| | - Tingting Yan
- Child and Adolescent Imaging Research (CAIR) Program, University of Calgary, Calgary, Alberta, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| | - Winnica Beltrano
- Child and Adolescent Imaging Research (CAIR) Program, University of Calgary, Calgary, Alberta, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| | - Sakshi Kaur
- Child and Adolescent Imaging Research (CAIR) Program, University of Calgary, Calgary, Alberta, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| | - Kate Godfrey
- Department of Neuroscience, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; Child and Adolescent Imaging Research (CAIR) Program, University of Calgary, Calgary, Alberta, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada
| | - Filomeno Cortese
- Department of Radiology, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| | - Signe Bray
- Department of Radiology, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; Child and Adolescent Imaging Research (CAIR) Program, University of Calgary, Calgary, Alberta, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| | - Ashley D Harris
- Department of Radiology, Cumming School of Medicine, University of Calgary, Calgary, Alberta, Canada; Child and Adolescent Imaging Research (CAIR) Program, University of Calgary, Calgary, Alberta, Canada; Alberta Children's Hospital Research Institute, University of Calgary, Calgary, Alberta, Canada; Hotchkiss Brain Institute, University of Calgary, Calgary, Alberta, Canada
| |
Collapse
|
46
|
Kaneshiro B, Nguyen DT, Norcia AM, Dmochowski JP, Berger J. Natural music evokes correlated EEG responses reflecting temporal structure and beat. Neuroimage 2020; 214:116559. [PMID: 31978543 DOI: 10.1016/j.neuroimage.2020.116559] [Citation(s) in RCA: 29] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2019] [Revised: 12/23/2019] [Accepted: 01/14/2020] [Indexed: 11/17/2022] Open
Abstract
The brain activity of multiple subjects has been shown to synchronize during salient moments of natural stimuli, suggesting that correlation of neural responses indexes a brain state operationally termed 'engagement'. While past electroencephalography (EEG) studies have considered both auditory and visual stimuli, the extent to which these results generalize to music-a temporally structured stimulus for which the brain has evolved specialized circuitry-is less understood. Here we investigated neural correlation during natural music listening by recording EEG responses from N=48 adult listeners as they heard real-world musical works, some of which were temporally disrupted through shuffling of short-term segments (measures), reversal, or randomization of phase spectra. We measured correlation between multiple neural responses (inter-subject correlation) and between neural responses and stimulus envelope fluctuations (stimulus-response correlation) in the time and frequency domains. Stimuli retaining basic musical features, such as rhythm and melody, elicited significantly higher behavioral ratings and neural correlation than did phase-scrambled controls. However, while unedited songs were self-reported as most pleasant, time-domain correlations were highest during measure-shuffled versions. Frequency-domain measures of correlation (coherence) peaked at frequencies related to the musical beat, although the magnitudes of these spectral peaks did not explain the observed temporal correlations. Our findings show that natural music evokes significant inter-subject and stimulus-response correlations, and suggest that the neural correlates of musical 'engagement' may be distinct from those of enjoyment.
Collapse
Affiliation(s)
- Blair Kaneshiro
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, USA; Center for the Study of Language and Information, Stanford University, Stanford, CA, USA; Department of Otolaryngology Head & Neck Surgery, Stanford University School of Medicine, Palo Alto, CA, USA.
| | - Duc T Nguyen
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, USA; Center for the Study of Language and Information, Stanford University, Stanford, CA, USA; Department of Biomedical Engineering, City College of New York, New York, NY, USA
| | - Anthony M Norcia
- Department of Psychology, Stanford University, Stanford, CA, USA
| | - Jacek P Dmochowski
- Department of Biomedical Engineering, City College of New York, New York, NY, USA; Department of Psychology, Stanford University, Stanford, CA, USA
| | - Jonathan Berger
- Center for Computer Research in Music and Acoustics, Stanford University, Stanford, CA, USA
| |
Collapse
|
47
|
Imhof MA, Schmälzle R, Renner B, Schupp HT. Strong health messages increase audience brain coupling. Neuroimage 2020; 216:116527. [PMID: 31954843 DOI: 10.1016/j.neuroimage.2020.116527] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/21/2019] [Revised: 12/03/2019] [Accepted: 01/07/2020] [Indexed: 10/25/2022] Open
Abstract
Mass media messaging is central for health communication. The success of these efforts, however, depends on whether health messages resonate with their target audiences. Here, we used electroencephalography (EEG) to capture brain responses of young adults - an important target group for alcohol prevention - while they viewed real-life video messages of varying perceived message effectiveness about risky alcohol use. We found that strong messages, which were rated to be more effective, prompted enhanced inter-subject correlation (ISC). In further analyses, we linked ISC to subsequent drinking behavior change and used time-resolved EEG-ISC to model functional neuroimaging data (fMRI) of an independent audience. The EEG measure was not only related to sensory-perceptual brain regions, but also to regions previously related to successful messaging, i.e., cortical midline regions and the insula. The findings suggest EEG-ISC as a marker for audience engagement and effectiveness of naturalistic health messages, which could quantify the impact of mass communication within the brains of small target audiences.
Collapse
Affiliation(s)
- Martin A Imhof
- Department of Psychology, University of Konstanz, 78457, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, University of Konstanz, 78457, Konstanz, Germany.
| | - Ralf Schmälzle
- Department of Communication, Michigan State University, East Lansing, MI, 48824, USA
| | - Britta Renner
- Department of Psychology, University of Konstanz, 78457, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, University of Konstanz, 78457, Konstanz, Germany
| | - Harald T Schupp
- Department of Psychology, University of Konstanz, 78457, Konstanz, Germany; Centre for the Advanced Study of Collective Behaviour, University of Konstanz, 78457, Konstanz, Germany
| |
Collapse
|
48
|
Kranzler EC, Schmälzle R, Pei R, Hornik RC, Falk EB. Message-Elicited Brain Response Moderates the Relationship Between Opportunities for Exposure to Anti-Smoking Messages and Message Recall. THE JOURNAL OF COMMUNICATION 2019; 69:589-611. [PMID: 32009669 PMCID: PMC6977712 DOI: 10.1093/joc/jqz035] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/09/2018] [Revised: 08/09/2019] [Accepted: 08/10/2019] [Indexed: 06/10/2023]
Abstract
Campaign success is contingent on adequate exposure; however, exposure opportunities (e.g., ad reach/frequency) are imperfect predictors of message recall. We hypothesized that the exposure-recall relationship would be contingent on message processing. We tested moderation hypotheses using 3 data sets pertinent to "The Real Cost" anti-smoking campaign: past 30-day ad recall from a rolling national survey of adolescents aged 13-17 (n = 5,110); ad-specific target rating points (TRPs), measuring ad reach and frequency; and ad-elicited response in brain regions implicated in social processing and memory encoding, from a separate adolescent sample aged 14-17 (n = 40). Average ad-level brain activation in these regions moderates the relationship between national TRPs and large-scale recall (p < .001), such that the positive exposure-recall relationship is more strongly observed for ads that elicit high levels of social processing and memory encoding in the brain. Findings advance communication theory by demonstrating conditional exposure effects, contingent on social and memory processes in the brain.
Collapse
Affiliation(s)
- Elissa C Kranzler
- Wharton Risk Management and Decision Processes Center, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Ralf Schmälzle
- College of Communication Arts and Sciences, Michigan State University, East Lansing, MI, 48824, USA
| | - Rui Pei
- Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Robert C Hornik
- Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA, 19104, USA
| | - Emily B Falk
- Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA, 19104, USA
- Department of Psychology, University of Pennsylvania, Philadelphia, PA, 19104, USA
- Wharton Marketing Department, University of Pennsylvania, Philadelphia, PA, 19104, USA
| |
Collapse
|
49
|
Zhang Y, Yin E, Li F, Zhang Y, Guo D, Yao D, Xu P. Hierarchical feature fusion framework for frequency recognition in SSVEP-based BCIs. Neural Netw 2019; 119:1-9. [DOI: 10.1016/j.neunet.2019.07.007] [Citation(s) in RCA: 18] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2019] [Revised: 05/13/2019] [Accepted: 07/07/2019] [Indexed: 11/26/2022]
|
50
|
Learning Desire Is Predicted by Similar Neural Processing of Naturalistic Educational Materials. eNeuro 2019; 6:ENEURO.0083-19.2019. [PMID: 31427402 PMCID: PMC6776790 DOI: 10.1523/eneuro.0083-19.2019] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2019] [Revised: 07/15/2019] [Accepted: 07/28/2019] [Indexed: 02/04/2023] Open
Abstract
Naturalistic stimuli can elicit highly similar brain activity across viewers. How do naturalistic educational materials engage human brains and evoke learning desire? Here, we presented 15 audiovisual course clips (each lasting ∼120 s) to university students and recorded their neural activity through electroencephalography. Upon finishing all the video viewings, subjects ranked 15 courses in order of learning desire and reported the reasons for high learning desire (i.e., “value” and “interest”). The brain activity during the video viewing was measured as the neural similarity via intersubject correlation (ISC), that is, correlation between each subject’s neural responses and those of others. Based on averaged learning desire rankings across subjects, course clips were classified with high versus medium versus low motivational effectiveness. We found that the ISC of high effective course clips was larger than that of low effective ones. The ISC difference (high vs low) was positively associated with subjects’ learning desire difference (high vs low). Such an association occurred when viewing time accumulated to ∼80 s. Moreover, ISC was correlated with “interest-based” rather than “value-based” learning desire. These findings advance our understanding of learning motivation via the neural similarity in the context of on-line education and provide potential neurophysiological suggestions for pedagogical practices.
Collapse
|