1
|
Borghini G, Ronca V, Giorgi A, Aricò P, Di Flumeri G, Capotorto R, Rooseleer F, Kirwan B, De Visscher I, Goman M, Pugh J, Abramov N, Granger G, Alarcon DPM, Humm E, Pozzi S, Babiloni F. Reducing flight upset risk and startle response: A study of the wake vortex alert with licensed commercial pilots. Brain Res Bull 2024; 215:111020. [PMID: 38909913 DOI: 10.1016/j.brainresbull.2024.111020] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2023] [Revised: 05/24/2024] [Accepted: 06/20/2024] [Indexed: 06/25/2024]
Abstract
The study aimed at investigating the impact of an innovative Wake Vortex Alert (WVA) avionics on pilots' operation and mental states, intending to improve aviation safety by mitigating the risks associated with wake vortex encounters (WVEs). Wake vortices, generated by jet aircraft, pose a significant hazard to trailing or crossing aircrafts. Despite existing separation rules, incidents involving WVEs continue to occur, especially affecting smaller aircrafts like business jets, resulting in aircraft upsets and occasional cabin injuries. To address these challenges, the study focused on developing and validating an alert system that can be presented to air traffic controllers, enabling them to warn flight crews. This empowers the flight crews to either avoid the wake vortex or secure the cabin to prevent injuries. The research employed a multidimensional approach including an analysis of human performance and human factors (HF) issues to determine the potential impact of the alert on pilots' roles, tasks, and mental states. It also utilizes Human Assurance Levels (HALs) to evaluate the necessary human factors support based on the safety criticality of the new system. Realistic flight simulations were conducted to collect data of pilots' behavioural, subjective and neurophysiological responses during WVEs. The data allowed for an objective evaluation of the WVA impact on pilots' operation, behaviour and mental states (mental workload, stress levels and arousal). In particular, the results highlighted the effectiveness of the alert system in facilitating pilots' preparation, awareness and crew resource management (CRM). The results also highlighted the importance of avionics able to enhance aviation safety and reducing risks associated with wake vortex encounters. In particular, we demonstrated how providing timely information and improving situational awareness, the WVA will minimize the occurrence of WVEs and contribute to safer aviation operations.
Collapse
Affiliation(s)
- Gianluca Borghini
- Department of Molecular Medicine, Sapienza University of Rome, Italy; BrainSigns srl, Rome, Italy.
| | - Vincenzo Ronca
- BrainSigns srl, Rome, Italy; Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome, Italy
| | - Andrea Giorgi
- BrainSigns srl, Rome, Italy; Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Italy
| | - Pietro Aricò
- BrainSigns srl, Rome, Italy; Department of Computer, Control, and Management Engineering "Antonio Ruberti", Sapienza University of Rome, Italy
| | - Gianluca Di Flumeri
- Department of Molecular Medicine, Sapienza University of Rome, Italy; BrainSigns srl, Rome, Italy
| | - Rossella Capotorto
- Department of Anatomical, Histological, Forensic and Orthopedic Sciences, Sapienza University of Rome, Italy
| | | | - Barry Kirwan
- EUROCONTROL, Centre du Bois des Bordes, Bretigny-sur-Orge, France
| | - Ivan De Visscher
- EUROCONTROL, Centre du Bois des Bordes, Bretigny-sur-Orge, France
| | - Mikhail Goman
- Faculty of Computing, Engineering and Media, De Montfort University, Leicester, United Kingdom
| | - Jonathan Pugh
- Faculty of Computing, Engineering and Media, De Montfort University, Leicester, United Kingdom
| | - Nikolay Abramov
- Faculty of Computing, Engineering and Media, De Montfort University, Leicester, United Kingdom
| | - Géraud Granger
- Safety Management Research Program, École Nationale de l'Aviation Civile (ENAC), France
| | | | | | | | - Fabio Babiloni
- Department of Physiology and Pharmacology "Vittorio Erspamer", Sapienza University of Rome, Italy; School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, China
| |
Collapse
|
2
|
Carollo A, Esposito G. Hyperscanning literature after two decades of neuroscientific research: A scientometric review. Neuroscience 2024; 551:345-354. [PMID: 38866073 DOI: 10.1016/j.neuroscience.2024.05.045] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/02/2024] [Revised: 05/24/2024] [Accepted: 05/30/2024] [Indexed: 06/14/2024]
Abstract
Hyperscanning, a neuroimaging approach introduced in 2002 for simultaneously recording the brain activity of multiple participants, has significantly contributed to our understanding of social interactions. Nevertheless, the existing literature requires systematic organization to advance our knowledge. This study, after two decades of hyperscanning research, aims to identify the primary thematic domains and the most influential documents in the field. We conducted a scientometric analysis to examine co-citation patterns quantitatively, using a sample of 548 documents retrieved from Scopus and their 32,022 cited references. Our analysis revealed ten major thematic domains in hyperscanning research, with the most impactful document authored by Czeszumski and colleagues in 2020. Notably, while hyperscanning was initially developed for functional magnetic resonance imaging (fMRI), our findings indicate a substantial influence of research conducted using electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). The introduction of fNIRS and advancements in EEG methods have enabled the implementation of more ecologically valid experiments for investigating social interactions. The study also highlights the need for more research that combines multi-brain neural stimulation with neuroimaging techniques to understand the causal role played by interpersonal neural synchrony in social interactions.
Collapse
Affiliation(s)
- Alessandro Carollo
- Department of Psychology and Cognitive Science, University of Trento, Rovereto, Italy.
| | - Gianluca Esposito
- Department of Psychology and Cognitive Science, University of Trento, Rovereto, Italy.
| |
Collapse
|
3
|
Pappalettera C, Mansi SA, Arnesano M, Vecchio F. Decoding influences of indoor temperature and light on neural activity: entropy analysis of electroencephalographic signals. Pflugers Arch 2024:10.1007/s00424-024-02988-z. [PMID: 39012352 DOI: 10.1007/s00424-024-02988-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/29/2024] [Revised: 05/22/2024] [Accepted: 07/03/2024] [Indexed: 07/17/2024]
Abstract
Understanding the neural responses to indoor characteristics like temperature and light is crucial for comprehending how the physical environment influences the human brain. Our study introduces an innovative approach using entropy analysis, specifically, approximate entropy (ApEn), applied to electroencephalographic (EEG) signals to investigate neural responses to temperature and light variations in indoor environments. By strategically placing electrodes over specific brain regions linked to temperature and light processing, we show how ApEn can be influenced by indoor factors. We also integrate heart indices from a multi-sensor bracelet to create a machine learning classifier for temperature conditions. Results showed that in anterior frontal and temporoparietal areas, neutral temperature conditions yield higher ApEn values. The anterior frontal area showed a trend of gradually decreasing ApEn values from neutral to warm conditions, with cold being in an intermediate position. There was a significant interaction between light and site factors, only evident in the temporoparietal region. Here, the neutral light condition had higher ApEn values compared to blue and red light conditions. Positive correlations between anterior frontal ApEn and thermal comfort scores suggest a link between entropy and perceived thermal comfort. Our quadratic SVM classifier, incorporating entropy and heart features, demonstrates strong performance (until 90% in terms of AUC, accuracy, sensitivity, and specificity) in classifying temperature sensations. This study offers insights into neural responses to indoor factors and presents a novel approach for temperature classification using EEG entropy and heart features.
Collapse
Affiliation(s)
- Chiara Pappalettera
- Brain Connectivity Laboratory, Department of Neuroscience and Neurorehabilitation, IRCCS San Raffaele Roma, Rome, Italy
- Department of Theoretical and Applied Sciences, eCampus University, Novedrate, Italy
| | - Silvia Angela Mansi
- Department of Theoretical and Applied Sciences, eCampus University, Novedrate, Italy
| | - Marco Arnesano
- Department of Theoretical and Applied Sciences, eCampus University, Novedrate, Italy
| | - Fabrizio Vecchio
- Brain Connectivity Laboratory, Department of Neuroscience and Neurorehabilitation, IRCCS San Raffaele Roma, Rome, Italy.
- Department of Theoretical and Applied Sciences, eCampus University, Novedrate, Italy.
| |
Collapse
|
4
|
Kawaguchi K, Nikai Y, Yomota S, Kawashima A, Inoue Y, Takahashi M. Effects of age and flight experience on prefrontal cortex activity in airline pilots: An fNIRS study. Heliyon 2024; 10:e30242. [PMID: 38707377 PMCID: PMC11066422 DOI: 10.1016/j.heliyon.2024.e30242] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/29/2022] [Revised: 04/13/2024] [Accepted: 04/22/2024] [Indexed: 05/07/2024] Open
Abstract
It is essential for airlines to have a deep understanding of the cognitive impact of aging among pilots. The current literature on executive function indicates that compensatory mechanisms in the brain may counteract age-related cognitive decline, at least up to certain task load levels. However, few studies have been administered to evaluate changes in aircrew competence as they age. The present study focuses on dorsolateral prefrontal cortex (DLPFC) activity as it is implicated in cognitive performance and working memory, which are associated with skill proficiency. We measured the DLPFC activity for airline pilots, including trainees, during maneuvering using a flight simulator. Our preliminary results indicated that only expert (aged) pilots demonstrated higher activity of the left DLPFC than the right one. However, for youth trainees, not only was the error rate high while using the flight simulator, but the activity of the DLFPC was also lower than that of the expert pilots, and there was no statistically significant difference between the left and right DLPFC. Although these findings partially differ from those reported in previous studies on age-related changes, it is evident that training as an airline pilot for over 20 years may affect such results. We believe that this noninvasive approach to objective quantification of skill will facilitate the development of effective assessment competence in aging.
Collapse
Affiliation(s)
- Kenji Kawaguchi
- Crew Resources Development, Flight Operation Center, ALL NIPPON AIRWAYS, Co., LTD., Tokyo 144-8525, Japan
| | - Yohei Nikai
- Crew Resources Development, Flight Operation Center, ALL NIPPON AIRWAYS, Co., LTD., Tokyo 144-8525, Japan
| | - Satoshi Yomota
- Analytical & Measuring Instruments Division, SHIMADZU Corporation, Kyoto 604-8511, Japan
| | - Akisato Kawashima
- Crew Resources Development, Flight Operation Center, ALL NIPPON AIRWAYS, Co., LTD., Tokyo 144-8525, Japan
| | - Yoshihiro Inoue
- Analytical & Measuring Instruments Division, SHIMADZU Corporation, Kyoto 604-8511, Japan
| | - Makoto Takahashi
- Crew Resources Development, Flight Operation Center, ALL NIPPON AIRWAYS, Co., LTD., Tokyo 144-8525, Japan
| |
Collapse
|
5
|
Tamburro G, Bruña R, Fiedler P, De Fano A, Raeisi K, Khazaei M, Zappasodi F, Comani S. An Analytical Approach for Naturalistic Cooperative and Competitive EEG-Hyperscanning Data: A Proof-of-Concept Study. SENSORS (BASEL, SWITZERLAND) 2024; 24:2995. [PMID: 38793851 PMCID: PMC11125252 DOI: 10.3390/s24102995] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 04/03/2024] [Accepted: 05/02/2024] [Indexed: 05/26/2024]
Abstract
Investigating the neural mechanisms underlying both cooperative and competitive joint actions may have a wide impact in many social contexts of human daily life. An effective pipeline of analysis for hyperscanning data recorded in a naturalistic context with a cooperative and competitive motor task has been missing. We propose an analytical pipeline for this type of joint action data, which was validated on electroencephalographic (EEG) signals recorded in a proof-of-concept study on two dyads playing cooperative and competitive table tennis. Functional connectivity maps were reconstructed using the corrected imaginary part of the phase locking value (ciPLV), an algorithm suitable in case of EEG signals recorded during turn-based competitive joint actions. Hyperbrain, within-, and between-brain functional connectivity maps were calculated in three frequency bands (i.e., theta, alpha, and beta) relevant during complex motor task execution and were characterized with graph theoretical measures and a clustering approach. The results of the proof-of-concept study are in line with recent findings on the main features of the functional networks sustaining cooperation and competition, hence demonstrating that the proposed pipeline is promising tool for the analysis of joint action EEG data recorded during cooperation and competition using a turn-based motor task.
Collapse
Affiliation(s)
- Gabriella Tamburro
- Behavioral Imaging and Neural Dynamics Center, G. d’Annunzio University of Chieti-Pescara, 66100 Chieti, Italy; (A.D.F.); (F.Z.); (S.C.)
- Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti–Pescara, 66100 Chieti, Italy; (K.R.); (M.K.)
| | - Ricardo Bruña
- Center for Cognitive and Computational Neuroscience (C3N), Universidad Complutense de Madrid, 28040 Madrid, Spain;
- Department of Radiology, Rehabilitation and Physiotherapy, School of Medicine, Universidad Complutense de Madrid, IdISSC, 28040 Madrid, Spain
| | - Patrique Fiedler
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, 98693 Ilmenau, Germany
| | - Antonio De Fano
- Behavioral Imaging and Neural Dynamics Center, G. d’Annunzio University of Chieti-Pescara, 66100 Chieti, Italy; (A.D.F.); (F.Z.); (S.C.)
- Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti–Pescara, 66100 Chieti, Italy; (K.R.); (M.K.)
| | - Khadijeh Raeisi
- Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti–Pescara, 66100 Chieti, Italy; (K.R.); (M.K.)
| | - Mohammad Khazaei
- Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti–Pescara, 66100 Chieti, Italy; (K.R.); (M.K.)
| | - Filippo Zappasodi
- Behavioral Imaging and Neural Dynamics Center, G. d’Annunzio University of Chieti-Pescara, 66100 Chieti, Italy; (A.D.F.); (F.Z.); (S.C.)
- Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti–Pescara, 66100 Chieti, Italy; (K.R.); (M.K.)
- Institute for Advanced Biomedical Technologies, University “Gabriele d’Annunzio” of Chieti–Pescara, 66100 Chieti, Italy
| | - Silvia Comani
- Behavioral Imaging and Neural Dynamics Center, G. d’Annunzio University of Chieti-Pescara, 66100 Chieti, Italy; (A.D.F.); (F.Z.); (S.C.)
- Department of Neuroscience, Imaging and Clinical Sciences, G. d’Annunzio University of Chieti–Pescara, 66100 Chieti, Italy; (K.R.); (M.K.)
| |
Collapse
|
6
|
Réveillé C, Vergotte G, Perrey S, Bosselut G. Using interbrain synchrony to study teamwork: A systematic review and meta-analysis. Neurosci Biobehav Rev 2024; 159:105593. [PMID: 38373643 DOI: 10.1016/j.neubiorev.2024.105593] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/24/2023] [Revised: 01/19/2024] [Accepted: 02/14/2024] [Indexed: 02/21/2024]
Abstract
It has been proposed that interbrain synchrony (IBS) may help to elucidate the neural mechanisms underpinning teamwork. As hyperscanning studies have provided abundant findings on IBS in team environments, the current review aims to synthesize the findings of hyperscanning studies in a way that is relevant to the teamwork research. A systematic review was conducted. Included studies were classified according to the IPO (i.e. input, process, output) model of teamwork. Three multi-level meta-analyses were performed to quantify the associations between IBS and the three IPO variables. The methodology followed PRISMA guidelines and the protocol was pre-registered (https://osf.io/7h8sa/). Of the 229 studies, 41 were included, representing 1326 teams. The three meta-analyses found statistically significant positive effects, indicating a positive association between IBS and the three IPO teamwork variables. This study provides evidence that IBS is a relevant measure of the teamwork process and argues for the continued use of IBS to study teamwork.
Collapse
Affiliation(s)
- Coralie Réveillé
- EuroMov Digital Health in Motion (Univ Montpellier, IMT Mines d'Alès), 700 avenue du Pic Saint Loup, Montpellier 34090, France.
| | - Grégoire Vergotte
- EuroMov Digital Health in Motion (Univ Montpellier, IMT Mines d'Alès), 700 avenue du Pic Saint Loup, Montpellier 34090, France
| | - Stéphane Perrey
- EuroMov Digital Health in Motion (Univ Montpellier, IMT Mines d'Alès), 700 avenue du Pic Saint Loup, Montpellier 34090, France
| | - Grégoire Bosselut
- EuroMov Digital Health in Motion (Univ Montpellier, IMT Mines d'Alès), 700 avenue du Pic Saint Loup, Montpellier 34090, France
| |
Collapse
|
7
|
Takeuchi N. A dual-brain therapeutic approach using noninvasive brain stimulation based on two-person neuroscience: A perspective review. MATHEMATICAL BIOSCIENCES AND ENGINEERING : MBE 2024; 21:5118-5137. [PMID: 38872529 DOI: 10.3934/mbe.2024226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2024]
Abstract
Our actions and decisions in everyday life are heavily influenced by social interactions, which are dynamic feedback loops involving actions, reactions, and internal cognitive processes between individual agents. Social interactions induce interpersonal synchrony, which occurs at different biobehavioral levels and comprises behavioral, physiological, and neurological activities. Hyperscanning-a neuroimaging technique that simultaneously measures the activity of multiple brain regions-has provided a powerful second-person neuroscience tool for investigating the phase alignment of neural processes during interactive social behavior. Neural synchronization, revealed by hyperscanning, is a phenomenon called inter-brain synchrony- a process that purportedly facilitates social interactions by prompting appropriate anticipation of and responses to each other's social behaviors during ongoing shared interactions. In this review, I explored the therapeutic dual-brain approach using noninvasive brain stimulation to target inter-brain synchrony based on second-person neuroscience to modulate social interaction. Artificially inducing synchrony between the brains is a potential adjunct technique to physiotherapy, psychotherapy, and pain treatment- which are strongly influenced by the social interaction between the therapist and patient. Dual-brain approaches to personalize stimulation parameters must consider temporal, spatial, and oscillatory factors. Multiple data fusion analysis, the assessment of inter-brain plasticity, a closed-loop system, and a brain-to-brain interface can support personalized stimulation.
Collapse
Affiliation(s)
- Naoyuki Takeuchi
- Department of Physical Therapy, Akita University Graduate School of Health Sciences, 1-1-1 Hondo, Akita, 010-8543, Japan
| |
Collapse
|
8
|
Kurihara Y, Takahashi T, Osu R. The topology of interpersonal neural network in weak social ties. Sci Rep 2024; 14:4961. [PMID: 38418895 DOI: 10.1038/s41598-024-55495-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/08/2023] [Accepted: 02/23/2024] [Indexed: 03/02/2024] Open
Abstract
The strategies for social interaction between strangers differ from those between acquaintances, whereas the differences in neural basis of social interaction have not been fully elucidated. In this study, we examined the geometrical properties of interpersonal neural networks in pairs of strangers and acquaintances during antiphase joint tapping. Dual electroencephalogram (EEG) of 29 channels per participant was measured from 14 strangers and 13 acquaintance pairs.Intra-brain synchronizations were calculated using the weighted phase lag index (wPLI) for intra-brain electrode combinations, and inter-brain synchronizations were calculated using the phase locking value (PLV) for inter-brain electrode combinations in the theta, alpha, and beta frequency bands. For each participant pair, electrode combinations with larger wPLI/PLV than their surrogates were defined as the edges of the neural networks. We calculated global efficiency, local efficiency, and modularity derived from graph theory for the combined intra- and inter-brain networks of each pair. In the theta band networks, stranger pairs showed larger local efficiency than acquaintance pairs, indicating that the two brains of stranger pairs were more densely connected. Hence, weak social ties require extensive social interactions and result in high efficiency of information transfer between neighbors in neural network.
Collapse
Affiliation(s)
- Yuto Kurihara
- Graduate School of Human Sciences, Waseda University, Tokorozawa, Saitama, Japan
| | - Toru Takahashi
- Advanced Research Center for Human Sciences, Waseda University, Tokorozawa, Saitama, Japan
| | - Rieko Osu
- Faculty of Human Sciences, Waseda University, 2-579-15 Mikajima, Tokorozawa, Saitama, Japan.
| |
Collapse
|
9
|
Dikker S, Brito NH, Dumas G. It takes a village: A multi-brain approach to studying multigenerational family communication. Dev Cogn Neurosci 2024; 65:101330. [PMID: 38091864 PMCID: PMC10716709 DOI: 10.1016/j.dcn.2023.101330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2023] [Revised: 08/27/2023] [Accepted: 12/07/2023] [Indexed: 12/17/2023] Open
Abstract
Grandparents play a critical role in child rearing across the globe. Yet, there is a shortage of neurobiological research examining the relationship between grandparents and their grandchildren. We employ multi-brain neurocomputational models to simulate how changes in neurophysiological processes in both development and healthy aging affect multigenerational inter-brain coupling - a neural marker that has been linked to a range of socio-emotional and cognitive outcomes. The simulations suggest that grandparent-child interactions may be paired with higher inter-brain coupling than parent-child interactions, raising the possibility that the former may be more advantageous under certain conditions. Critically, this enhancement of inter-brain coupling for grandparent-child interactions is more pronounced in tri-generational interactions that also include a parent, which may speak to findings that grandparent involvement in childrearing is most beneficial if the parent is also an active household member. Together, these findings underscore that a better understanding of the neurobiological basis of cross-generational interactions is vital, and that such knowledge can be helpful in guiding interventions that consider the whole family. We advocate for a community neuroscience approach in developmental social neuroscience to capture the diversity of child-caregiver relationships in real-world settings.
Collapse
|
10
|
Santamaria L, Koopman ACM, Bekinschtein T, Lewis P. Effects of Targeted Memory Reactivation on Cortical Networks. Brain Sci 2024; 14:114. [PMID: 38391689 PMCID: PMC10886727 DOI: 10.3390/brainsci14020114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/15/2023] [Revised: 01/16/2024] [Accepted: 01/17/2024] [Indexed: 02/24/2024] Open
Abstract
Sleep is a complex physiological process with an important role in memory consolidation characterised by a series of spatiotemporal changes in brain activity and connectivity. Here, we investigate how task-related responses differ between pre-sleep wake, sleep, and post-sleep wake. To this end, we trained participants on a serial reaction time task using both right and left hands using Targeted Memory Reactivation (TMR), in which auditory cues are associated with learned material and then re-presented in subsequent wake or sleep periods in order to elicit memory reactivation. The neural responses just after each cue showed increased theta band connectivity between frontal and other cortical regions, as well as between hemispheres, in slow wave sleep compared to pre- or post-sleep wake. This pattern was consistent across the cues associated with both right- and left-handed movements. We also searched for hand-specific connectivity and found that this could be identified in within-hemisphere connectivity after TMR cues during sleep and post-sleep sessions. The fact that we could identify which hand had been cued during sleep suggests that these connectivity measures could potentially be used to determine how successfully memory is reactivated by our manipulation. Collectively, these findings indicate that TMR modulates the brain cortical networks showing clear differences between wake and sleep connectivity patterns.
Collapse
Affiliation(s)
| | | | | | - Penelope Lewis
- School of Psychology, Cardiff University, Wales CF10 3AT, UK
| |
Collapse
|
11
|
Chuang TM, Peng PC, Su YK, Lin SH, Tseng YL. Exploring Inter-Brain Electroencephalogram Patterns for Social Cognitive Assessment During Jigsaw Puzzle Solving. IEEE Trans Neural Syst Rehabil Eng 2024; 32:422-430. [PMID: 38198273 DOI: 10.1109/tnsre.2024.3352036] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2024]
Abstract
Social interaction enables the smooth progression of our daily lives. Mounting evidence from recent hyperscanning neuroimaging studies indicates that key components of social behavior can be evaluated using inter-brain oscillations and connectivity. However, mapping out inter-brain networks and developing neurocognitive theories that explain how humans co-create and share information during social interaction remains challenging. In this study, we developed a jigsaw puzzle-solving game with hyperscanning electroencephalography (EEG) signals recorded to investigate inter-brain activities during social interactions involving cooperation and competition. Participants were recruited and paired into dyads to participate in the multiplayer jigsaw puzzle game with 32-channel EEG signals recorded. The corresponding event-related potentials (ERPs), brain oscillations, and inter-brain functional connectivity were analyzed. The results showed different ERP morphologies of P3 patterns in competitive and cooperative contexts, and brain oscillations in the low-frequency band may be an indicator of social cognitive activities. Furthermore, increased inter-brain functional connectivity in the delta, theta, alpha, and beta frequency bands was observed in the competition mode compared to the cooperation mode. By presenting comparable and valid hyperscanning EEG results alongside those of previous studies using traditional paradigms, this study demonstrates the potential of utilizing hyperscanning techniques in real-life game-playing scenarios to quantitatively assess social cognitive interactions involving cooperation and competition. Our approach offers a promising platform with potential applications in the flexible assessment of psychiatric disorders related to social functioning.
Collapse
|
12
|
Minagawa Y, Hata M, Yamamoto E, Tsuzuki D, Morimoto S. Inter-brain synchrony during mother-infant interactive parenting in 3-4-month-old infants with and without an elevated likelihood of autism spectrum disorder. Cereb Cortex 2023; 33:11609-11622. [PMID: 37885119 PMCID: PMC10724871 DOI: 10.1093/cercor/bhad395] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 09/27/2023] [Accepted: 09/28/2023] [Indexed: 10/28/2023] Open
Abstract
Maternal bonding for mammalian infants is critical for their survival. Additionally, it is important for human infants' development into social creatures. However, despite the ample neurobiological evidence of attachment for the mother's brain, the interplay of this system in infants is poorly understood. We aimed to identify the neural substrates of synchrony in mothers and infants under three interactive conditions and compare the differences between groups with (n = 16) and without (n = 71) an elevated likelihood of autism spectrum disorder by examining the inter-brain synchrony between mothers and their 3-4-month-old infants. Mother-infant hyperscanning with functional near-infrared spectroscopy was performed during breastfeeding and while each of the mother and experimenter was holding the infants. The results showed almost no group differences, with both groups demonstrating the strongest inter-brain coupling for breastfeeding. The cerebral foci underlying these couplings differed between mothers and infants: the ventral prefrontal cortex, focusing on the right orbitofrontal cortex, in the mother and the left temporoparietal junction in the infant were chiefly involved in connecting the two brains. Furthermore, these synchronizations revealed many significant correlations with behavioral measures, including subsequent language development. The maternal reward-motivational system and the infant's elementary mentalization system seem to underlie mother-infant coupling during breastfeeding.
Collapse
Affiliation(s)
- Yasuyo Minagawa
- Department of Psychology, Faculty of Letters, Keio University, 4-1-1 Hiyoshi, Kohoku-ku, Yokohama 223-8521, Japan
- Human Biology-Microbiome-Quantum Research Center (WPI-Bio2Q), Keio University, 35 Shinanomachi, Shinjuku-ku, Tokyo 160-8582, Japan
- Center for Advanced Research of Logic and Sensibility, Global Research Institute, Keio University, 2-15-45 Mita, Minato-ku, Tokyo 108-8345, Japan
| | - Masahiro Hata
- Center for Advanced Research of Logic and Sensibility, Global Research Institute, Keio University, 2-15-45 Mita, Minato-ku, Tokyo 108-8345, Japan
| | - Eriko Yamamoto
- Center for Advanced Research of Logic and Sensibility, Global Research Institute, Keio University, 2-15-45 Mita, Minato-ku, Tokyo 108-8345, Japan
| | - Daisuke Tsuzuki
- Department of Information Science, Faculty of Science and Technology, Kochi University, 2-5-1 Akebono-cho, kochi-shi, Kochi 780-8072, Japan
| | - Satoshi Morimoto
- Center for Advanced Research of Logic and Sensibility, Global Research Institute, Keio University, 2-15-45 Mita, Minato-ku, Tokyo 108-8345, Japan
| |
Collapse
|
13
|
Tamburro G, Fiedler P, De Fano A, Raeisi K, Khazaei M, Vaquero L, Bruña R, Oppermann H, Bertollo M, Filho E, Zappasodi F, Comani S. An ecological study protocol for the multimodal investigation of the neurophysiological underpinnings of dyadic joint action. Front Hum Neurosci 2023; 17:1305331. [PMID: 38125713 PMCID: PMC10730734 DOI: 10.3389/fnhum.2023.1305331] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2023] [Accepted: 11/15/2023] [Indexed: 12/23/2023] Open
Abstract
A novel multimodal experimental setup and dyadic study protocol were designed to investigate the neurophysiological underpinnings of joint action through the synchronous acquisition of EEG, ECG, EMG, respiration and kinematic data from two individuals engaged in ecologic and naturalistic cooperative and competitive joint actions involving face-to-face real-time and real-space coordinated full body movements. Such studies are still missing because of difficulties encountered in recording reliable neurophysiological signals during gross body movements, in synchronizing multiple devices, and in defining suitable study protocols. The multimodal experimental setup includes the synchronous recording of EEG, ECG, EMG, respiration and kinematic signals of both individuals via two EEG amplifiers and a motion capture system that are synchronized via a single-board microcomputer and custom Python scripts. EEG is recorded using new dry sports electrode caps. The novel study protocol is designed to best exploit the multimodal data acquisitions. Table tennis is the dyadic motor task: it allows naturalistic and face-to-face interpersonal interactions, free in-time and in-space full body movement coordination, cooperative and competitive joint actions, and two task difficulty levels to mimic changing external conditions. Recording conditions-including minimum table tennis rally duration, sampling rate of kinematic data, total duration of neurophysiological recordings-were defined according to the requirements of a multilevel analytical approach including a neural level (hyperbrain functional connectivity, Graph Theoretical measures and Microstate analysis), a cognitive-behavioral level (integrated analysis of neural and kinematic data), and a social level (extending Network Physiology to neurophysiological data recorded from two interacting individuals). Four practical tests for table tennis skills were defined to select the study population, permitting to skill-match the dyad members and to form two groups of higher and lower skilled dyads to explore the influence of skill level on joint action performance. Psychometric instruments are included to assess personality traits and support interpretation of results. Studying joint action with our proposed protocol can advance the understanding of the neurophysiological mechanisms sustaining daily life joint actions and could help defining systems to predict cooperative or competitive behaviors before being overtly expressed, particularly useful in real-life contexts where social behavior is a main feature.
Collapse
Affiliation(s)
- Gabriella Tamburro
- Department of Neuroscience Imaging and Clinical Sciences, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
- Behavioral Imaging and Neural Dynamics Center, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
| | - Patrique Fiedler
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
| | - Antonio De Fano
- Department of Neuroscience Imaging and Clinical Sciences, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
- Behavioral Imaging and Neural Dynamics Center, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
| | - Khadijeh Raeisi
- Department of Neuroscience Imaging and Clinical Sciences, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
| | - Mohammad Khazaei
- Department of Neuroscience Imaging and Clinical Sciences, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
| | - Lucia Vaquero
- Center for Cognitive and Computational Neuroscience, Universidad Complutense de Madrid, Madrid, Spain
- Department of Experimental Pschology, Cognitive Processes and Speech Therapy, Universidad Complutense de Madrid, Madrid, Spain
| | - Ricardo Bruña
- Center for Cognitive and Computational Neuroscience, Universidad Complutense de Madrid, Madrid, Spain
- Department of Radiology, Universidad Complutense de Madrid, IdISSC, Madrid, Spain
| | - Hannes Oppermann
- Institute of Biomedical Engineering and Informatics, Technische Universität Ilmenau, Ilmenau, Germany
| | - Maurizio Bertollo
- Behavioral Imaging and Neural Dynamics Center, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
- Department of Medicine and Sciences of Aging, “University G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
| | - Edson Filho
- Wheelock College of Education and Human Development, Boston University, Boston, MA, United States
| | - Filippo Zappasodi
- Department of Neuroscience Imaging and Clinical Sciences, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
- Behavioral Imaging and Neural Dynamics Center, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
| | - Silvia Comani
- Department of Neuroscience Imaging and Clinical Sciences, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
- Behavioral Imaging and Neural Dynamics Center, University “G. d’Annunzio” of Chieti–Pescara, Chieti, Italy
| |
Collapse
|
14
|
Hakim U, De Felice S, Pinti P, Zhang X, Noah JA, Ono Y, Burgess PW, Hamilton A, Hirsch J, Tachtsidis I. Quantification of inter-brain coupling: A review of current methods used in haemodynamic and electrophysiological hyperscanning studies. Neuroimage 2023; 280:120354. [PMID: 37666393 DOI: 10.1016/j.neuroimage.2023.120354] [Citation(s) in RCA: 15] [Impact Index Per Article: 15.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/08/2023] [Revised: 08/25/2023] [Accepted: 08/28/2023] [Indexed: 09/06/2023] Open
Abstract
Hyperscanning is a form of neuroimaging experiment where the brains of two or more participants are imaged simultaneously whilst they interact. Within the domain of social neuroscience, hyperscanning is increasingly used to measure inter-brain coupling (IBC) and explore how brain responses change in tandem during social interaction. In addition to cognitive research, some have suggested that quantification of the interplay between interacting participants can be used as a biomarker for a variety of cognitive mechanisms aswell as to investigate mental health and developmental conditions including schizophrenia, social anxiety and autism. However, many different methods have been used to quantify brain coupling and this can lead to questions about comparability across studies and reduce research reproducibility. Here, we review methods for quantifying IBC, and suggest some ways moving forward. Following the PRISMA guidelines, we reviewed 215 hyperscanning studies, across four different brain imaging modalities: functional near-infrared spectroscopy (fNIRS), functional magnetic resonance (fMRI), electroencephalography (EEG) and magnetoencephalography (MEG). Overall, the review identified a total of 27 different methods used to compute IBC. The most common hyperscanning modality is fNIRS, used by 119 studies, 89 of which adopted wavelet coherence. Based on the results of this literature survey, we first report summary statistics of the hyperscanning field, followed by a brief overview of each signal that is obtained from each neuroimaging modality used in hyperscanning. We then discuss the rationale, assumptions and suitability of each method to different modalities which can be used to investigate IBC. Finally, we discuss issues surrounding the interpretation of each method.
Collapse
Affiliation(s)
- U Hakim
- Department of Medical Physics and Biomedical Engineering, University College London, Malet Place Engineering Building, Gower Street, London WC1E 6BT, United Kingdom.
| | - S De Felice
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom; Department of Psychology, University of Cambridge, United Kingdom
| | - P Pinti
- Department of Medical Physics and Biomedical Engineering, University College London, Malet Place Engineering Building, Gower Street, London WC1E 6BT, United Kingdom; Centre for Brain and Cognitive Development, Birkbeck, University of London, London, United Kingdom
| | - X Zhang
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, United States
| | - J A Noah
- Department of Psychiatry, Yale School of Medicine, New Haven, CT, United States
| | - Y Ono
- Department of Electronics and Bioinformatics, School of Science and Technology, Meiji University, Kawasaki, Kanagawa, Japan
| | - P W Burgess
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom
| | - A Hamilton
- Institute of Cognitive Neuroscience, University College London, London, United Kingdom
| | - J Hirsch
- Department of Medical Physics and Biomedical Engineering, University College London, Malet Place Engineering Building, Gower Street, London WC1E 6BT, United Kingdom; Department of Psychiatry, Yale School of Medicine, New Haven, CT, United States; Departments of Neuroscience and Comparative Medicine, Yale School of Medicine, New Haven, CT, United States; Yale University, Wu Tsai Institute, New Haven, CT, United States
| | - I Tachtsidis
- Department of Medical Physics and Biomedical Engineering, University College London, Malet Place Engineering Building, Gower Street, London WC1E 6BT, United Kingdom
| |
Collapse
|
15
|
Yang M, Li X, Sang B, Deng X. Age differences in interbrain synchronization during peer cooperation: an EEG hyperscanning study. Cereb Cortex 2023; 33:10614-10623. [PMID: 37615349 DOI: 10.1093/cercor/bhad308] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/23/2023] [Revised: 07/31/2023] [Accepted: 08/01/2023] [Indexed: 08/25/2023] Open
Abstract
Healthy peer relationships could provide emotional and social support for adolescents experiencing dramatic physical and environmental changes. Examining age differences in cognitive neural processing during peer interaction provides insight into adolescent interpersonal contact and "social brain" development. The present study compared the age differences between adolescents and adults by examining the behavior and interbrain synchronization of pairs in a cooperative computer game task. 32 pairs of adolescents and 31 pairs of adults were recruited as participants. The reaction times and interbrain synchronization of the participants were measured. The results revealed that interbrain synchronization activation following the onset of the "ready signal" was primarily detected in low-frequency bands such as delta and theta. Adolescent pairs' interbrain synchronization activations were significantly higher than those of adult pairs in the anterior and central brain regions, such as the frontal, frontal-central, and parietal lobes. Correlation analysis indicated a positive correlation between occipital region interbrain synchronization and behavioral performance. The findings provide behavioral and neurophysiological evidence for the characteristics of adolescent interpersonal cognitive processing and point to the significance of low-frequency interbrain synchronization in interpersonal coordination.
Collapse
Affiliation(s)
- Meng Yang
- School of Psychology, Shenzhen University, Shenzhen 518060, China
- School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
| | - Xinqi Li
- School of Psychology, Shenzhen University, Shenzhen 518060, China
| | - Biao Sang
- Lab for Educational Big Data and Policymaking, Shanghai Academy of Educational Sciences, Shanghai 200032, China
- School of Psychology and Cognitive Science, East China Normal University, Shanghai 200062, China
| | - Xinmei Deng
- School of Psychology, Shenzhen University, Shenzhen 518060, China
- The Shenzhen Humanities & Social Sciences Key Research Bases of the Center for Mental Health, Shenzhen University, Shenzhen 518060, China
| |
Collapse
|
16
|
Park J, Shin J, Lee J, Jeong J. Inter-Brain Synchrony Pattern Investigation on Triadic Board Game Play-Based Social Interaction: An fNIRS Study. IEEE Trans Neural Syst Rehabil Eng 2023; 31:2923-2932. [PMID: 37410649 DOI: 10.1109/tnsre.2023.3292844] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/08/2023]
Abstract
Recent advances in functional neuroimaging techniques, including methodologies such as fNIRS, have enabled the evaluation of inter-brain synchrony (IBS) induced by interpersonal interactions. However, the social interactions assumed in existing dyadic hyperscanning studies do not sufficiently emulate polyadic social interactions in the real world. Therefore, we devised an experimental paradigm that incorporates the Korean folk board game "Yut-nori" to reproduce social interactions that emulate social activities in the real world. We recruited 72 participants aged 25.2 ± 3.9 years (mean ± standard deviation) and divided them into 24 triads to play Yut-nori, following the standard or modified rules. The participants either competed against an opponent (standard rule) or cooperated with an opponent (modified rule) to achieve a goal efficiently. Three different fNIRS devices were employed to record cortical hemodynamic activations in the prefrontal cortex both individually and simultaneously. Wavelet transform coherence (WTC) analyses were performed to assess prefrontal IBS within a frequency range of 0.05-0.2 Hz. Consequently, we observed that cooperative interactions increased prefrontal IBS across overall frequency bands of interest. In addition, we also found that different purposes for cooperation generated different spectral characteristics of IBS depending on the frequency bands. Moreover, IBS in the frontopolar cortex (FPC) reflected the influence of verbal interactions. The findings of our study suggest that future hyperscanning studies should consider polyadic social interactions to reveal the properties of IBS in real-world interactions.
Collapse
|
17
|
Koul A, Ahmar D, Iannetti GD, Novembre G. Spontaneous dyadic behaviour predicts the emergence of interpersonal neural synchrony. Neuroimage 2023:120233. [PMID: 37348621 DOI: 10.1016/j.neuroimage.2023.120233] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Revised: 06/14/2023] [Accepted: 06/15/2023] [Indexed: 06/24/2023] Open
Abstract
Synchronization of neural activity across brains - interpersonal neural synchrony (INS) - is emerging as a powerful marker of social interaction that predicts success of multi-person coordination, communication, and cooperation. As the origins of INS are poorly understood, we tested whether and how INS might emerge from spontaneous dyadic behavior. We recorded neural activity (EEG) and human behavior (full-body kinematics, eye movements and facial expressions) while dyads of participants were instructed to look at each other without speaking or making co-verbal gestures. We made four fundamental observations. First, despite the absence of a structured social task, INS emerged spontaneously only when participants were able to see each other. Second, we show that such spontaneous INS, comprising specific spectral and topographic profiles, did not merely reflect intra-personal modulations of neural activity, but it rather reflected real-time and dyad-specific coupling of neural activities. Third, using state-of-art video-image processing and deep learning, we extracted the temporal unfolding of three notable social behavioral cues - body movement, eye contact, and smiling - and demonstrated that these behaviors also spontaneously synchronized within dyads. Fourth, we probed the correlates of INS in such synchronized social behaviors. Using cross-correlation and Granger causality analyses, we show that synchronized social behaviors anticipate and in fact Granger cause INS. These results provide proof-of-concept evidence for studying interpersonal neural and behavioral synchrony under natural and unconstrained conditions. Most importantly, the results suggest that INS could be conceptualized as an emergent property of two coupled neural systems: an entrainment phenomenon, promoted by real-time dyadic behavior.
Collapse
Affiliation(s)
- Atesh Koul
- Neuroscience of Perception and Action Lab, Italian Institute of Technology (IIT), Viale Regina Elena 291, Rome, Italy.
| | - Davide Ahmar
- Neuroscience of Perception and Action Lab, Italian Institute of Technology (IIT), Viale Regina Elena 291, Rome, Italy
| | - Gian Domenico Iannetti
- Neuroscience and Behavior Lab, Italian Institute of Technology (IIT), Viale Regina Elena 291, Rome, Italy; Department of Neuroscience, Physiology and Pharmacology, University College London (UCL), WC1E 6BT, London, UK
| | - Giacomo Novembre
- Neuroscience of Perception and Action Lab, Italian Institute of Technology (IIT), Viale Regina Elena 291, Rome, Italy.
| |
Collapse
|
18
|
Xu K, Liu R, Chen X, Chen X, Yang Y, Wang Q, Yang J. Research on brain functions related to visual information processing and body coordination function of pilots based on the low-frequency amplitude method. Front Hum Neurosci 2023; 17:796526. [PMID: 37007677 PMCID: PMC10050347 DOI: 10.3389/fnhum.2023.796526] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2022] [Accepted: 02/27/2023] [Indexed: 03/17/2023] Open
Abstract
ObjectiveResearch on the differences in physiological and psychological mechanisms of practitioners in different occupations is a current hot spot, such as pilots. This study explores the frequency-dependent changes of pilots’ low-frequency amplitudes in the classical frequency band and sub-frequency band between pilots and general occupations. The goal of the current work is to provide objective brain images for the selection and evaluation of outstanding pilots.MethodsTwenty-six pilots and 23 age-, sex-, and education-matched healthy controls were included in this study. Then the mean low-frequency amplitude (mALFF) of the classical frequency band and sub-frequency band was calculated. The two-sample t-test was performed on SPM12 to analyze the differences between the flight group and control group in the classic frequency band. To explore the main effects and the inter-band effects of the mean low-frequency amplitude (mALFF), the mixed design analysis of variance was applied in the sub-frequency bands.ResultsCompared with the control group, left cuneiform lobe and the right cerebellum six area of pilots show significant difference in the classic frequency band. And the main effect results in the sub-frequency bands show that the area with higher mALFF in the flight group is located on the left middle occipital gyrus, the left cuneiform lobe, the right superior occipital gyrus, the right superior gyrus, and the left lateral central lobule. However, the area where the value of mALFF decreased is mainly located on the left rectangular cleft with surrounding cortex and the right dorsolateral superior frontal gyrus. Besides, compared with the slow-4 frequency band, the mALFF of the left middle orbital middle frontal gyrus of the slow-5 frequency band was increased, while the mALFF value of the left putamen, left fusiform gyrus, and right thalamus was decreased. The sensitivity of the slow-5 frequency band and the slow-4 frequency band to the pilots’ different brain areas was also different. Also, the different brain areas in the classic frequency band and the sub-frequency band were significantly correlated with pilots’ flight hours.ConclusionOur findings showed that the left cuneiform brain area and the right cerebellum of pilots changed significantly during resting state. And there was a positive correlation between the mALFF value of those brain area and flight hours. The comparative analysis of sub-frequency bands found that the slow-5 band could elucidate a wider range of different brain regions, providing new ideas for exploring the brain mechanisms of pilots.
Collapse
|
19
|
Pérez A, Davis MH. Speaking and listening to inter-brain relationships. Cortex 2023; 159:54-63. [PMID: 36608420 DOI: 10.1016/j.cortex.2022.12.002] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2022] [Revised: 10/11/2022] [Accepted: 12/06/2022] [Indexed: 12/23/2022]
Abstract
Studies of inter-brain relationships thrive, and yet many reservations regarding their scope and interpretation of these phenomena have been raised by the scientific community. It is thus essential to establish common ground on methodological and conceptual definitions related to this topic and to open debate about any remaining points of uncertainty. We here offer insights to improve the conceptual clarity and empirical standards offered by social neuroscience studies of inter-personal interaction using hyperscanning with a particular focus on verbal communication.
Collapse
Affiliation(s)
- Alejandro Pérez
- MRC Cognition and Brain Sciences Unit, University of Cambridge, UK.
| | - Matthew H Davis
- MRC Cognition and Brain Sciences Unit, University of Cambridge, UK
| |
Collapse
|
20
|
Russo C, Senese VP. Functional near-infrared spectroscopy is a useful tool for multi-perspective psychobiological study of neurophysiological correlates of parenting behaviour. Eur J Neurosci 2023; 57:258-284. [PMID: 36485015 DOI: 10.1111/ejn.15890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2022] [Revised: 12/02/2022] [Accepted: 12/08/2022] [Indexed: 12/13/2022]
Abstract
The quality of the relationship between caregiver and child has long-term effects on the cognitive and socio-emotional development of children. A process involved in human parenting is the bio-behavioural synchrony that occurs between the partners in the relationship during interaction. Through interaction, bio-behavioural synchronicity allows the adaptation of the physiological systems of the parent to those of the child and promotes the positive development and modelling of the child's social brain. The role of bio-behavioural synchrony in building social bonds could be investigated using functional near-infrared spectroscopy (fNIRS). In this paper we have (a) highlighted the importance of the quality of the caregiver-child relationship for the child's cognitive and socio-emotional development, as well as the relevance of infantile stimuli in the activation of parenting behaviour; (b) discussed the tools used in the study of the neurophysiological substrates of the parental response; (c) proposed fNIRS as a particularly suitable tool for the study of parental responses; and (d) underlined the need for a multi-systemic psychobiological approach to understand the mechanisms that regulate caregiver-child interactions and their bio-behavioural synchrony. We propose to adopt a multi-system psychobiological approach to the study of parental behaviour and social interaction.
Collapse
Affiliation(s)
- Carmela Russo
- Psychometric Laboratory, Department of Psychology, University of Campania "Luigi Vanvitelli", Caserta, Italy
| | - Vincenzo Paolo Senese
- Psychometric Laboratory, Department of Psychology, University of Campania "Luigi Vanvitelli", Caserta, Italy
| |
Collapse
|
21
|
Provenzi L, Roberti E, Capelli E. Envisioning translational hyperscanning: how applied neuroscience might improve family-centered care. Soc Cogn Affect Neurosci 2022; 18:6953329. [PMID: 36542821 PMCID: PMC9910277 DOI: 10.1093/scan/nsac061] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/24/2022] [Revised: 11/17/2022] [Accepted: 12/20/2022] [Indexed: 12/24/2022] Open
Abstract
From the very beginning of their life, human beings are immersed in a social and interactive environment that contributes to shaping their social and cognitive development under typical and at-risk conditions. In order to understand human development in its bidirectional relationship with the social environment, we need to develop a 'complexity-sensitive' approach in neuroscience. Recent advances have started to do so with the application of hyperscanning techniques which involve recording adult and child neural activity simultaneously and highlighting the presence of similar patterns of brain activity in the dyad. Numerous studies focused on typically developing children have been published in recent years with the application of this technique to different fields of developmental research. However, hyperscanning techniques could also be extremely beneficial and effective in studying development in atypical and clinical populations. Such application, namely translational hyperscanning, should foster the transition toward a two-brain translational neuroscience. In this paper, we envision how the application of hyperscanning to atypical and clinical child populations can inform family-centered care for children and their parents.
Collapse
Affiliation(s)
- Livio Provenzi
- Correspondence should be addressed to Livio Provenzi, Department of Brain and Behavioral Sciences, University of Pavia, via Mondino 2, Pavia 27100, Italy. E-mail:
| | - Elisa Roberti
- Developmental Psychobiology Lab, IRCCS Mondino Foundation, Pavia 27100, Italy
| | - Elena Capelli
- Developmental Psychobiology Lab, IRCCS Mondino Foundation, Pavia 27100, Italy
| |
Collapse
|
22
|
Batuhan Dirik H, Darendeli A, Ertan H. The New Wireless EEG Device Mentalab Explore is a Valid and Reliable System for the Measurement of Resting state EEG Spectral Features. Brain Res 2022; 1798:148164. [DOI: 10.1016/j.brainres.2022.148164] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Revised: 11/04/2022] [Accepted: 11/08/2022] [Indexed: 11/18/2022]
|
23
|
van Weelden E, Alimardani M, Wiltshire TJ, Louwerse MM. Aviation and neurophysiology: A systematic review. APPLIED ERGONOMICS 2022; 105:103838. [PMID: 35939991 DOI: 10.1016/j.apergo.2022.103838] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/09/2021] [Revised: 06/22/2022] [Accepted: 06/23/2022] [Indexed: 05/24/2023]
Abstract
This paper systematically reviews 20 years of publications (N = 54) on aviation and neurophysiology. The main goal is to provide an account of neurophysiological changes associated with flight training with the aim of identifying neurometrics indicative of pilot's flight training level and task relevant mental states, as well as to capture the current state-of-art of (neuro)ergonomic design and practice in flight training. We identified multiple candidate neurometrics of training progress and workload, such as frontal theta power, the EEG Engagement Index and the Cognitive Stability Index. Furthermore, we discovered that several types of classifiers could be used to accurately detect mental states, such as the detection of drowsiness and mental fatigue. The paper advances practical guidelines on terminology usage, simulator fidelity, and multimodality, as well as future research ideas including the potential of Virtual Reality flight simulations for training, and a brain-computer interface for flight training.
Collapse
Affiliation(s)
- Evy van Weelden
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands.
| | - Maryam Alimardani
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands
| | - Travis J Wiltshire
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands
| | - Max M Louwerse
- Department of Cognitive Science & Artificial Intelligence, Tilburg University, the Netherlands
| |
Collapse
|
24
|
Lu H, Zhang Y, Huang P, Zhang Y, Cheng S, Zhu X. Transcranial Electrical Stimulation Offers the Possibility of Improving Teamwork Among Military Pilots: A Review. Front Neurosci 2022; 16:931265. [PMID: 35911997 PMCID: PMC9327643 DOI: 10.3389/fnins.2022.931265] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2022] [Accepted: 06/24/2022] [Indexed: 11/13/2022] Open
Abstract
Effective teamwork among military pilots is key to successful mission completion. The underlying neural mechanism of teamwork is thought to be inter-brain synchronization (IBS). IBS could also be explained as an incidental phenomenon of cooperative behavior, but the causality between IBS and cooperative behavior could be clarified by directly producing IBS through extra external stimuli applied to functional brain regions. As a non-invasive technology for altering brain function, transcranial electrical stimulation might have the potential to explore whether top-down enhancement of the synchronization of multiple brains can change cooperative behavioral performance among members of a team. This review focuses on the characteristic features of teamwork among military pilots and variations in neuroimaging obtained by hyper-scanning. Furthermore, we discuss the possibility that transcranial electrical stimulation could be used to improve teamwork among military pilots, try to provide a feasible design for doing so, and emphasize crucial aspects to be addressed by future research.
Collapse
|
25
|
Wikström V, Saarikivi K, Falcon M, Makkonen T, Martikainen S, Putkinen V, Cowley BU, Tervaniemi M. Inter-brain synchronization occurs without physical co-presence during cooperative online gaming. Neuropsychologia 2022; 174:108316. [PMID: 35810882 DOI: 10.1016/j.neuropsychologia.2022.108316] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 07/01/2022] [Accepted: 07/02/2022] [Indexed: 11/25/2022]
Abstract
Inter-brain synchronization during social interaction has been linked with several positive phenomena, including closeness, cooperation, prosociality, and team performance. However, the temporal dynamics of inter-brain synchronization during collaboration are not yet fully understood. Furthermore, with collaboration increasingly happening online, the dependence of inter-brain phase synchronization of oscillatory activity on physical presence is an important but understudied question. In this study, physically isolated participants performed a collaborative coordination task in the form of a cooperative multiplayer game. We measured EEG from 42 subjects working together as pairs in the task. During the measurement, the only interaction between the participants happened through on-screen movement of a racing car, controlled by button presses of both participants working with distinct roles, either controlling the speed or the direction of the car. Pairs working together in the task were found to have elevated neural coupling in the alpha, beta, and gamma frequency bands, compared to performance matched false pairs. Higher gamma synchrony was associated with better momentary performance within dyads and higher alpha synchrony was associated with better mean performance across dyads. These results are in line with previous findings of increased inter-brain synchrony during interaction, and show that phase synchronization of oscillatory activity occurs during online real-time joint coordination without any physical co-presence or video and audio connection. Synchrony decreased during a playing session, but was found to be higher during the second session compared to the first. The novel paradigm, developed for the measurement of real-time collaborative performance, demonstrates that changes in inter-brain EEG phase synchrony can be observed continuously during interaction.
Collapse
Affiliation(s)
- Valtteri Wikström
- Cognitive Brain Research Unit, Faculty of Medicine, University of Helsinki, P.O. Box 21, Helsinki, 00014, Finland; Department of Education, Faculty of Educational Sciences, University of Helsinki, P.O. Box 9, Helsinki, 00014, Finland.
| | - Katri Saarikivi
- Cognitive Brain Research Unit, Faculty of Medicine, University of Helsinki, P.O. Box 21, Helsinki, 00014, Finland; Department of Education, Faculty of Educational Sciences, University of Helsinki, P.O. Box 9, Helsinki, 00014, Finland
| | - Mari Falcon
- Cognitive Brain Research Unit, Faculty of Medicine, University of Helsinki, P.O. Box 21, Helsinki, 00014, Finland; Department of Education, Faculty of Educational Sciences, University of Helsinki, P.O. Box 9, Helsinki, 00014, Finland
| | - Tommi Makkonen
- Cognitive Brain Research Unit, Faculty of Medicine, University of Helsinki, P.O. Box 21, Helsinki, 00014, Finland
| | - Silja Martikainen
- Cognitive Brain Research Unit, Faculty of Medicine, University of Helsinki, P.O. Box 21, Helsinki, 00014, Finland; Department of Education, Faculty of Educational Sciences, University of Helsinki, P.O. Box 9, Helsinki, 00014, Finland
| | - Vesa Putkinen
- Turku PET Centre, University of Turku, P.O. Box 52, Turku, 20521, Finland; Turku University Hospital, P.O. Box 52, Turku, 20521, Finland
| | - Benjamin Ultan Cowley
- Department of Education, Faculty of Educational Sciences, University of Helsinki, P.O. Box 9, Helsinki, 00014, Finland; Cognitive Science, Department of Digital Humanities, Faculty of Arts, University of Helsinki, P.O. Box 24, Helsinki, 00014, Finland
| | - Mari Tervaniemi
- Cognitive Brain Research Unit, Faculty of Medicine, University of Helsinki, P.O. Box 21, Helsinki, 00014, Finland; Department of Education, Faculty of Educational Sciences, University of Helsinki, P.O. Box 9, Helsinki, 00014, Finland
| |
Collapse
|
26
|
Andrea B, Atiqah A, Gianluca E. Reproducible Inter-Personal Brain Coupling Measurements in Hyperscanning Settings With functional Near Infra-Red Spectroscopy. Neuroinformatics 2022; 20:665-675. [PMID: 34716564 DOI: 10.1007/s12021-021-09551-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/21/2021] [Indexed: 12/31/2022]
Abstract
Despite a huge advancement in neuroimaging techniques and growing importance of inter-personal brain research, few studies assess the most appropriate computational methods to measure brain-brain coupling. Here, we focus on the signal processing methods to detect brain-coupling in dyads. From a public dataset of functional Near Infra-Red Spectroscopy signals (N=24 dyads), we derived a synthetic control condition by randomization, we investigated the effectiveness of four most used signal similarity metrics: Cross Correlation, Mutual Information, Wavelet Coherence and Dynamic Time Warping. We also accounted for temporal variations between signals by allowing for misalignments up to a maximum lag. Starting from the observed effect sizes, computed in terms of Cohen's d, the power analysis indicated that a high sample size ([Formula: see text]) would be required to detect significant brain-coupling. We therefore discuss the need for specialized statistical approaches and propose bootstrap as an alternative method to avoid over-penalizing the results. In our settings, and based on bootstrap analyses, Cross Correlation and Dynamic Time Warping outperform Mutual Information and Wavelet Coherence for all considered maximum lags, with reproducible results. These results highlight the need to set specific guidelines as the high degree of customization of the signal processing procedures prevents the comparability between studies, their reproducibility and, ultimately, undermines the possibility of extracting new knowledge.
Collapse
Affiliation(s)
- Bizzego Andrea
- Department of Psychology and Cognitive Science, University of Trento, Trento, Italy
| | - Azhari Atiqah
- Psychology Program, School of Social Sciences, Nanyang Technological University, Singapore, Singapore
| | - Esposito Gianluca
- Department of Psychology and Cognitive Science, University of Trento, Trento, Italy. .,Psychology Program, School of Social Sciences, Nanyang Technological University, Singapore, Singapore. .,Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.
| |
Collapse
|
27
|
Borghini G, Arico P, Di Flumeri G, Sciaraffa N, Di Florio A, Ronca V, Giorgi A, Mezzadri L, Gasparini R, Tartaglino R, Trettel A, Babiloni F. Real-time Pilot Crew's Mental Workload and Arousal Assessment During Simulated Flights for Training Evaluation: a Case Study. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2022; 2022:3568-3571. [PMID: 36086259 DOI: 10.1109/embc48229.2022.9871893] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
Training assessment is usually done by evaluating information derived from instructor's supervision related to the pilot's operational performance and behavior. However, this approach lacks objective measures, especially regarding the pilots' mental states while accomplishing the flight training tasks. The study therefore aimed at developing and testing a method for gathering and analyzing in real-time pilots' brain activity and skin conductance to improve the training evaluation. In this regard, Novice pilots' neurophysiological signals were acquired throughout multi-crew training sessions. The results demonstrated how the methodology proposed was able to endow real-time pilots' mental workload and arousal assessment for i) better evaluating training progress and operational behavior during the training session, and ii) for objectively comparing different training sessions.
Collapse
|
28
|
“We Will Let You Know”: An Assessment of Digital vs. Face-to-Face Job Interviews via EEG Connectivity Analysis. INFORMATION 2022. [DOI: 10.3390/info13070312] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
We focused on job interviews as critical examples of complex social interaction in organizational contexts. We aimed at investigating the effect of face-to-face vs. computer-mediated interaction, of role (candidate, recruiter), and of the interview phase (introductory, attitudinal, technical, conclusive) on intra-brain and inter-brain connectivity measures and autonomic synchronization. Twenty expert recruiters and potential candidates took part in a hyperscanning investigation. Namely, electroencephalography (delta, theta, alpha, beta bands) and autonomic (skin-conductance, heart-rate) data were collected in candidate-recruiter dyads during a simulated job interview and then concurrently analyzed. Analyses highlighted a link between face-to-face condition and greater intra-/inter-brain connectivity indices in delta and theta bands. Furthermore, intra-brain and inter-brain connectivity measures were higher for delta and theta bands in the final interview phases compared to the first ones. Consistently, autonomic synchronization was higher during the final interview phases, specifically in the face-to-face condition. Finally, recruiters showed higher intra-brain connectivity in the delta range over frontal and temporoparietal areas, while candidates showed higher intra-brain connectivity in the theta range over frontal areas. Findings highlight the value of hyperscanning investigations in exploring social attunement in professional contexts and hint at their potential to foster neuroscience-informed practices in human resource management processes.
Collapse
|
29
|
Holroyd CB. Interbrain synchrony: on wavy ground. Trends Neurosci 2022; 45:346-357. [PMID: 35236639 DOI: 10.1016/j.tins.2022.02.002] [Citation(s) in RCA: 36] [Impact Index Per Article: 18.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2021] [Revised: 01/08/2022] [Accepted: 02/04/2022] [Indexed: 12/15/2022]
Abstract
In recent years the study of dynamic, between-brain coupling mechanisms has taken social neuroscience by storm. In particular, interbrain synchrony (IBS) is a putative neural mechanism said to promote social interactions by enabling the functional integration of multiple brains. In this article, I argue that this research is beset with three pervasive and interrelated problems. First, the field lacks a widely accepted definition of IBS. Second, IBS wants for theories that can guide the design and interpretation of experiments. Third, a potpourri of tasks and empirical methods permits undue flexibility when testing the hypothesis. These factors synergistically undermine IBS as a theoretical construct. I finish by recommending measures that can address these issues.
Collapse
Affiliation(s)
- Clay B Holroyd
- Department of Experimental Psychology, Ghent University, Henri Dunantlaan 2, 9000 Gent, Belgium.
| |
Collapse
|
30
|
Müller V. Neural Synchrony and Network Dynamics in Social Interaction: A Hyper-Brain Cell Assembly Hypothesis. Front Hum Neurosci 2022; 16:848026. [PMID: 35572007 PMCID: PMC9101304 DOI: 10.3389/fnhum.2022.848026] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/03/2022] [Accepted: 03/25/2022] [Indexed: 11/13/2022] Open
Abstract
Mounting neurophysiological evidence suggests that interpersonal interaction relies on continual communication between cell assemblies within interacting brains and continual adjustments of these neuronal dynamic states between the brains. In this Hypothesis and Theory article, a Hyper-Brain Cell Assembly Hypothesis is suggested on the basis of a conceptual review of neural synchrony and network dynamics and their roles in emerging cell assemblies within the interacting brains. The proposed hypothesis states that such cell assemblies can emerge not only within, but also between the interacting brains. More precisely, the hyper-brain cell assembly encompasses and integrates oscillatory activity within and between brains, and represents a common hyper-brain unit, which has a certain relation to social behavior and interaction. Hyper-brain modules or communities, comprising nodes across two or several brains, are considered as one of the possible representations of the hypothesized hyper-brain cell assemblies, which can also have a multidimensional or multilayer structure. It is concluded that the neuronal dynamics during interpersonal interaction is brain-wide, i.e., it is based on common neuronal activity of several brains or, more generally, of the coupled physiological systems including brains.
Collapse
Affiliation(s)
- Viktor Müller
- Center for Lifespan Psychology, Max Planck Institute for Human Development, Berlin, Germany
| |
Collapse
|
31
|
It Takes Two: Interpersonal Neural Synchrony Is Increased after Musical Interaction. Brain Sci 2022; 12:brainsci12030409. [PMID: 35326366 PMCID: PMC8946180 DOI: 10.3390/brainsci12030409] [Citation(s) in RCA: 8] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2021] [Revised: 03/11/2022] [Accepted: 03/13/2022] [Indexed: 02/05/2023] Open
Abstract
Music’s deeply interpersonal nature suggests that music-derived neuroplasticity relates to interpersonal temporal dynamics, or synchrony. Interpersonal neural synchrony (INS) has been found to correlate with increased behavioral synchrony during social interactions and may represent mechanisms that support them. As social interactions often do not have clearly delineated boundaries, and many start and stop intermittently, we hypothesize that a neural signature of INS may be detectable following an interaction. The present study aimed to investigate this hypothesis using a pre-post paradigm, measuring interbrain phase coherence before and after a cooperative dyadic musical interaction. Ten dyads underwent synchronous electroencephalographic (EEG) recording during silent, non-interactive periods before and after a musical interaction in the form of a cooperative tapping game. Significant post-interaction increases in delta band INS were found in the post-condition and were positively correlated with the duration of the preceding interaction. These findings suggest a mechanism by which social interaction may be efficiently continued after interruption and hold the potential for measuring neuroplastic adaption in longitudinal studies. These findings also support the idea that INS during social interaction represents active mechanisms for maintaining synchrony rather than mere parallel processing of stimuli and motor activity.
Collapse
|
32
|
Park J, Shin J, Jeong J. Inter-Brain Synchrony Levels According to Task Execution Modes and Difficulty Levels: an fNIRS/GSR Study. IEEE Trans Neural Syst Rehabil Eng 2022; 30:194-204. [PMID: 35041606 DOI: 10.1109/tnsre.2022.3144168] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Hyperscanning is a brain imaging technique that measures brain synchrony caused by social interactions. Recent research on hyperscanning has revealed substantial inter-brain synchrony (IBS), but little is known about the link between IBS and mental workload. To study this link, we conducted an experiment consisting of button-pressing tasks of three different difficulty levels for the cooperation and competition modes with 56 participants aged 23.7±3.8 years (mean±standard deviation). We attempted to observe IBS using functional near-infrared spectroscopy (fNIRS) and galvanic skin response (GSR) to assess the activities of the human autonomic nervous system. We found that the IBS levels increased in a frequency band of 0.075-0.15 Hz, which was unrelated to the task repetition frequency in the cooperation mode according to the task difficulty level. Significant relative inter-brain synchrony (RIBS) increases were observed in three and 10 channels out of 15 for the hard tasks compared to the normal and easy tasks, respectively. We observed that the average GSR values increased with increasing task difficulty levels for the competition mode only. Thus, our results suggest that the IBS revealed by fNIRS and GSR is not related to the hemodynamic changes induced by mental workload, simple behavioral synchrony such as button-pressing timing, or autonomic nervous system activity. IBS is thus explicitly caused by social interactions such as cooperation.
Collapse
|
33
|
Saul MA, He X, Black S, Charles F. A Two-Person Neuroscience Approach for Social Anxiety: A Paradigm With Interbrain Synchrony and Neurofeedback. Front Psychol 2022; 12:568921. [PMID: 35095625 PMCID: PMC8796854 DOI: 10.3389/fpsyg.2021.568921] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/25/2021] [Accepted: 12/06/2021] [Indexed: 12/11/2022] Open
Abstract
Social anxiety disorder has been widely recognised as one of the most commonly diagnosed mental disorders. Individuals with social anxiety disorder experience difficulties during social interactions that are essential in the regular functioning of daily routines; perpetually motivating research into the aetiology, maintenance and treatment methods. Traditionally, social and clinical neuroscience studies incorporated protocols testing one participant at a time. However, it has been recently suggested that such protocols are unable to directly assess social interaction performance, which can be revealed by testing multiple individuals simultaneously. The principle of two-person neuroscience highlights the interpersonal aspect of social interactions that observes behaviour and brain activity from both (or all) constituents of the interaction, rather than analysing on an individual level or an individual observation of a social situation. Therefore, two-person neuroscience could be a promising direction for assessment and intervention of the social anxiety disorder. In this paper, we propose a novel paradigm which integrates two-person neuroscience in a neurofeedback protocol. Neurofeedback and interbrain synchrony, a branch of two-person neuroscience, are discussed in their own capacities for their relationship with social anxiety disorder and relevance to the paradigm. The newly proposed paradigm sets out to assess the social interaction performance using interbrain synchrony between interacting individuals, and to employ a multi-user neurofeedback protocol for intervention of the social anxiety.
Collapse
Affiliation(s)
- Marcia A. Saul
- Faculty of Media and Communication, Centre for Digital Entertainment, Bournemouth University, Poole, United Kingdom
| | - Xun He
- Department of Psychology, Faculty of Science and Technology, Bournemouth University, Poole, United Kingdom
- *Correspondence: Xun He
| | - Stuart Black
- Applied Neuroscience Solutions Ltd., Frimley Green, United Kingdom
| | - Fred Charles
- Department of Creative Technology, Faculty of Science and Technology, Bournemouth University, Poole, United Kingdom
- Fred Charles
| |
Collapse
|
34
|
Chabin T, Gabriel D, Comte A, Haffen E, Moulin T, Pazart L. Interbrain emotional connection during music performances is driven by physical proximity and individual traits. Ann N Y Acad Sci 2021; 1508:178-195. [PMID: 34750828 DOI: 10.1111/nyas.14711] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Revised: 09/30/2021] [Accepted: 10/06/2021] [Indexed: 12/23/2022]
Abstract
How musical emotions and the pleasure derived from music, regardless of the musical valence, can be shared between individuals is a fascinating question, and investigating it can shed light on the function of musical reward. We carried out our investigations in a natural setting during an international competition for orchestra conductors. Participants (n = 15) used a dedicated smartphone app to report their subjective emotional experiences in real time while we recorded their cerebral activity using electroencephalography and their electrodermal activity. The overall behavioral real-time behavioral ratings suggest a possible social influence on the reported and felt pleasure. The physically closer the participants, the more similar their reported pleasure. By calculating the interindividual cerebral coherence (n = 21 pairs), we showed that when people simultaneously reported either high or low pleasure, their cerebral activities were closer than for simultaneous neutral pleasure reports. Participants' skin conductance levels were also more coupled when reporting higher emotional degrees simultaneously. More importantly, the participants who were physically closer had higher cerebral coherence, but only when they simultaneously reported a high level of pleasure. We propose that emotional contagion and/or emotional resonance mechanisms could explain why a form of "emotional connecting force" arises between people during shared appraisal situations.
Collapse
Affiliation(s)
- Thibault Chabin
- Centre Hospitalier Universitaire de Besançon, Centre d'Investigation Clinique INSERM CIC 1431, Besançon, France.,Plateforme de Neuroimagerie Fonctionnelle et Neurostimulation Neuraxess, Centre Hospitalier Universitaire de Besançon, Université de Bourgogne Franche-Comté, Besançon, France.,Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive, Université Bourgogne Franche-Comté, Besançon, France
| | - Damien Gabriel
- Centre Hospitalier Universitaire de Besançon, Centre d'Investigation Clinique INSERM CIC 1431, Besançon, France.,Plateforme de Neuroimagerie Fonctionnelle et Neurostimulation Neuraxess, Centre Hospitalier Universitaire de Besançon, Université de Bourgogne Franche-Comté, Besançon, France.,Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive, Université Bourgogne Franche-Comté, Besançon, France
| | - Alexandre Comte
- Centre Hospitalier Universitaire de Besançon, Centre d'Investigation Clinique INSERM CIC 1431, Besançon, France.,Plateforme de Neuroimagerie Fonctionnelle et Neurostimulation Neuraxess, Centre Hospitalier Universitaire de Besançon, Université de Bourgogne Franche-Comté, Besançon, France.,Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive, Université Bourgogne Franche-Comté, Besançon, France
| | - Emmanuel Haffen
- Centre Hospitalier Universitaire de Besançon, Centre d'Investigation Clinique INSERM CIC 1431, Besançon, France.,Plateforme de Neuroimagerie Fonctionnelle et Neurostimulation Neuraxess, Centre Hospitalier Universitaire de Besançon, Université de Bourgogne Franche-Comté, Besançon, France.,Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive, Université Bourgogne Franche-Comté, Besançon, France
| | - Thierry Moulin
- Centre Hospitalier Universitaire de Besançon, Centre d'Investigation Clinique INSERM CIC 1431, Besançon, France.,Plateforme de Neuroimagerie Fonctionnelle et Neurostimulation Neuraxess, Centre Hospitalier Universitaire de Besançon, Université de Bourgogne Franche-Comté, Besançon, France.,Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive, Université Bourgogne Franche-Comté, Besançon, France
| | - Lionel Pazart
- Centre Hospitalier Universitaire de Besançon, Centre d'Investigation Clinique INSERM CIC 1431, Besançon, France.,Plateforme de Neuroimagerie Fonctionnelle et Neurostimulation Neuraxess, Centre Hospitalier Universitaire de Besançon, Université de Bourgogne Franche-Comté, Besançon, France.,Laboratoire de Recherches Intégratives en Neurosciences et Psychologie Cognitive, Université Bourgogne Franche-Comté, Besançon, France
| |
Collapse
|
35
|
Léné P, Karran AJ, Labonté-Lemoyne E, Sénécal S, Fredette M, Johnson KJ, Léger PM. Is there collaboration specific neurophysiological activation during collaborative task activity? An analysis of brain responses using electroencephalography and hyperscanning. Brain Behav 2021; 11:e2270. [PMID: 34617691 PMCID: PMC8613430 DOI: 10.1002/brb3.2270] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/23/2020] [Revised: 06/02/2021] [Accepted: 06/14/2021] [Indexed: 12/20/2022] Open
Abstract
Collaboration between two individuals is thought to be associated with the synchrony of two different brain activities. Indeed, prefrontal cortical activation and alpha frequency band modulation has been widely reported, but little is known about interbrain synchrony (IBS) changes occurring during social interaction such as collaboration or competition. In this study, we assess the dynamic of IBS variation in order to provide novel insights into the frequency band modulation underlying collaboration. To address this question, we used electroencephalography (EEG) to simultaneously record the brain activity of two individuals playing a computer-based game facing four different conditions: collaboration, competition, single participation, and passive observation. The computer-based game consisted of a fast button response task. Using data recorded in sensor space, we calculated an IBS value for each frequency band using both wavelet coherence transform and phase-locking value and performed single-subject analysis to compare each condition. We found significant IBS in frontal electrodes only present during collaboration associated with alpha frequency band modulation. In addition, we observed significant IBS in the theta frequency band for both collaboration and competition conditions, along with a significant single-subject cortical activity. Competition is distinguishable through single-subject activity in several regions and frequency bands of the brain. Performance is correlated with single-subject frontal activation during collaboration in the alpha and beta frequency band.
Collapse
Affiliation(s)
- Paul Léné
- Département de management, HEC Montréal, Montréal, Quebec, Canada
| | - Alexander J Karran
- Département de technologies de l'information, HEC Montréal, Montréal, Quebec, Canada
| | - Elise Labonté-Lemoyne
- Département de technologies de l'information, HEC Montréal, Montréal, Quebec, Canada
| | - Sylvain Sénécal
- Département de technologies de l'information, HEC Montréal, Montréal, Quebec, Canada
| | - Marc Fredette
- Département de technologies de l'information, HEC Montréal, Montréal, Quebec, Canada
| | - Kevin J Johnson
- Département de management, HEC Montréal, Montréal, Quebec, Canada
| | | |
Collapse
|
36
|
Susnoschi Luca I, Putri FD, Ding H, Vuckovič A. Brain Synchrony in Competition and Collaboration During Multiuser Neurofeedback-Based Gaming. FRONTIERS IN NEUROERGONOMICS 2021; 2:749009. [PMID: 38235241 PMCID: PMC10790838 DOI: 10.3389/fnrgo.2021.749009] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Accepted: 10/04/2021] [Indexed: 01/19/2024]
Abstract
EEG hyperscanning during multiuser gaming offers opportunities to study brain characteristics of social interaction under various paradigms. In this study, we aimed to characterize neural signatures and phase-based functional connectivity patterns of gaming strategies during collaborative and competitive alpha neurofeedback games. Twenty pairs of participants with no close relationship took part in three sessions of collaborative or competitive multiuser neurofeedback (NF), with identical graphical user interface, using Relative Alpha (RA) power as a control signal. Collaborating dyads had to keep their RA within 5% of each other for the team to be awarded a point, while members of competitive dyads scored points if their RA was 10% above their opponent's. Interbrain synchrony existed only during gaming but not during baseline in either collaborative or competitive gaming. Spectral analysis and interbrain connectivity showed that in collaborative gaming, players with higher resting state alpha content were more active in regulating their RA to match those of their partner. Moreover, interconnectivity was the strongest between homologous brain structures of the dyad in theta and alpha bands, indicating a similar degree of planning and social exchange. Competitive gaming emphasized the difference between participants who were able to relax and, in this way, maintain RA, and those who had an unsuccessful approach. Analysis of interbrain connections shows engagement of frontal areas in losers, but not in winners, indicating the formers' attempt to mentalise and apply strategies that might be suitable for conventional gaming, but inappropriate for the alpha neurofeedback-based game. We show that in gaming based on multiplayer non-verbalized NF, the winning strategy is dependent on the rules of the game and on the behavior of the opponent. Mental strategies that characterize successful gaming in the physical world might not be adequate for NF-based gaming.
Collapse
Affiliation(s)
- Ioana Susnoschi Luca
- Biomedical Research Division, School of Engineering, University of Glasgow, Glasgow, United Kingdom
| | - Finda Dwi Putri
- Biomedical Research Division, School of Engineering, University of Glasgow, Glasgow, United Kingdom
| | - Hao Ding
- Section of Movement Disorders and Neurostimulation, Biomedical Statistics and Multimodal Signal Processing Unit, Department of Neurology, Focus Program Translational Neuroscience (FTN), University Medical Center of the Johannes Gutenberg-University Mainz, Mainz, Germany
| | - Aleksandra Vuckovič
- Biomedical Research Division, School of Engineering, University of Glasgow, Glasgow, United Kingdom
| |
Collapse
|
37
|
Conflict-Related Brain Activity after Individualized Cognitive Training in Preschoolers from Poor Homes. JOURNAL OF COGNITIVE ENHANCEMENT 2021. [DOI: 10.1007/s41465-021-00223-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
38
|
Lübbert A, Göschl F, Krause H, Schneider TR, Maye A, Engel AK. Socializing Sensorimotor Contingencies. Front Hum Neurosci 2021; 15:624610. [PMID: 34602990 PMCID: PMC8480310 DOI: 10.3389/fnhum.2021.624610] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/31/2020] [Accepted: 08/24/2021] [Indexed: 12/27/2022] Open
Abstract
The aim of this review is to highlight the idea of grounding social cognition in sensorimotor interactions shared across agents. We discuss an action-oriented account that emerges from a broader interpretation of the concept of sensorimotor contingencies. We suggest that dynamic informational and sensorimotor coupling across agents can mediate the deployment of action-effect contingencies in social contexts. We propose this concept of socializing sensorimotor contingencies (socSMCs) as a shared framework of analysis for processes within and across brains and bodies, and their physical and social environments. In doing so, we integrate insights from different fields, including neuroscience, psychology, and research on human-robot interaction. We review studies on dynamic embodied interaction and highlight empirical findings that suggest an important role of sensorimotor and informational entrainment in social contexts. Furthermore, we discuss links to closely related concepts, such as enactivism, models of coordination dynamics and others, and clarify differences to approaches that focus on mentalizing and high-level cognitive representations. Moreover, we consider conceptual implications of rethinking cognition as social sensorimotor coupling. The insight that social cognitive phenomena like joint attention, mutual trust or empathy rely heavily on the informational and sensorimotor coupling between agents may provide novel remedies for people with disturbed social cognition and for situations of disturbed social interaction. Furthermore, our proposal has potential applications in the field of human-robot interaction where socSMCs principles might lead to more natural and intuitive interfaces for human users.
Collapse
Affiliation(s)
- Annika Lübbert
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Florian Göschl
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Hanna Krause
- Department of Psychiatry and Psychotherapy, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Till R. Schneider
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Alexander Maye
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Andreas K. Engel
- Department of Neurophysiology and Pathophysiology, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
39
|
Maltez-Laurienti A, Minniear A, Moore R, McGovern T, Newman P, Brearly T. Exploring Neurocognitive Performance Differences in Military Aviation Personnel. Aerosp Med Hum Perform 2021; 92:702-709. [PMID: 34645550 DOI: 10.3357/amhp.5788.2021] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/24/2022]
Abstract
INTRODUCTION: Military aeromedical evaluations are common, but specialized neuropsychological norms for aviation personnel are sparse, resulting in a need to rely on norms from the general population. Little has been published regarding aviation subpopulations and how their neuropsychological profiles may differ from general population normative data. This study investigated neuropsychological test results of aeromedical service members to evaluate consistency with general population norms, and to delineate differences between aviation subpopulations.METHODS: Analyses were conducted on demographic variables and test scores of military aviators (N 26) and nonaviator crewmembers (N 36) referred for evaluation due to a clinical problem requiring a waiver for flight status. Performance differences between subsamples were investigated with general linear modeling. Base rates for low scores were described.RESULTS: Mean test scores in both subsamples were 0 to 1 standard deviation (SD) above the general populations means, with the largest discrepancies being found on measures of visuospatial ability (crewmembers) and verbal learning (aviators). Modeling revealed a significant difference between aviators and crewmembers on Trail Making Test Part B, after accounting for education. Aviators produced fewer low scores than crewmembers, even when using education adjusted normative data.DISCUSSION: Results suggest the cognitive profile of aviators is uniquely strong in specific domains, with fewer low scores. The development of aviator-specific norms may enhance sensitivity to cognitive decrements in this population. Future studies might separately assess crewmember roles to further assess cognitive performance standards across specialties.Maltez-Laurienti A, Minniear A, Moore R, McGovern T, Newman P, Brearly T. Exploring neurocognitive performance differences in military aviation personnel. Aerosp Med Hum Perform. 2021; 92(9):702709.
Collapse
|
40
|
Cho PS, Escoffier N, Mao Y, Green C, Davis RC. Beyond physical entrainment: competitive and cooperative mental stances during identical joint-action tasks differently affect inter-subjective neural synchrony and judgments of agency. Soc Neurosci 2021; 15:368-379. [PMID: 32031918 DOI: 10.1080/17470919.2020.1727949] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/07/2023]
Abstract
Little work has examined how mental stance alone, apart from physical entrainment, affects between-participant neural synchrony during joint social interaction. We report the first findings on how cooperative and competitive mental stances, even during identical visuomotor joint-action tasks, result in distinct neural oscillatory signatures in low beta and theta band between-participant phase synchrony. Two participants jointly controlled a cursor and were instructed to either compete or cooperate to move it to one of three targets. The visuomotor output was identical for both the compete and cooperate conditions because participants were privately given the same target for experimental trials. Cooperation enhanced theta band between-participant phase-locking value (PLV) midtrial at 1-2 seconds, reflecting activation of systems for social coordination to move the cursor in a shared direction. Competition enhanced low beta between-participant PLV, shifting from temporal to frontal regions, indicating that participants focused only on their target and later evaluated self-agency as winner or loser. This interpretation of the neural signature was corroborated by participants' greater post-trial ratings of the degree of control over the cursor during competition. Top-down cooperative and competitive mental stances shape perceptions of social context and affect interpersonal neural synchrony important for representation of self and others' actions.
Collapse
Affiliation(s)
- Philip S Cho
- Center for Science and Engineering Applications in Social Science, Yonsei University , Seoul, Korea.,Underwood International College, Yonsei University , Seoul, Korea
| | - Nicolas Escoffier
- Department of Psychology, National University of Singapore , Singapore.,Life Sciences Institute Programme in Neurobiology and Ageing, National University of Singapore , Singapore
| | - Yinan Mao
- Department of Statistics, National University of Singapore , Singapore
| | - Christopher Green
- Diagnostic Radiology and Psychiatry, Wayne State School of Medicine , Detroit, MI, USA
| | - Richard C Davis
- School of Information Systems, Singapore Management University , Singapore, Singapore
| |
Collapse
|
41
|
Liu H, Zhao C, Wang F, Zhang D. Inter-brain amplitude correlation differentiates cooperation from competition in a motion-sensing sports game. Soc Cogn Affect Neurosci 2021; 16:552-564. [PMID: 33693825 PMCID: PMC8138086 DOI: 10.1093/scan/nsab031] [Citation(s) in RCA: 13] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/22/2020] [Revised: 02/07/2021] [Accepted: 03/08/2021] [Indexed: 11/23/2022] Open
Abstract
Cooperation and competition are two basic modes of human interaction. Their underlying neural mechanisms, especially from an interpersonal perspective, have not been fully explored. Using the electroencephalograph-based hyperscanning technique, the present study investigated the neural correlates of both cooperation and competition within the same ecological paradigm using a classic motion-sensing tennis game. Both the inter-brain coupling (the inter-brain amplitude correlation and inter-brain phase-locking) and the intra-brain spectral power were analyzed. Only the inter-brain amplitude correlation showed a significant difference between cooperation and competition, with different spatial patterns at theta, alpha and beta frequency bands. Further inspection revealed distinct inter-brain coupling patterns for cooperation and competition; cooperation elicited positive inter-brain amplitude correlation at the delta and theta bands in extensive brain regions, while competition was associated with negative occipital inter-brain amplitude correlation at the alpha and beta bands. These findings add to our knowledge of the neural mechanisms of cooperation and competition and suggest the significance of adopting an inter-brain perspective in exploring the neural underpinnings of social interaction in ecological contexts.
Collapse
Affiliation(s)
- Huashuo Liu
- Department of Psychology, School of Social Sciences, Tsinghua University, Beijing 100084, China
| | - Chenying Zhao
- Department of Biomedical Engineering, School of Medicine, Tsinghua University, Beijing 100084, China
| | - Fei Wang
- Department of Psychology, School of Social Sciences, Tsinghua University, Beijing 100084, China.,Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China
| | - Dan Zhang
- Department of Psychology, School of Social Sciences, Tsinghua University, Beijing 100084, China.,Tsinghua Laboratory of Brain and Intelligence, Tsinghua University, Beijing 100084, China
| |
Collapse
|
42
|
Yang Y, Wu M, Vázquez-Guardado A, Wegener AJ, Grajales-Reyes JG, Deng Y, Wang T, Avila R, Moreno JA, Minkowicz S, Dumrongprechachan V, Lee J, Zhang S, Legaria AA, Ma Y, Mehta S, Franklin D, Hartman L, Bai W, Han M, Zhao H, Lu W, Yu Y, Sheng X, Banks A, Yu X, Donaldson ZR, Gereau RW, Good CH, Xie Z, Huang Y, Kozorovitskiy Y, Rogers JA. Wireless multilateral devices for optogenetic studies of individual and social behaviors. Nat Neurosci 2021; 24:1035-1045. [PMID: 33972800 PMCID: PMC8694284 DOI: 10.1038/s41593-021-00849-x] [Citation(s) in RCA: 75] [Impact Index Per Article: 25.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2019] [Accepted: 03/26/2021] [Indexed: 12/31/2022]
Abstract
Advanced technologies for controlled delivery of light to targeted locations in biological tissues are essential to neuroscience research that applies optogenetics in animal models. Fully implantable, miniaturized devices with wireless control and power-harvesting strategies offer an appealing set of attributes in this context, particularly for studies that are incompatible with conventional fiber-optic approaches or battery-powered head stages. Limited programmable control and narrow options in illumination profiles constrain the use of existing devices. The results reported here overcome these drawbacks via two platforms, both with real-time user programmability over multiple independent light sources, in head-mounted and back-mounted designs. Engineering studies of the optoelectronic and thermal properties of these systems define their capabilities and key design considerations. Neuroscience applications demonstrate that induction of interbrain neuronal synchrony in the medial prefrontal cortex shapes social interaction within groups of mice, highlighting the power of real-time subject-specific programmability of the wireless optogenetic platforms introduced here.
Collapse
Affiliation(s)
- Yiyuan Yang
- Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
| | - Mingzheng Wu
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
| | | | - Amy J Wegener
- US Army Research Laboratory, Aberdeen Proving Ground, MD, USA
- US Army Medical Research Institute of Chemical Defense, Aberdeen Proving Ground, MD, USA
| | - Jose G Grajales-Reyes
- Washington University Pain Center and Department of Anesthesiology, Washington University, St. Louis, MO, USA
| | - Yujun Deng
- Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
- State Key Laboratory of Mechanical System and Vibration, Shanghai Jiao Tong University, Shanghai, China
- Department of Civil and Environmental Engineering, Northwestern University, Evanston, IL, USA
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
| | - Taoyi Wang
- Department of Physics, Tsinghua University, Beijing, China
| | - Raudel Avila
- Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
- Department of Civil and Environmental Engineering, Northwestern University, Evanston, IL, USA
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
| | - Justin A Moreno
- US Army Research Laboratory, Aberdeen Proving Ground, MD, USA
- US Army Medical Research Institute of Chemical Defense, Aberdeen Proving Ground, MD, USA
- SURVICE Engineering, Belcamp, MD, USA
| | - Samuel Minkowicz
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
| | - Vasin Dumrongprechachan
- Department of Neurobiology, Northwestern University, Evanston, IL, USA
- Chemistry of Life Processes Institutes, Northwestern University, Evanston, IL, USA
| | | | - Shuangyang Zhang
- Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
- Department of Civil and Environmental Engineering, Northwestern University, Evanston, IL, USA
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
- School of Civil Engineering, Southwest JiaoTong University, Chengdu, China
| | - Alex A Legaria
- Washington University Pain Center and Department of Anesthesiology, Washington University, St. Louis, MO, USA
| | - Yuhang Ma
- School of Chemical Engineering and Technology, Tianjin University, Tianjin, People's Republic of China
| | - Sunita Mehta
- CSIR-Central Scientific Instruments Organization, Ministry of Science & Technology, Sector 30-C, Chandigarh, India
| | - Daniel Franklin
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
| | - Layne Hartman
- Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA
| | - Wubin Bai
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA
| | - Mengdi Han
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
| | - Hangbo Zhao
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
| | - Wei Lu
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
| | - Yongjoon Yu
- Chemistry of Life Processes Institutes, Northwestern University, Evanston, IL, USA
| | - Xing Sheng
- Department of Electronic Engineering, Tsinghua University, Beijing, China
| | - Anthony Banks
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
- Neurolux Inc, Evanston, IL, USA
- Simpson Querrey Institute & Feinberg Medical School, Northwestern University, Evanston, IL, USA
| | - Xinge Yu
- Department of Biomedical Engineering, City University of Hong Kong, Kowloong Tong, Hong Kong
| | - Zoe R Donaldson
- Psychology and Neuroscience, Molecular Cellular and Developmental Biology, University of Colorado Boulder, Boulder, CO, USA
| | - Robert W Gereau
- Washington University Pain Center and Department of Anesthesiology, Washington University, St. Louis, MO, USA
| | - Cameron H Good
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA
- US Army Research Laboratory, Aberdeen Proving Ground, MD, USA
- US Army Medical Research Institute of Chemical Defense, Aberdeen Proving Ground, MD, USA
| | - Zhaoqian Xie
- State Key Laboratory of Structural Analysis for Industrial Equipment, Department of Engineering Mechanics, Dalian University of Technology, Dalian, P.R. China.
| | - Yonggang Huang
- Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA.
- Center for Bio-Integrated Electronics, Northwestern University, Evanston, IL, USA.
- Department of Civil and Environmental Engineering, Northwestern University, Evanston, IL, USA.
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA.
| | - Yevgenia Kozorovitskiy
- Department of Neurobiology, Northwestern University, Evanston, IL, USA.
- Chemistry of Life Processes Institutes, Northwestern University, Evanston, IL, USA.
| | - John A Rogers
- Department of Mechanical Engineering, Northwestern University, Evanston, IL, USA.
- Department of Materials Science and Engineering, Northwestern University, Evanston, IL, USA.
- Neurolux Inc, Evanston, IL, USA.
- Department of Biomedical Engineering, Northwestern University, Evanston, IL, USA.
- Simpson Querrey Institute & Feinberg Medical School, Northwestern University, Evanston, IL, USA.
- Department of Chemistry, Northwestern University, Evanston, IL, USA.
- Department of Neurological Surgery, Northwestern University, Evanston, IL, USA.
- Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, USA.
- Department of Computer Science, Northwestern University, Evanston, IL, USA.
| |
Collapse
|
43
|
Richard C, Karić MS, McConnell M, Poole J, Rupp G, Fink A, Meghdadi A, Berka C. Elevated Inter-Brain Coherence Between Subjects With Concordant Stances During Discussion of Social Issues. Front Hum Neurosci 2021; 15:611886. [PMID: 34054446 PMCID: PMC8160431 DOI: 10.3389/fnhum.2021.611886] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2020] [Accepted: 04/20/2021] [Indexed: 02/03/2023] Open
Abstract
Social media platforms offer convenient, instantaneous social sharing on a mass scale with tremendous impact on public perceptions, opinions, and behavior. There is a need to understand why information spreads including the human motivations, cognitive processes, and neural dynamics of large-scale sharing. This study introduces a novel approach for investigating the effect social media messaging and in-person discussion has on the inter-brain dynamics within small groups of participants. The psychophysiological impact of information campaigns and narrative messaging within a closed social media environment was assessed using 24-channel wireless EEG. Data were acquired from three- or four-person groups while subjects debated contemporary social issues framed by four scenarios of varying controversy: (a) investing in ethical vs. unethical corporations, (b) selecting travel destination based on social awareness, (c) determining verdict in a murder trial and the punishment of life in prison or death penalty, and (d) decision to vaccinate. Pre-/post-scenario questionnaires assess the effects of the social media information. Inter-brain coherence between subject pairs on each social issue discussed by subjects was analyzed by concordance, agreement vs. disagreement, and by group unanimity, unanimous vs. not unanimous. Subject pairs that agreed on the social issues raised in the scenarios had significantly greater inter-brain coherence in gamma frequency range than disagreeing pairs over cortical regions known to be involved in social interactions. These effects were magnified when comparing groups where subject pairs were unanimous in their stance on the social issues for some but not all scenarios. While there was considerable overlap between scenarios in what EEG channels were significant, there was enough variability to indicate the possibility of scenario-specific effects on inter-brain coherence.
Collapse
|
44
|
Farahzadi Y, Kekecs Z. Towards a multi-brain framework for hypnosis: a review of quantitative methods. AMERICAN JOURNAL OF CLINICAL HYPNOSIS 2021; 63:389-403. [PMID: 33999773 DOI: 10.1080/00029157.2020.1865129] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
Most real-world applications of hypnosis involve a pair of actors: a hypnotist and a subject. Accordingly, most current models of hypnosis acknowledge the relevance of social factors in the development of the hypnotic response. Yet, psychophysiological research on hypnosis has been mostly restricted to techniques that are studying one individual, neglecting the complexity of hypnosis as a social phenomenon. In this paper, we review evidence suggesting that a multi-brain approach to studying the psychophysiology of hypnosis could lead to a breakthrough in our understanding of the neural correlates of hypnosis. In particular, we aim to highlight how this approach which relies on the information conveyed by complex verbal stimuli can be utilized to deal with the multifaceted nature of hypnosis. Furthermore, we present analytical approaches to assessing brain-to-brain coupling developed in the field of social cognitive neuroscience in the past decade, to aid the design of similar multi-brain studies in hypnosis research.
Collapse
Affiliation(s)
| | - Zoltan Kekecs
- Eötvös Loránd University, Budapest, Hungary
- Lund University, Lund, Sweden
| |
Collapse
|
45
|
Mancini M, Cherubino P, Cartocci G, Martinez A, Borghini G, Guastamacchia E, di Flumeri G, Rossi D, Modica E, Menicocci S, Lupo V, Trettel A, Babiloni F. Forefront Users' Experience Evaluation by Employing Together Virtual Reality and Electroencephalography: A Case Study on Cognitive Effects of Scents. Brain Sci 2021; 11:256. [PMID: 33670698 PMCID: PMC7922691 DOI: 10.3390/brainsci11020256] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Revised: 02/12/2021] [Accepted: 02/13/2021] [Indexed: 01/02/2023] Open
Abstract
Scents have the ability to affect peoples' mental states and task performance with to different extents. It has been widely demonstrated that the lemon scent, included in most all-purpose cleaners, elicits stimulation and activation, while the lavender scent elicits relaxation and sedative effects. The present study aimed at investigating and fostering a novel approach to evaluate users' experience with respect to scents' effects through the joint employment of Virtual Reality and users' neurophysiological monitoring, in particular Electroencephalography. In particular, this study, involving 42 participants, aimed to compare the effects of lemon and lavender scents on the deployment of cognitive resources during a daily life experience consisting in a train journey carried out in virtual reality. Our findings showed a significant higher request of cognitive resources during the processing of an informative message for subjects exposed to the lavender scent with respect to the lemon exposure. No differences were found between lemon and lavender conditions on the self-reported items of pleasantness and involvement; as this study demonstrated, the employment of the lavender scent preserves the quality of the customer experience to the same extent as the more widely used lemon scent.
Collapse
Affiliation(s)
- Marco Mancini
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
- Department of Economics, Management and Business Law, University of Bari Aldo Moro (UniBa), Via Camillo Rosalba, 53, 70124 Bari, Italy
| | - Patrizia Cherubino
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
| | - Giulia Cartocci
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
| | - Ana Martinez
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
- Department of Communication and Social Research, Sapienza University of Rome, Via Salaria, 113, 00198 Rome, Italy
| | - Gianluca Borghini
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina 306, 00179 Rome, Italy
| | - Elena Guastamacchia
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
| | - Gianluca di Flumeri
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
- IRCCS Fondazione Santa Lucia, Neuroelectrical Imaging and BCI Lab, Via Ardeatina 306, 00179 Rome, Italy
| | - Dario Rossi
- Department of Anatomical, Histological, Forensic & Orthopedic Sciences, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185 Rome, Italy; (D.R.); (E.M.)
| | - Enrica Modica
- Department of Anatomical, Histological, Forensic & Orthopedic Sciences, Sapienza University of Rome, Piazzale Aldo Moro, 5, 00185 Rome, Italy; (D.R.); (E.M.)
| | - Stefano Menicocci
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
| | - Viviana Lupo
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
| | - Arianna Trettel
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
| | - Fabio Babiloni
- BrainSigns Srl, Via Lungotevere Michelangelo, 9, 00192 Rome, Italy; (P.C.); (G.C.); (A.M.); (G.B.); (E.G.); (G.d.F.); (S.M.); (V.L.); (A.T.); (F.B.)
- Department of Molecular Medicine, Sapienza University of Rome, Viale Regina Elena, 291, 00161 Rome, Italy
| |
Collapse
|
46
|
Kazi S, Khaleghzadegan S, Dinh JV, Shelhamer MJ, Sapirstein A, Goeddel LA, Chime NO, Salas E, Rosen MA. Team Physiological Dynamics: A Critical Review. HUMAN FACTORS 2021; 63:32-65. [PMID: 31557057 DOI: 10.1177/0018720819874160] [Citation(s) in RCA: 19] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/10/2023]
Abstract
OBJECTIVE Review the use of physiological measurement in team settings and propose recommendations to improve the state of the science. BACKGROUND New sensor and analytical capabilities enable exploration of relationships between team members' physiological dynamics. We conducted a review of physiological measures used in research on teams to understand (1) how these measures are theoretically and operationally related to team constructs and (2) what types of validity evidence exist for physiological measurement in team settings. METHOD We identified 32 articles that investigated task-performing teams using physiological data. Articles were coded on several dimensions, including team characteristics. Study findings were categorized by relationships tested between team physiological dynamics (TPD) and team inputs, mediators/processes, outputs, or psychometric properties. RESULTS TPD researchers overwhelmingly measure single physiological systems. Although there is research linking TPD to inputs and outputs, the research on processes is underdeveloped. CONCLUSION We recommend several theoretical, methodological, and statistical themes to expand the growth of the TPD field. APPLICATION Physiological measures, once established as reliable indicators of team functioning, might be used to diagnose suboptimal team states and cue interventions to ameliorate these states.
Collapse
Affiliation(s)
| | | | | | | | | | | | - Nnenna O Chime
- Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| | | | - Michael A Rosen
- Johns Hopkins University School of Medicine, Baltimore, Maryland, USA
| |
Collapse
|
47
|
Ayrolles A, Brun F, Chen P, Djalovski A, Beauxis Y, Delorme R, Bourgeron T, Dikker S, Dumas G. HyPyP: a Hyperscanning Python Pipeline for inter-brain connectivity analysis. Soc Cogn Affect Neurosci 2021; 16:72-83. [PMID: 33031496 PMCID: PMC7812632 DOI: 10.1093/scan/nsaa141] [Citation(s) in RCA: 34] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/03/2020] [Revised: 09/22/2020] [Accepted: 10/07/2020] [Indexed: 12/24/2022] Open
Abstract
The bulk of social neuroscience takes a 'stimulus-brain' approach, typically comparing brain responses to different types of social stimuli, but most of the time in the absence of direct social interaction. Over the last two decades, a growing number of researchers have adopted a 'brain-to-brain' approach, exploring similarities between brain patterns across participants as a novel way to gain insight into the social brain. This methodological shift has facilitated the introduction of naturalistic social stimuli into the study design (e.g. movies) and, crucially, has spurred the development of new tools to directly study social interaction, both in controlled experimental settings and in more ecologically valid environments. Specifically, 'hyperscanning' setups, which allow the simultaneous recording of brain activity from two or more individuals during social tasks, has gained popularity in recent years. However, currently, there is no agreed-upon approach to carry out such 'inter-brain connectivity analysis', resulting in a scattered landscape of analysis techniques. To accommodate a growing demand to standardize analysis approaches in this fast-growing research field, we have developed Hyperscanning Python Pipeline, a comprehensive and easy open-source software package that allows (social) neuroscientists to carry-out and to interpret inter-brain connectivity analyses.
Collapse
Affiliation(s)
- Anaël Ayrolles
- Department of Neuroscience, Institut Pasteur, Paris, France
- Child and Adolescent Psychiatry Department, Assistance Publique - Hôpitaux de Paris, Robert Debré Hospital, Paris, France
| | - Florence Brun
- Department of Neuroscience, Institut Pasteur, Paris, France
| | - Phoebe Chen
- Department of Psychology, New York University, New York City, USA
| | - Amir Djalovski
- Baruch Ivcher School of Psychology, Center for Developmental Social Neuroscience, Interdiscilinary Center Herzliya, Baruch Ivcher School of Psychology, Herzliya, Israel
- Department of Psychology, Bar-Ilan University, Ramat Gan, Israel
| | - Yann Beauxis
- Department of Neuroscience, Institut Pasteur, Paris, France
| | - Richard Delorme
- Department of Neuroscience, Institut Pasteur, Paris, France
- Child and Adolescent Psychiatry Department, Assistance Publique - Hôpitaux de Paris, Robert Debré Hospital, Paris, France
| | | | - Suzanne Dikker
- Department of Psychology, New York University, New York City, USA
- Department of Clinical Psychology, Free University Amsterdam, Amsterdam, The Netherlands
| | - Guillaume Dumas
- Department of Neuroscience, Institut Pasteur, Paris, France
- Center for Complex Systems and Brain Sciences, Florida Atlantic University, Center for Complex Systems and Brain Sciences, Boca Raton, FL, USA
- Departement of Psychiatry, Université de Montréal, Montreal, QC, Canada
- Precision Psychiatry and Social Physiology laboratory, CHU Sainte-Justine Centre de Recherche, Precision Psychiatry and Social Physiology Laboratory, Montreal, QC, Canada
| |
Collapse
|
48
|
Dolmans TC, Poel M, van ’t Klooster JWJR, Veldkamp BP. Perceived Mental Workload Classification Using Intermediate Fusion Multimodal Deep Learning. Front Hum Neurosci 2021; 14:609096. [PMID: 33505259 PMCID: PMC7829255 DOI: 10.3389/fnhum.2020.609096] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Accepted: 12/01/2020] [Indexed: 11/21/2022] Open
Abstract
A lot of research has been done on the detection of mental workload (MWL) using various bio-signals. Recently, deep learning has allowed for novel methods and results. A plethora of measurement modalities have proven to be valuable in this task, yet studies currently often only use a single modality to classify MWL. The goal of this research was to classify perceived mental workload (PMWL) using a deep neural network (DNN) that flexibly makes use of multiple modalities, in order to allow for feature sharing between modalities. To achieve this goal, an experiment was conducted in which MWL was simulated with the help of verbal logic puzzles. The puzzles came in five levels of difficulty and were presented in a random order. Participants had 1 h to solve as many puzzles as they could. Between puzzles, they gave a difficulty rating between 1 and 7, seven being the highest difficulty. Galvanic skin response, photoplethysmograms, functional near-infrared spectrograms and eye movements were collected simultaneously using LabStreamingLayer (LSL). Marker information from the puzzles was also streamed on LSL. We designed and evaluated a novel intermediate fusion multimodal DNN for the classification of PMWL using the aforementioned four modalities. Two main criteria that guided the design and implementation of our DNN are modularity and generalisability. We were able to classify PMWL within-level accurate (0.985 levels) on a seven-level workload scale using the aforementioned modalities. The model architecture allows for easy addition and removal of modalities without major structural implications because of the modular nature of the design. Furthermore, we showed that our neural network performed better when using multiple modalities, as opposed to a single modality. The dataset and code used in this paper are openly available.
Collapse
Affiliation(s)
- Tenzing C. Dolmans
- Data Management and Biometrics, University of Twente, Enschede, Netherlands
- Behavioural, Management and Social Sciences Lab, University of Twente, Enschede, Netherlands
| | - Mannes Poel
- Data Management and Biometrics, University of Twente, Enschede, Netherlands
| | | | - Bernard P. Veldkamp
- Research Methodology, Measurement, and Data Analysis, University of Twente, Enschede, Netherlands
| |
Collapse
|
49
|
Abstract
Advances in unmanned aircraft systems (UAS) have paved the way for progressively higher levels of intelligence and autonomy, supporting new modes of operation, such as the one-to-many (OTM) concept, where a single human operator is responsible for monitoring and coordinating the tasks of multiple unmanned aerial vehicles (UAVs). This paper presents the development and evaluation of cognitive human-machine interfaces and interactions (CHMI2) supporting adaptive automation in OTM applications. A CHMI2 system comprises a network of neurophysiological sensors and machine-learning based models for inferring user cognitive states, as well as the adaptation engine containing a set of transition logics for control/display functions and discrete autonomy levels. Models of the user’s cognitive states are trained on past performance and neurophysiological data during an offline calibration phase, and subsequently used in the online adaptation phase for real-time inference of these cognitive states. To investigate adaptive automation in OTM applications, a scenario involving bushfire detection was developed where a single human operator is responsible for tasking multiple UAV platforms to search for and localize bushfires over a wide area. We present the architecture and design of the UAS simulation environment that was developed, together with various human-machine interface (HMI) formats and functions, to evaluate the CHMI2 system’s feasibility through human-in-the-loop (HITL) experiments. The CHMI2 module was subsequently integrated into the simulation environment, providing the sensing, inference, and adaptation capabilities needed to realise adaptive automation. HITL experiments were performed to verify the CHMI2 module’s functionalities in the offline calibration and online adaptation phases. In particular, results from the online adaptation phase showed that the system was able to support real-time inference and human-machine interface and interaction (HMI2) adaptation. However, the accuracy of the inferred workload was variable across the different participants (with a root mean squared error (RMSE) ranging from 0.2 to 0.6), partly due to the reduced number of neurophysiological features available as real-time inputs and also due to limited training stages in the offline calibration phase. To improve the performance of the system, future work will investigate the use of alternative machine learning techniques, additional neurophysiological input features, and a more extensive training stage.
Collapse
|
50
|
Fan S, Blanco‐Davis E, Zhang J, Bury A, Warren J, Yang Z, Yan X, Wang J, Fairclough S. The Role of the Prefrontal Cortex and Functional Connectivity during Maritime Operations: An fNIRS study. Brain Behav 2021; 11:e01910. [PMID: 33151030 PMCID: PMC7821565 DOI: 10.1002/brb3.1910] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/23/2020] [Revised: 09/08/2020] [Accepted: 10/04/2020] [Indexed: 01/09/2023] Open
Abstract
INTRODUCTION Watchkeeping is a significant activity during maritime operations, and failures of sustained attention and decision-making can increase the likelihood of a collision. METHODS A study was conducted in a ship bridge simulator where 40 participants (20 experienced/20 inexperienced) performed: (1) a 20-min period of sustained attention to locate a target vessel and (2) a 10-min period of decision-making/action selection to perform an evasive maneuver. Half of the participants also performed an additional task of verbally reporting the position of their vessel. Activation of the prefrontal cortex (PFC) was captured via a 15-channel functional near-infrared spectroscopy (fNIRS) montage, and measures of functional connectivity were calculated frontal using graph-theoretic measures. RESULTS Neurovascular activation of right lateral area of the PFC decreased during sustained attention and increased during decision-making. The graph-theoretic analysis revealed that density declined during decision-making in comparison with the previous period of sustained attention, while local clustering declined during sustained attention and increased when participants prepared their evasive maneuver. A regression analysis revealed an association between network measures and behavioral outcomes, with respect to spotting the target vessel and making an evasive maneuver. CONCLUSIONS The right lateral area of the PFC is sensitive to watchkeeping and decision-making during operational performance. Graph-theoretic measures allow us to quantify patterns of functional connectivity and were predictive of safety-critical performance.
Collapse
Affiliation(s)
- Shiqi Fan
- Intelligent Transport Systems Research CentreWuhan University of TechnologyWuhanChina
- National Engineering Research Centre for Water Transport Safety (WTSC)MOSTWuhanChina
- Liverpool LogisticsOffshore and Marine (LOOM) Research InstituteLiverpool John Moores UniversityLiverpoolUK
| | - Eduardo Blanco‐Davis
- Liverpool LogisticsOffshore and Marine (LOOM) Research InstituteLiverpool John Moores UniversityLiverpoolUK
| | - Jinfen Zhang
- Intelligent Transport Systems Research CentreWuhan University of TechnologyWuhanChina
- National Engineering Research Centre for Water Transport Safety (WTSC)MOSTWuhanChina
| | - Alan Bury
- Liverpool LogisticsOffshore and Marine (LOOM) Research InstituteLiverpool John Moores UniversityLiverpoolUK
| | - Jonathan Warren
- Liverpool LogisticsOffshore and Marine (LOOM) Research InstituteLiverpool John Moores UniversityLiverpoolUK
| | - Zaili Yang
- Liverpool LogisticsOffshore and Marine (LOOM) Research InstituteLiverpool John Moores UniversityLiverpoolUK
| | - Xinping Yan
- Intelligent Transport Systems Research CentreWuhan University of TechnologyWuhanChina
- National Engineering Research Centre for Water Transport Safety (WTSC)MOSTWuhanChina
| | - Jin Wang
- Liverpool LogisticsOffshore and Marine (LOOM) Research InstituteLiverpool John Moores UniversityLiverpoolUK
| | | |
Collapse
|