Who needs a referee? How incorrect basketball actions are automatically detected by basketball players' brain

Brain Activity of Professional Dancers During Audiovisual Stimuli Exposure: A Systematic Review

Many studies have shown the effect of dance to the brain. It seems that long-term practice modulates brain plasticity and visuomotor skills, as it activates the Action Observation Network (AON). The aim of this systematic review was to evaluate potential differences in the brain activity (visuomotor skills) between professional dancers and non-dancer adults, measured by electroencephalography (EEG), during the observation of an individual who is dancing (video dance stimuli). This literature search was conducted from February to June 2022, according to the PRISMA guidelines, in the PubMed database using advanced search, mesh terms, and extensive manual search. The included articles were published in English. Specifically, case-control studies were selected, which used healthy adults, professional dancers, and non-dancers as participants, who were exposed to video dance clips and measured by EEG. The articles were excluded if they were based on different type of study, unhealthy population, control group with athletic background, different type of stimuli (rhythmic), or different type of task and procedure. The ratings of quality of evidence were conducted using the Joanna Briggs Institute's (JBI) critical appraisal tool. Five case-control studies were included with 193 participants in total, 87% females. The participating groups of professional dancers (n = 12-25) had mean age 25.14 years, with at least 9-19 years of professional training, whereas control groups had the same sample size, mean age of 24.14 years, and no experience in dancing. Most of the studies presented high methodological quality. All studies showed significant differences in dancers' brain activity, especially regarding the visuomotor skills. The results showed faster activation of AON demonstrated by higher P300 at the frontocentral regions and increased sensitivity of the occipital temporal cortex. Dancers could cope easier with familiar-unfamiliar and effortful-effortless movements. They also demonstrated faster alpha band peak frequency, stronger synchrony over the bands theta, beta, gamma during the audiovisual stimuli, and the ability to encode faster the visual information. The results demonstrate that dancers had better visuomotor skills suggesting dance-enhanced neuroplasticity, as professional dancers processed their actions easier. Dance, which includes visuomotor tasks, could help in prevention, therapy, and rehabilitation of neurodegenerative diseases or movement disorders.

Expertise Influences Congruency Monitoring During Action Observation at the Motor Level

Congruency monitoring of action occurs in individuals with relevant motor experience while observing actions. However, it remains unclear whether congruency monitoring can occur at the motor level and the extent to which expertise contributes. Here, we examined the behavioral performance and electrophysiological brain activity of individuals with and without domain-specific expertise when judging the action congruency of occluded video clips of a snowboard halfpipe trick and normal walking. For the halfpipe trick, experts exhibited better task performance and greater midline theta oscillations before possible incongruency compared with controls. Source reconstruction for the theta oscillation revealed a stronger activation in the middle and superior frontal gyrus for experts in response to incongruency compared with controls. Incongruent halfpipe actions elicited a higher N400 amplitude in experts compared with congruent actions, while no such differences were observed in controls. Source reconstruction revealed the activation in the board frontal regions and middle temporal gyrus for experts. These findings suggest that congruency monitoring can occur at the motor level during action observations and is modulated by individual expertise. The modulation of expertise reflects in the special N400 effect and midline theta oscillation.

Doing it Wrong: A Systematic Review on Electrocortical and Behavioral Correlates of Error Monitoring in Patients with Neurological Disorders

Detecting errors in one's own and other's actions is a crucial ability for learning and adapting behavior to everchanging, highly volatile environments. Studies in healthy people demonstrate that monitoring errors in one's own and others' actions are underpinned by specific neural systems that are dysfunctional in a variety of neurological disorders. In this review, we first briefly discuss the main findings concerning error detection and error awareness in healthy subjects, the current theoretical models, and the tasks usually applied to investigate these processes. Then, we report a systematic search for evidence of dysfunctional error monitoring among neurological populations (basal ganglia, neurodegenerative, white-matter diseases and acquired brain injury). In particular, we examine electrophysiological and behavioral evidence for specific alterations of error processing in neurological disorders. Error-related negativity (ERN) amplitude were reduced in most (although not all) neurological patient groups, whereas Positivity Error (Pe) amplitude appeared not to be affected in most patient groups. Also theta activity was reduced in some neurological groups, but consistent evidence on the oscillatory activity has not been provided thus far. Behaviorally, we did not observe relevant patterns of pronounced dysfunctional (post-) error processing. Finally, we discuss limitations of the existing literature, conclusive points, open questions and new possible methodological approaches for clinical studies.

Multimodal Recognition of Emotions in Music and Facial Expressions

The aim of the study was to investigate the neural processing of congruent vs. incongruent affective audiovisual information (facial expressions and music) by means of ERPs (Event Related Potentials) recordings. Stimuli were 200 infant faces displaying Happiness, Relaxation, Sadness, Distress and 32 piano musical pieces conveying the same emotional states (as specifically assessed). Music and faces were presented simultaneously, and paired so that in half cases they were emotionally congruent or incongruent. Twenty subjects were told to pay attention and respond to infrequent targets (adult neutral faces) while their EEG was recorded from 128 channels. The face-related N170 (160-180 ms) component was the earliest response affected by the emotional content of faces (particularly by distress), while visual P300 (250-450 ms) and auditory N400 (350-550 ms) responses were specifically modulated by the emotional content of both facial expressions and musical pieces. Face/music emotional incongruence elicited a wide N400 negativity indicating the detection of a mismatch in the expressed emotion. A swLORETA inverse solution applied to N400 (difference wave Incong. - Cong.), showed the crucial role of Inferior and Superior Temporal Gyri in the multimodal representation of emotional information extracted from faces and music. Furthermore, the prefrontal cortex (superior and medial, BA 10) was also strongly active, possibly supporting working memory. The data hints at a common system for representing emotional information derived by social cognition and music processing, including uncus and cuneus.

Electrophysiological Correlates of Racial In-group Bias in Observing Nonverbal Social Encounters

Despite evidence identifying the role of group membership in social cognition, the neural mechanisms associated with the perception and evaluation of nonverbal behaviors displayed by in-group versus out-group members remain unclear. Here, 42 white participants underwent electroencephalographic recording while observing social encounters involving dynamic displays of nonverbal behaviors by racial in-group and out-group avatar characters. Dynamic behaviors included approach and avoidance poses and expressions, followed by the participants' ratings of the avatars displaying them. Behaviorally, participants showed longer RTs when evaluating in-group approach behavior compared with other behaviors, possibly suggesting increased interest and attention devoted to processing positive social encounters with their in-group members. Analyses of ERPs revealed differential sensitivity of the N450 and late positivity components to social cues, with the former showing initial sensitivity to the presence of a humanoid avatar character at the beginning of social encounters and the latter showing sensitivity to dynamic nonverbal behaviors displayed by the avatars. Moreover, time-frequency analysis of electroencephalography data also identified suppression of beta-range power linked to the observation of dynamic nonverbal behaviors. Notably, the magnitude of these responses was modulated by the degree of behavioral racial in-group bias. This suggests that differential neural sensitivity to nonverbal cues while observing social encounters is associated with subsequent in-group bias manifested in the evaluation of such encounters. Collectively, these findings shed light on the mechanisms of racial in-group bias in social cognition and have implications for understanding factors related to successful interactions with individuals from diverse racial backgrounds.

How face blurring affects body language processing of static gestures in women and men

The role of facial coding in body language comprehension was investigated by event-related potential recordings in 31 participants viewing 800 photographs of gestures (iconic, deictic and emblematic), which could be congruent or incongruent with their caption. Facial information was obscured by blurring in half of the stimuli. The task consisted of evaluating picture/caption congruence. Quicker response times were observed in women than in men to congruent stimuli, and a cost for incongruent vs congruent stimuli was found only in men. Face obscuration did not affect accuracy in women as reflected by omission percentages, nor reduced their cognitive potentials, thus suggesting a better comprehension of face deprived pantomimes. N170 response (modulated by congruity and face presence) peaked later in men than in women. Late positivity was much larger for congruent stimuli in the female brain, regardless of face blurring. Face presence specifically activated the right superior temporal and fusiform gyri, cingulate cortex and insula, according to source reconstruction. These regions have been reported to be insufficiently activated in face-avoiding individuals with social deficits. Overall, the results corroborate the hypothesis that females might be more resistant to the lack of facial information or better at understanding body language in face-deprived social information.

Bilateral engagement of the occipito-temporal cortex in response to dance kinematics in experts

Previous evidence has shown neuroplastic changes in brain anatomy and connectivity associated with the acquisition of professional visuomotor skills. Reduced hemispherical asymmetry was found in the sensorimotor and visual areas in expert musicians and athletes compared with non-experts. Moreover, increased expertise with faces, body, and objects resulted in an enhanced engagement of the occipito-temporal cortex (OTC) during stimulus observation. The present study aimed at investigating whether intense and extended practice with dance would result in an enhanced symmetric response of OTC at an early stage of action processing. Expert ballet dancers and non-dancer controls were presented with videos depicting ballet steps during EEG recording. The observation of the moving dancer elicited a posterior N2 component, being larger over the left hemisphere in dancers than controls. The source reconstruction (swLORETA) of the negativity showed the engagement of the bilateral inferior and middle temporal regions in experts, while right-lateralized activity was found in controls. The dancers also showed an early P2 and enhanced P300 responses, indicating faster stimulus processing and subsequent recognition. This evidence seemed to suggest expertise-related increased sensitivity of the OTC in encoding body kinematics. Thus, we speculated that long-term whole-body practice would result in enriched and refined action processing.

Error, rather than its probability, elicits specific electrocortical signatures: a combined EEG-immersive virtual reality study of action observation

Detecting errors in one’s own actions, and in the actions of others, is a crucial ability for adaptable and flexible behavior. Studies show that specific EEG signatures underpin the monitoring of observed erroneous actions (error-related negativity, error positivity, mid-frontal theta oscillations). However, the majority of studies on action observation used sequences of trials where erroneous actions were less frequent than correct actions. Therefore, it was not possible to disentangle whether the activation of the performance monitoring system was due to an error, as a violation of the intended goal, or to a surprise/novelty effect, associated with a rare and unexpected event. Combining EEG and immersive virtual reality (IVR-CAVE system), we recorded the neural signal of 25 young adults who observed, in first-person perspective, simple reach-to-grasp actions performed by an avatar aiming for a glass. Importantly, the proportion of erroneous actions was higher than correct actions. Results showed that the observation of erroneous actions elicits the typical electrocortical signatures of error monitoring, and therefore the violation of the action goal is still perceived as a salient event. The observation of correct actions elicited stronger alpha suppression. This confirmed the role of the alpha-frequency band in the general orienting response to novel and infrequent stimuli. Our data provide novel evidence that an observed goal error (the action slip) triggers the activity of the performance-monitoring system even when erroneous actions, which are, typically, relevant events, occur more often than correct actions and thus are not salient because of their rarity.

NEW & NOTEWORTHY Activation of the performance-monitoring system (PMS) is typically investigated when errors in a sequence are comparatively rare. However, whether the PMS is activated by errors per se or by their infrequency is not known. Combining EEG-virtual reality techniques, we found that observing frequent (70%) action errors performed by avatars elicits electrocortical error signatures suggesting that deviation from the prediction of how learned actions should correctly deploy, rather than its frequency, is coded in the PMS.

"Embodied Body Language": an electrical neuroimaging study with emotional faces and bodies

To date, most investigations in the field of affective neuroscience mainly focused on the processing of facial expressions, overlooking the exploration of emotional body language (EBL), its capability to express our emotions notwithstanding. Few electrophysiological studies investigated the time course and the neural correlates of EBL and the integration of face and body emotion-related information. The aim of the present study was to investigate both the time course and the neural correlates underlying the integration of affective information conveyed by faces and bodies. We analysed EEG activities evoked during an expression matching task, requiring the judgment of emotional congruence between sequentially presented pairs of stimuli belonging to the same category (face-face or body-body), and between stimuli belonging to different categories (face-body or body-face). We focused on N400 time window and results showed that incongruent stimuli elicited a modulation of the N400 in all comparisons except for body-face condition. This modulation was mainly detected in the Middle Temporal Gyrus and within regions related to the mirror mechanism. More specifically, while the perception of incongruent facial expressions activates somatosensory-related representations, incongruent emotional body postures also require the activation of motor and premotor representations, suggesting a strict link between emotion and action.

tDCS application over the STG improves the ability to recognize and appreciate elements involved in humor processing

The superior temporal gyrus (STG) has been found to play a crucial role in the recognition of actions and facial expressions and may, therefore, be critical for the processing of humorous information. Here we investigated whether tDCS application to the STG would modulate the ability to recognize and appreciate the comic element in serious and comedic situations of misfortune. To this aim, the effects of different types of tDCS stimulation on the STG were analyzed during a task in which the participants were instructed to categorize various misfortunate situations as "comic" or "not comic". Participants underwent three different tDCS conditions: Anodal-right/Cathodal-left; Cathodal-right/Anodal-left; Sham. Images depicting people involved in accidents were grouped into three categories based on the facial expression of the victim: angry or painful (Affective); bewildered and funny (Comic); and images that did not contain the victim's face (No Face). An improvement in mean reaction times in response to both the Comic and No Face stimuli was observed following Anodal-left/Cathodal-right stimulation when compared to sham stimulation. This suggests that this stimulation type reduced the reaction times to socio-emotional complex scenes, regardless of facial expression. The Anodal-right/Cathodal-left stimulation reduced the mean reaction times for Comic stimuli only, suggesting that specifically the right STG may be involved in facial expression recognition and in the appreciation of the comic element in misfortunate situations. These results suggest a functional hemispheric asymmetry in STG response to social stimuli: the left STG might have a role in a general comprehension of social complex situations, while the right STG may be involved in the ability to recognize and integrate specific emotional aspects in a complex scene.

Error-related negativity in the skilled brain of pianists reveals motor simulation

Evidences have been provided of a crucial role of multimodal audio-visuomotor processing in subserving the musical ability. In this paper we investigated whether musical audiovisual stimulation might trigger the activation of motor information in the brain of professional pianists, due to the presence of permanent gestures/sound associations. At this aim EEG was recorded in 24 pianists and naive participants engaged in the detection of rare targets while watching hundreds of video clips showing a pair of hands in the act of playing, along with a compatible or incompatible piano soundtrack. Hands size and apparent distance allowed self-ownership and agency illusions, and therefore motor simulation. Event-related potentials (ERPs) and relative source reconstruction showed the presence of an Error-related negativity (ERN) to incongruent trials at anterior frontal scalp sites, only in pianists, with no difference in naïve participants. ERN was mostly explained by an anterior cingulate cortex (ACC) source. Other sources included "hands" IT regions, the superior temporal gyrus (STG) involved in conjoined auditory and visuomotor processing, SMA and cerebellum (representing and controlling motor subroutines), and regions involved in body parts representation (somatosensory cortex, uncus, cuneus and precuneus). The findings demonstrate that instrument-specific audiovisual stimulation is able to trigger error shooting and correction neural responses via motor resonance and mirroring, being a possible aid in learning and rehabilitation.

Embodying Others in Immersive Virtual Reality: Electro-Cortical Signatures of Monitoring the Errors in the Actions of an Avatar Seen from a First-Person Perspective

Brain monitoring of errors in one's own and other's actions is crucial for a variety of processes, ranging from the fine-tuning of motor skill learning to important social functions, such as reading out and anticipating the intentions of others. Here, we combined immersive virtual reality and EEG recording to explore whether embodying the errors of an avatar by seeing it from a first-person perspective may activate the error monitoring system in the brain of an onlooker. We asked healthy participants to observe, from a first- or third-person perspective, an avatar performing a correct or an incorrect reach-to-grasp movement toward one of two virtual mugs placed on a table. At the end of each trial, participants reported verbally how much they embodied the avatar's arm. Ratings were maximal in first-person perspective, indicating that immersive virtual reality can be a powerful tool to induce embodiment of an artificial agent, even through mere visual perception and in the absence of any cross-modal boosting. Observation of erroneous grasping from a first-person perspective enhanced error-related negativity and medial-frontal theta power in the trials where human onlookers embodied the virtual character, hinting at the tight link between early, automatic coding of error detection and sense of embodiment. Error positivity was similar in 1PP and 3PP, suggesting that conscious coding of errors is similar for self and other. Thus, embodiment plays an important role in activating specific components of the action monitoring system when others' errors are coded as if they are one's own errors.

SIGNIFICANCE STATEMENT

Detecting errors in other's actions is crucial for social functions, such as reading out and anticipating the intentions of others. Using immersive virtual reality and EEG recording, we explored how the brain of an onlooker reacted to the errors of an avatar seen from a first-person perspective. We found that mere observation of erroneous actions enhances electrocortical markers of error detection in the trials where human onlookers embodied the virtual character. Thus, the cerebral system for action monitoring is maximally activated when others' errors are coded as if they are one's own errors. The results have important implications for understanding how the brain can control the external world and thus creating new brain-computer interfaces.

Semantic brain areas are involved in gesture comprehension: An electrical neuroimaging study

While the mechanism of sign language comprehension in deaf people has been widely investigated, little is known about the neural underpinnings of spontaneous gesture comprehension in healthy speakers. Bioelectrical responses to 800 pictures of actors showing common Italian gestures (e.g., emblems, deictic or iconic gestures) were recorded in 14 persons. Stimuli were selected from a wider corpus of 1122 gestures. Half of the pictures were preceded by an incongruent description. ERPs were recorded from 128 sites while participants decided whether the stimulus was congruent. Congruent pictures elicited a posterior P300 followed by late positivity, while incongruent gestures elicited an anterior N400 response. N400 generators were investigated with swLORETA reconstruction. Processing of congruent gestures activated face- and body-related visual areas (e.g., BA19, BA37, BA22), the left angular gyrus, mirror fronto/parietal areas. The incongruent-congruent contrast particularly stimulated linguistic and semantic brain areas, such as the left medial and the superior temporal lobe.

The effect of musical practice on gesture/sound pairing

Learning to play a musical instrument is a demanding process requiring years of intense practice. Dramatic changes in brain connectivity, volume, and functionality have been shown in skilled musicians. It is thought that music learning involves the formation of novel audio visuomotor associations, but not much is known about the gradual acquisition of this ability. In the present study, we investigated whether formal music training enhances audiovisual multisensory processing. To this end, pupils at different stages of education were examined based on the hypothesis that the strength of audio/visuomotor associations would be augmented as a function of the number of years of conservatory study (expertise). The study participants were violin and clarinet students of pre-academic and academic levels and of different chronological ages, ages of acquisition, and academic levels. A violinist and a clarinetist each played the same score, and each participant viewed the video corresponding to his or her instrument. Pitch, intensity, rhythm, and sound duration were matched across instruments. In half of the trials, the soundtrack did not match (in pitch) the corresponding musical gestures. Data analysis indicated a correlation between the number of years of formal training (expertise) and the ability to detect an audiomotor incongruence in music performance (relative to the musical instrument practiced), thus suggesting a direct correlation between knowing how to play and perceptual sensitivity.

Audio-visuomotor processing in the musician's brain: an ERP study on professional violinists and clarinetists

The temporal dynamics of brain activation during visual and auditory perception of congruent vs. incongruent musical video clips was investigated in 12 musicians from the Milan Conservatory of music and 12 controls. 368 videos of a clarinetist and a violinist playing the same score with their instruments were presented. The sounds were similar in pitch, intensity, rhythm and duration. To produce an audiovisual discrepancy, in half of the trials, the visual information was incongruent with the soundtrack in pitch. ERPs were recorded from 128 sites. Only in musicians for their own instruments was a N400-like negative deflection elicited due to the incongruent audiovisual information. SwLORETA applied to the N400 response identified the areas mediating multimodal motor processing: the prefrontal cortex, the right superior and middle temporal gyrus, the premotor cortex, the inferior frontal and inferior parietal areas, the EBA, somatosensory cortex, cerebellum and SMA. The data indicate the existence of audiomotor mirror neurons responding to incongruent visual and auditory information, thus suggesting that they may encode multimodal representations of musical gestures and sounds. These systems may underlie the ability to learn how to play a musical instrument.

Why do we laugh at misfortunes? An electrophysiological exploration of comic situation processing

The goal of the present study was to shed some light on a particular kind of humour, called slapstick, by measuring brain bioelectrical activity during the perception of funny vs. non-funny pictures involving misfortunate circumstances. According to our hypothesis, the element mostly providing a comic feature in a misfortunate situation is the facial expression of the victims: the observer׳s reaction will usually be laughing only if the victims will show a funny bewilderment face and not a painful or anger expression. Several coloured photographs depicting people involved in misfortunate situations were presented to 30 Italian healthy volunteers, while their EEG was recorded. Three different situations were considered: people showing a painful or an angry expression (Affective); people showing a bewilderment expression and, so, a comic look (Comic); people engaged in similar misfortunate situations but with no face visible (No Face). Results showed that the mean amplitude of both the posterior N170 and anterior N220 components was much larger in amplitude to comic pictures, than the other stimuli. This early response could be considered the first identification of a comic element and evidence of the compelling and automatic response that usually characterizes people amused reaction during a misfortune. In addition, we observed a larger P300 amplitude in response to comic than affective pictures, probably reflecting a more conscious processing of the comic element. Finally, no face pictures elicited an anteriorly distributed N400, which might reflect the effort to comprehend the nature of the situation displayed without any affective facial information, and a late positivity, possibly indexing a re-analysis processing of the unintelligible misfortunate situation (comic or unhappy) depicted in the No Face stimuli. These data support the hypothesis that the facial expression of the victims acts as a specific trigger for the amused feeling that observers usually experience when someone falls down. Overall, the data indicate the existence of a neural circuit that is capable of recognize and appreciate the comic element of a misfortunate situation in a group of young adults.

Spatial generalization in operant learning: lessons from professional basketball

In operant learning, behaviors are reinforced or inhibited in response to the consequences of similar actions taken in the past. However, because in natural environments the “same” situation never recurs, it is essential for the learner to decide what “similar” is so that he can generalize from experience in one state of the world to future actions in different states of the world. The computational principles underlying this generalization are poorly understood, in particular because natural environments are typically too complex to study quantitatively. In this paper we study the principles underlying generalization in operant learning of professional basketball players. In particular, we utilize detailed information about the spatial organization of shot locations to study how players adapt their attacking strategy in real time according to recent events in the game. To quantify this learning, we study how a make \ miss from one location in the court affects the probabilities of shooting from different locations. We show that generalization is not a spatially-local process, nor is governed by the difficulty of the shot. Rather, to a first approximation, players use a simplified binary representation of the court into 2 pt and 3 pt zones. This result indicates that rather than using low-level features, generalization is determined by high-level cognitive processes that incorporate the abstract rules of the game.

According to the law of effect, formulated a century ago by Edward Thorndike, actions which are rewarded in a particular situation are more likely to be executed when that same situation recurs. However, in natural settings the same situation never recurs and therefore, generalization from one state of the world to other states is an essential part of the process of learning. In this paper we utilize basketball statistics to study the computational principles underlying generalization in operant learning of professional basketball players. We show that players are more likely to attempt a field goal from the vicinity of a previously made shot than they are from the vicinity of a missed shot, as expected from the law of effect. However, the outcome of a shot can also affect the likelihood of attempting another shot at a different location. Using hierarchical clustering we characterize the spatial pattern of generalization and show that generalization is primarily determined by the type of shot, 3 pt vs. 2 pt. This result indicates that rather than using low-level features, generalization is determined by high-level cognitive processes that incorporate the abstract rules of the game.

Comprehending body language and mimics: an ERP and neuroimaging study on Italian actors and viewers

In this study, the neural mechanism subserving the ability to understand people’s emotional and mental states by observing their body language (facial expression, body posture and mimics) was investigated in healthy volunteers. ERPs were recorded in 30 Italian University students while they evaluated 280 pictures of highly ecological displays of emotional body language that were acted out by 8 male and female Italian actors. Pictures were briefly flashed and preceded by short verbal descriptions (e.g., “What a bore!”) that were incongruent half of the time (e.g., a picture of a very attentive and concentrated person shown after the previous example verbal description). ERP data and source reconstruction indicated that the first recognition of incongruent body language occurred 300 ms post-stimulus. swLORETA performed on the N400 identified the strongest generators of this effect in the right rectal gyrus (BA11) of the ventromedial orbitofrontal cortex, the bilateral uncus (limbic system) and the cingulate cortex, the cortical areas devoted to face and body processing (STS, FFA EBA) and the premotor cortex (BA6), which is involved in action understanding. These results indicate that face and body mimics undergo a prioritized processing that is mostly represented in the affective brain and is rapidly compared with verbal information. This process is likely able to regulate social interactions by providing on-line information about the sincerity and trustfulness of others.