Neuromagnetic brain responses to other person's eye blinks seen on video

Construction of a fiber-optically connected MEG hyperscanning system for recording brain activity during real-time communication

Communication is one of the most important abilities in human society, which makes clarification of brain functions that underlie communication of great importance to cognitive neuroscience. To investigate the rapidly changing cortical-level brain activity underlying communication, a hyperscanning system with both high temporal and spatial resolution is extremely desirable. The modality of magnetoencephalography (MEG) would be ideal, but MEG hyperscanning systems suitable for communication studies remain rare. Here, we report the establishment of an MEG hyperscanning system that is optimized for natural, real-time, face-to-face communication between two adults in sitting positions. Two MEG systems, which are installed 500m away from each other, were directly connected with fiber optic cables. The number of intermediate devices was minimized, enabling transmission of trigger and auditory signals with almost no delay (1.95-3.90 μs and 3 ms, respectively). Additionally, video signals were transmitted at the lowest latency ever reported (60-100 ms). We furthermore verified the function of an auditory delay line to synchronize the audio with the video signals. This system is thus optimized for natural face-to-face communication, and additionally, music-based communication which requires higher temporal accuracy is also possible via audio-only transmission. Owing to the high temporal and spatial resolution of MEG, our system offers a unique advantage over existing hyperscanning modalities of EEG, fNIRS, or fMRI. It provides novel neuroscientific methodology to investigate communication and other forms of social interaction, and could potentially aid in the development of novel medications or interventions for communication disorders.

Semi-automated brain responses in communication: A magnetoencephalographic hyperscanning study

Face to face communication is interactive, and involves continuous feedforward and feedback of information, thoughts, and feelings to the opposite party. To accurately assess the neural processing underlying these interactions, synchronous and simultaneous recording of the brain activity from both parties is needed, a method known as hyperscanning. Here, we investigated the neural processing underlying nonverbal face-to-face communication using a magnetoencephalographic (MEG) hyperscanning system, comprising two fiber optically connected MEGs. Eight pairs of subjects participated. Each individual in each pair viewed a combined 80 randomized 20 s trials of 40 real-time and 40 recorded (hereafter, real and simulated, respectively) videos of the opposite party's face. Non-verbal communication through actions such as gaze, eye blinks, and facial expression was intrinsically only possible during real videos. After each trial, subjects individually subjectively discriminated whether the viewed video was real or simulated. Overall subjective discrimination accuracies were slightly but significantly above chance level. Statistical analysis of brain activity revealed a significant three way interaction between theta-band rhythm amplitude, video type, and subjective discrimination response in the right frontal cortex. Additionally, when subjects responded that videos were simulated, theta activity was significantly lower for real videos compared with simulated videos (p = 0.01). This result not only demonstrates the importance of right frontal theta activity during non-verbal communication, but also indicates the existence of unconscious, semi-automated neural processing during non-verbal communication that underlies one's ability to subjectively discriminate whether or not the opposite party is real.

The Hawthorne Effect in Eye-blinking: Awareness that One's Blinks are Being Counted Alters Blink Behavior

Purpose: Spontaneous eye-blink rate is the number of involuntary blinks performed during a certain period of time. Assessing blink rate in humans provides valuable physiological and behavioral data for studying neuropathology and emotional states. Cognitive states, including awareness on the part of the subject that his or her blinks are being counted, may influence blink activity and confound blink rate measurements. The goal of this study was to provide direct experimental evaluation of the hypothesis that subject awareness affects tasked-based blink activity. Materials and methods: 30 young healthy adult males with normal vision underwent a series of tasks - viewing images, talking, sitting quietly, and cross fixation - while being video recorded. Each subject completed the tasks naively, then repeated them after being explicitly told their blink rate was being measured. Blink rate was measured through minute-by-minute blink counts by human raters. Results: We found a transitory impact on blink count during the first and third minute of a passive image-viewing task that occurred immediately after subjects were informed of their eye blinks being counted. However, the overall blink rate across the 7-min passive image-viewing task was not influenced. In three other tasks - fixation, silence, and conversation - we observed no statistically significant difference in minute-by-minute blink count or overall blink rate. Conclusions: We conclude that informing a subject that his eye blinks are being counted exerts a modest but significant acute influence on blinking activity, but critically does not appear to confound blink rate over prolonged tasks.

Eye blinks are perceived as communicative signals in human face-to-face interaction

In face-to-face communication, recurring intervals of mutual gaze allow listeners to provide speakers with visual feedback (e.g. nodding). Here, we investigate the potential feedback function of one of the subtlest of human movements—eye blinking. While blinking tends to be subliminal, the significance of mutual gaze in human interaction raises the question whether the interruption of mutual gaze through blinking may also be communicative. To answer this question, we developed a novel, virtual reality-based experimental paradigm, which enabled us to selectively manipulate blinking in a virtual listener, creating small differences in blink duration resulting in ‘short’ (208 ms) and ‘long’ (607 ms) blinks. We found that speakers unconsciously took into account the subtle differences in listeners’ blink duration, producing substantially shorter answers in response to long listener blinks. Our findings suggest that, in addition to physiological, perceptual and cognitive functions, listener blinks are also perceived as communicative signals, directly influencing speakers’ communicative behavior in face-to-face communication. More generally, these findings may be interpreted as shedding new light on the evolutionary origins of mental-state signaling, which is a crucial ingredient for achieving mutual understanding in everyday social interaction.

Recognition method of construction conflict based on driver's eye movement

Drivers eye movement data in simulated construction conflicts at different speeds were collected and analyzed to find the relationship between the drivers' eye movement and the construction conflict. On the basis of the relationship between the drivers' eye movement and the construction conflict, the peak point of wavelet processed pupil diameter, the first point on the left side of the peak point and the first blink point after the peak point are selected as key points for locating construction conflict periods. On the basis of the key points and the GSA, a construction conflict recognition method so called the CCFRM is proposed. And the construction conflict recognition speed and location accuracy of the CCFRM are verified. The good performance of the CCFRM verified the feasibility of proposed key points in construction conflict recognition.

Blink: Characteristics, Controls, and Relation to Dry Eyes

Blink is a complex phenomenon that is profoundly affected by diverse endogenous and exogenous stimuli. It has been studied in the context of cognition, emotional, and psychological states, as an indicator of fatigue and sleepiness, particularly in the automobile and transportation industry, in visual tasking, and finally, as it relates to tear film stability and ocular surface health. The fact that it is highly variable and has input from so many sources makes it very difficult to study. In the present review, the behavior of blink in many of these systems is discussed, ultimately returning in each instance to a discussion of how these factors affect blink in the context of dry eyes. Blink is important to ocular surface health and to an individual's optimal functioning and quality of life. Disturbances in blink, as cause or effect, result in a breakdown of tear film stability, optical clarity, and visual function.

Attending to and neglecting people: bridging neuroscience, psychology and sociology

Human behaviour is context-dependent-based on predictions and influenced by the environment and other people. We live in a dynamic world where both the social stimuli and their context are constantly changing. Similar dynamic, natural stimuli should, in the future, be increasingly used to study social brain functions, with parallel development of appropriate signal-analysis methods. Understanding dynamic neural processes also requires accurate time-sensitive characterization of the behaviour. To go beyond the traditional stimulus-response approaches, brain activity should be recorded simultaneously from two interacting subjects to reveal why human social interaction is critically different from just reacting to each other. This theme issue on Attending to and neglecting people contains original work and review papers on person perception and social interaction. The articles cover research from neuroscience, psychology, robotics, animal interaction research and microsociology. Some of the papers are co-authored by scientists who presented their own, independent views in the recent Attention and Performance XXVI conference but were brave enough to join forces with a colleague having a different background and views. In the future, information needs to converge across disciplines to provide us a more holistic view of human behaviour, its interactive nature, as well as the temporal dynamics of our social world.

Brain responds to another person's eye blinks in a natural setting-the more empathetic the viewer the stronger the responses

An observer's brain is known to respond to another person's small nonverbal signals, such as gaze shifts and eye blinks. Here we aimed to find out how an observer's brain reacts to a speaker's eye blinks in the presence of other audiovisual information. Magnetoencephalographic brain responses along with eye gaze were recorded from 13 adults who watched a video of a person telling a story. The video was presented first without sound (visual), then with sound (audiovisual), and finally the audio story was presented with a still-frame picture on the screen (audio control). The viewers mainly gazed at the eye region of the speaker. Their saccades were suppressed at about 180 ms after the start of the speaker's blinks, a subsequent increase of saccade occurence to the base level, or higher, at around 340 ms. The suppression occurred in visual and audiovisual conditions but not during the control audio presentation. Prominent brain responses to blinks peaked in the viewer's occipital cortex at about 250 ms, with no differences in mean peak amplitudes or latencies between visual and audiovisual conditions. During the audiovisual, but not visual-only, presentation, the responses were the stronger the more empathetic the subject was according to the Empathic Concern score of the Interpersonal Reactivity Index questionnaire (Spearman's rank correlation, 0.73). The other person's eye blinks, nonverbal signs that often go unnoticed, thus elicited clear brain responses even in the presence of attention-attracting audiovisual information from the narrative, with stronger responses in people with higher empathy scores.

Magnetoencephalographic study on facial movements

In this review, we introduced our three studies that focused on facial movements. In the first study, we examined the temporal characteristics of neural responses elicited by viewing mouth movements, and assessed differences between the responses to mouth opening and closing movements and an averting eyes condition. Our results showed that the occipitotemporal area, the human MT/V5 homologue, was active in the perception of both mouth and eye motions. Viewing mouth and eye movements did not elicit significantly different activity in the occipitotemporal area, which indicated that perception of the movement of facial parts may be processed in the same manner, and this is different from motion in general. In the second study, we investigated whether early activity in the occipitotemporal region evoked by eye movements was influenced by the facial contour and/or features such as the mouth. Our results revealed specific information processing for eye movements in the occipitotemporal region, and this activity was significantly influenced by whether movements appeared with the facial contour and/or features, in other words, whether the eyes moved, even if the movement itself was the same. In the third study, we examined the effects of inverting the facial contour (hair and chin) and features (eyes, nose, and mouth) on processing for static and dynamic face perception. Our results showed the following: (1) In static face perception, activity in the right fusiform area was affected more by the inversion of features while that in the left fusiform area was affected more by a disruption in the spatial relationship between the contour and features; and (2) In dynamic face perception, activity in the right occipitotemporal area was affected by the inversion of the facial contour.