The influence on cognitive processing from the switches of shooting angles in videos of real-world events: An ERP study

Measurement of cognitive dynamics during video watching through event-related potentials (ERPs) and oscillations (EROs)

Event-related potentials (ERPs) and oscillations (EROs) are reliable measures of cognition, but they require time-locked electroencephalographic (EEG) data to repetitive triggers that are not available in continuous sensory input streams. However, such real-life-like stimulation by videos or virtual-reality environments may serve as powerful means of creating specific cognitive or affective states and help to investigate dysfunctions in psychiatric and neurological disorders more efficiently. This study aims to develop a method to generate ERPs and EROs during watching videos. Repeated luminance changes were introduced on short video segments, while EEGs of 10 subjects were recorded. The ERP/EROs time-locked to these distortions were analyzed in time and time-frequency domains and tested for their cognitive significance through a long term memory test that included frames from the watched videos. For each subject, ERPs and EROs corresponding to video segments of recalled images with 25% shortest and 25% longest reaction times were compared. ERPs produced by transient luminance changes displayed statistically significant fluctuations both in time and time-frequency domains. Statistical analyses showed that a positivity around 450 ms, a negativity around 500 ms and delta and theta EROs correlated with memory performance. Few studies mixed video streams with simultaneous ERP/ERO experiments with discrete task-relevant or passively presented auditory or somatosensory stimuli, while the present study, by obtaining ERPs and EROs to task-irrelevant events in the same sensory modality as that of the continuous sensory input, produces minimal interference with the main focus of attention on the video stream.

Semantic association of ecologically unrelated synchronous audio-visual information in cognitive integration: an event-related potential study

In this paper, we aimed to study the semantic association of ecologically unrelated synchronous audio-visual information in cognitive integration. A moving particle, which speed varied, was taken as a visual stimulus, while a simple tone, which frequency varied, was used as an auditory stimulus, both were synchronously presented to subjects in the form of a video. Behavioral results confirmed our hypothesis that the moving particle with varied speed and the simple tone with varied frequency were highly associated. Event-related potential (ERP) results showed that an N400 effect and a late posterior negativity (LPN) were elicited under the Incongruent condition as compared to the Congruent condition. It was further determined that there was semantic association between ecologically unrelated synchronous audio-visual information in cognitive integration. We considered that the N400 effect in our results reflected the process that stimulus-driven activities are bound together through a temporal semantic network (TSN) to form multimodal representations, while the state of this temporal semantic network was determined by both long-term learned association among stimuli and short-term experience of incoming information. The LPN might reflect the process that the human brain searches and retrieves context-specifying information in order to make a judgment, and the context-specifying information might have originated from the long-term learned association stored in the brain.

Cognitive processing of traffic signs in immersive virtual reality environment: An ERP study

The virtual reality environment can provide an immersive feeling as in the real word. So, using virtual reality technology to construct realistic experimental scenarios, the mechanism of cognitive processing in the human brain could be better studied. In this paper, we have designed an experiment, where through the presentation of traffic signs with correct or incorrect background colors in a virtual reality traffic environment, and studied the cognitive processing in the human brain using event-related potential (ERP) method. The results showed that whether the background colors of traffic signs were correct or not, the degrees of familiarity to these traffic signs in the human brain were similar, and the degree of contrast between the background colors and foreground colors of traffic signs would influence the degree of difficulty in cognitive processing. The degree of complexity in contents of traffic signs appears related to the cognitive speed in the human brain. In sum, simpler contents and larger contrast between the background colors and foreground colors of traffic signs would make the human brain respond faster.