Involvement of memory-comparison-based change detection in visual distraction

AtAwAR Translate: Attention-Aware Language Translation Application in Augmented Reality for Mobile Phones

As lightweight, low-cost EEG headsets emerge, the feasibility of consumer-oriented brain-computer interfaces (BCI) increases. The combination of portable smartphones and easy-to-use EEG dry electrode headbands offers intriguing new applications and methods of human-computer interaction. In previous research, augmented reality (AR) scenarios have been identified to profit from additional user state information-such as that provided by a BCI. In this work, we implemented a system that integrates user attentional state awareness into a smartphone application for an AR written language translator. The attentional state of the user is classified in terms of internally and externally directed attention by using the Muse 2 electroencephalography headband with four frontal electrodes. The classification results are used to adapt the behavior of the translation app, which uses the smartphone's camera to display translated text as augmented reality elements. We present the first mobile BCI system that uses a smartphone and a low-cost EEG device with few electrodes to provide attention awareness to an AR application. Our case study with 12 participants did not fully support the assumption that the BCI improves usability. However, we are able to show that the classification accuracy and ease of setup are promising paths toward mobile consumer-oriented BCI usage. For future studies, other use cases, applications, and adaptations will be tested for this setup to explore the usability.

Detection of deviance in Japanese kanji compound words

Reading fluency is based on the automatic visual recognition of words. As a manifestation of the automatic processing of words, an automatic deviance detection of visual word stimuli can be observed in the early stages of visual recognition. To clarify whether this phenomenon occurs with Japanese kanji compounds—since their lexicality is related to semantic association—we investigated the brain response by utilizing three types of deviants: differences in font type, lexically correct or incorrect Japanese kanji compound words and pseudo-kanji characters modified from correct and incorrect compounds. We employed magnetoencephalography (MEG) to evaluate the spatiotemporal profiles of the related brain regions. The study included 22 adult native Japanese speakers (16 females). The abovementioned three kinds of stimuli containing 20% deviants were presented during the MEG measurement. Activity in the occipital pole region of the brain was observed upon the detection of font-type deviance within 250 ms of stimulus onset. Although no significant activity upon detecting lexically correct/incorrect kanji compounds or pseudo-kanji character deviations was observed, the activity in the posterior transverse region of the collateral sulcus (pCoS)—which is a fusiform neighboring area—was larger when detecting lexically correct kanji compounds than when detecting pseudo-kanji characters. Taken together, these results support the notion that the automatic detection of deviance in kanji compounds may be limited to a low-level feature, such as the stimulus stroke thickness.

Subtle Distinctions: How Attentional Templates Influence EEG Parameters of Cognitive Control in a Spatial Cuing Paradigm

Using event-related potentials (ERPs) of the electroencephalogram, we investigated how cognitive control is altered by the scope of an attentional template currently activated in visual working memory. Participants performed a spatial cuing task where an irrelevant color singleton cue was presented prior to a target array. Blockwise, the target was either a red circle or a gray square and had to be searched within homogenous (gray circles) or heterogeneous non-targets (differently colored circles or various shapes). Thereby we aimed to trigger the adoption of different attentional templates: a broader singleton or a narrower, more specific feature template. ERP markers of attentional selection and inhibitory control showed that the amount of cognitive control was overall enhanced when participants searched on the basis of a feature-specific template: the analysis revealed reduced selection (N2pc, frontal P2) and pronounced inhibition (negative shift of frontal N2) of the irrelevant color cue when participants searched for a feature target. On behavioral level attentional capture was most pronounced in the color condition with no differentiation between the task-induced scopes of the attentional template.

Preattentive Processing of Numerical Visual Information

Humans can perceive and estimate approximate numerical information, even when accurate counting is impossible e.g., due to short presentation time. If the number of objects to be estimated is small, typically around 1–4 items, observers are able to give very fast and precise judgments with high confidence—an effect that is called subitizing. Due to its speed and effortless nature subitizing has usually been assumed to be preattentive, putting it into the same category as other low level visual features like color or orientation. More recently, however, a number of studies have suggested that subitizing might be dependent on attentional resources. In our current study we investigated the potentially preattentive nature of visual numerical perception in the subitizing range by means of EEG. We presented peripheral, task irrelevant sequences of stimuli consisting of a certain number of circular patches while participants were engaged in a demanding, non-numerical detection task at the fixation point drawing attention away from the number stimuli. Within a sequence of stimuli of a given number of patches (called “standards”) we interspersed some stimuli of different numerosity (“oddballs”). We compared the evoked responses to visually identical stimuli that had been presented in two different conditions, serving as standard in one condition and as oddball in the other. We found significant visual mismatch negativity (vMMN) responses over parieto-occipital electrodes. In addition to the event-related potential (ERP) analysis, we performed a time-frequency analysis (TFA) to investigate whether the vMMN was accompanied by additional oscillatory processes. We found a concurrent increase in evoked theta power of similar strength over both hemispheres. Our results provide clear evidence for a preattentive processing of numerical visual information in the subitizing range.

Automatic prediction regarding the next state of a visual object: Electrophysiological indicators of prediction match and mismatch

Behavioral phenomena such as representational momentum suggest that the brain can automatically predict the next state of a visual object, based on sequential rules embedded in its preceding spatiotemporal context. To identify electrophysiological indicators of automatic visual prediction in terms of prediction match and mismatch, we recorded event-related brain potentials (ERPs) while participants passively viewed three types of task-irrelevant sequences of a bar stimulus: (1) an oddball sequence, which contained a sequential rule defined by stimulus repetition, providing repetition-rule-conforming (standard) and -violating (deviant) stimuli; (2) a rotating-oddball sequence, which contained a sequential rule defined by stimulus change (i.e., rotation), providing change-rule-conforming (regular) and -violating (irregular) stimuli; and (3) a random sequence, which did not contain a sequential rule, providing a neutral (control) stimulus. This protocol allowed us to expect that (1) an ERP effect that reflects a prediction-mismatch process should be exclusively observed in both the deviant-minus-control and irregular-minus-control comparisons and (2) an ERP effect that reflects a prediction-match process should be exclusively observed in both the standard-minus-control and regular-minus-control comparisons. The results showed that the ERP effect that met the criterion for prediction mismatch was an occipito-temporal negative deflection at around 170-300ms (visual mismatch negativity), while the ERP effect that met the criterion for prediction match was a frontal/central negative deflection at around 150-270ms (probably, the reduction of P2). These two contrasting ERP effects support a hypothetical view that automatic visual prediction would involve both an increase in the neural response to prediction-incongruent (i.e., novel) events and a decrease in the neural response to prediction-congruent (i.e., redundant) events. This article is part of a Special Issue entitled SI: Prediction and Attention.

Visual mismatch negativity: a predictive coding view

An increasing number of studies investigate the visual mismatch negativity (vMMN) or use the vMMN as a tool to probe various aspects of human cognition. This paper reviews the theoretical underpinnings of vMMN in the light of methodological considerations and provides recommendations for measuring and interpreting the vMMN. The following key issues are discussed from the experimentalist's point of view in a predictive coding framework: (1) experimental protocols and procedures to control "refractoriness" effects; (2) methods to control attention; (3) vMMN and veridical perception.

Object-related regularities are processed automatically: evidence from the visual mismatch negativity

One of the most challenging tasks of our visual systems is to structure and integrate the enormous amount of incoming information into distinct coherent objects. It is an ongoing debate whether or not the formation of visual objects requires attention. Implicit behavioral measures suggest that object formation can occur for task-irrelevant and unattended visual stimuli. The present study investigated pre-attentive visual object formation by combining implicit behavioral measures and an electrophysiological indicator of pre-attentive visual irregularity detection, the visual mismatch negativity (vMMN) of the event-related potential. Our displays consisted of two symmetrically arranged, task-irrelevant ellipses, the objects. In addition, there were two discs of either high or low luminance presented on the objects, which served as targets. Participants had to indicate whether the targets were of the same or different luminance. In separate conditions, the targets either usually were enclosed in the same object or in two different objects (standards). Occasionally, the regular target-to-object assignment was changed (deviants). That is, standards and deviants were exclusively defined on the basis of the task-irrelevant target-to-object assignment but not on the basis of some feature regularity. Although participants did not notice the regularity nor the occurrence of the deviation in the sequences, task-irrelevant deviations resulted in increased reaction times. Moreover, compared with physically identical standard displays deviating target-to-object assignments elicited a negative potential in the 246–280 ms time window over posterio-temporal electrode positions which was identified as vMMN. With variable resolution electromagnetic tomography (VARETA) object-related vMMN was localized to the inferior temporal gyrus. Our results support the notion that the visual system automatically structures even task-irrelevant aspects of the incoming information into objects.

P3a from white noise

P3a and P3b event-related brain potentials (ERPs) were elicited with an auditory three-stimulus (target, distracter, and standard) discrimination task in which subjects responded only to the target. Distracter stimuli consisted of white noise or novel sounds with stimulus characteristics perceptually matched. Target/standard discrimination difficulty was manipulated by varying target/standard pitch differences to produce relatively easy, medium, and hard tasks. Error rate and response time increased with increases in task difficulty. P3a was larger for the white noise compared to novel sounds, maximum over the central/parietal recording sites, and did not differ in size across difficulty levels. P3b was unaffected by distracter type, decreased as task difficulty increased, and maximum over the parietal recording sites. The findings indicate that P3a from white noise is robust and should be useful for applied studies as it removes stimulus novelty variability. Theoretical perspectives are discussed.

Time-course of hemispheric preference for processing contralateral relevant shapes: P1pc, N1pc, N2pc, N3pc

A most sensitive and specific electrophysiological indicator of selective processing of visual stimuli is the N2pc component. N2pc is a negative EEG potential peaking 250 ms after stimulus onset, recorded from posterior sites contralateral to relevant stimuli. Additional deflections preceding or following N2pc have been obtained in previous studies, possibly produced by specific stimulus features or specific prime-target sequences. To clarify the entire time-course of the contralateral- ipsilateral (C-I) difference recorded from the scalp above visual cortex in response to left-right pairs of targets and distracters, C-I differences were here compared between two types of stimuli and between stimuli that were or were not preceded by masked neutral primes. The C-I difference waveform consisted of several peaks, termed here P1pc (60-100 ms after target onset), N1pc (120-160 ms), N2pc (220-280 ms), and N3pc (360-400 ms). Being markedly enhanced when stimuli were preceded by the neutral primes, P1pc may indicate a response to stimulus change. Also, when stimuli were primed, N2pc reached its peak earlier, thereby tending to merge with N1pc. N3pc seemed to increase when target discrimination was difficult. N1pc, N2pc, and N3pc appear as three periods of one process. N3pc probably corresponds to L400 or SPCN as described in other studies. These observations suggest that the neurophysiological basis of stimulus-driven focusing of attention on target stimuli is a process that lasts for hundreds of milliseconds, with the relevant hemisphere being activated in an oscillating manner as long as required by the task.

Human Visual System Automatically Encodes Sequential Regularities of Discrete Events

For our adaptive behavior in a dynamically changing environment, an essential task of the brain is to automatically encode sequential regularities inherent in the environment into a memory representation. Recent studies in neuroscience have suggested that sequential regularities embedded in discrete sensory events are automatically encoded into a memory representation at the level of the sensory system. This notion is largely supported by evidence from investigations using auditory mismatch negativity (auditory MMN), an event-related brain potential (ERP) correlate of an automatic memory-mismatch process in the auditory sensory system. However, it is still largely unclear whether or not this notion can be generalized to other sensory modalities. The purpose of the present study was to investigate the contribution of the visual sensory system to the automatic encoding of sequential regularities using visual mismatch negativity (visual MMN), an ERP correlate of an automatic memory-mismatch process in the visual sensory system. To this end, we conducted a sequential analysis of visual MMN in an oddball sequence consisting of infrequent deviant and frequent standard stimuli, and tested whether the underlying memory representation of visual MMN generation contains only a sensory memory trace of standard stimuli (trace-mismatch hypothesis) or whether it also contains sequential regularities extracted from the repetitive standard sequence (regularity-violation hypothesis). The results showed that visual MMN was elicited by first deviant (deviant stimuli following at least one standard stimulus), second deviant (deviant stimuli immediately following first deviant), and first standard (standard stimuli immediately following first deviant), but not by second standard (standard stimuli immediately following first standard). These results are consistent with the regularity-violation hypothesis, suggesting that the visual sensory system automatically encodes sequential regularities. In combination with a wide range of auditory MMN studies, the present study highlights the critical role of sensory systems in automatically encoding sequential regularities when modeling the world.