Comparison of the N300 and N400 ERPs to picture stimuli in congruent and incongruent contexts

Using Event-Related Potentials to Evidence the Visual and Semantic Impact: A Pilot Study with N400 Effect and Food Packaging

Packaging design is pivotal in motivating consumer decisions, as a key communication tool from creation to purchase. Currently, the interpretation and evaluation of packaging's impact are shifting toward non-traditional methods. This pilot study evaluated the packaging perception of York Ham and Turkey Breast products. The event-related potential (ERP) technique, the methodology priming words (positive and negative), and target images (original and modified packaging) were applied. A total of 23 participants were sampled using a 32-channels scalp elastic electrode cap and viewed 200 trials of word-image matching. Participants responded whether the images and adjectives matched or not, using the two groups of images. The results demonstrate an N400 effect in the parietal area. This region was observed to show evidence of cognitive processing related to congruency or incongruency, by contrasting the priming and target of this study. The evaluation positioned the York Ham packaging as the best rated. The findings show a relevant contribution to ERPs and research related to the food packaging perception.

When experience with scenes foils attentional orienting: ERP evidence against flexible target-context mapping in visual search

Visual search is speeded when a target is repeatedly presented in an invariant scene context of nontargets (contextual cueing), demonstrating observers' capability for using statistical long-term memory (LTM) to make predictions about upcoming sensory events, thus improving attentional orienting. In the current study, we investigated whether expectations arising from individual, learned environmental structures can encompass multiple target locations. We recorded event-related potentials (ERPs) while participants performed a contextual cueing search task with repeated and non-repeated spatial item configurations. Notably, a given search display could be associated with either a single target location (standard contextual cueing) or two possible target locations. Our result showed that LTM-guided attention was always limited to only one target position in single- but also in the dual-target displays, as evidenced by expedited reaction times (RTs) and enhanced N1pc and N2pc deflections contralateral to one ("dominant") target of up to two repeating target locations. This contrasts with the processing of non-learned ("minor") target positions (in dual-target displays), which revealed slowed RTs alongside an initial N1pc "misguidance" signal that then vanished in the subsequent N2pc. This RT slowing was accompanied by enhanced N200 and N400 waveforms over fronto-central electrodes, suggesting that control mechanisms regulate the competition between dominant and minor targets. Our study thus reveals a dissociation in processing dominant versus minor targets: While LTM templates guide attention to dominant targets, minor targets necessitate control processes to overcome the automatic bias towards previously learned, dominant target locations.

Between bodily action and conventionalized structure: The neural mechanisms of constructed action in sign language comprehension

Sign languages (SLs) are expressed through different bodily actions, ranging from re-enactment of physical events (constructed action, CA) to sequences of lexical signs with internal structure (plain telling, PT). Despite the prevalence of CA in signed interactions and its significance for SL comprehension, its neural dynamics remain unexplored. We examined the processing of different types of CA (subtle, reduced, and overt) and PT in 35 adult deaf or hearing native signers. The electroencephalographic-based processing of signed sentences with incongruent targets was recorded. Attenuated N300 and early N400 were observed for CA in deaf but not in hearing signers. No differences were found between sentences with CA types in all signers, suggesting a continuum from PT to overt CA. Deaf signers focused more on body movements; hearing signers on faces. We conclude that CA is processed less effortlessly than PT, arguably because of its strong focus on bodily actions.

Anaphoric distance dependencies in visual narrative structure and processing

Linguistic syntax has often been claimed as uniquely complex due to features like anaphoric relations and distance dependencies. However, visual narratives of sequential images, like those in comics, have been argued to use sequencing mechanisms analogous to those in language. These narrative structures include "refiner" panels that "zoom in" on the contents of another panel. Similar to anaphora in language, refiners indexically connect inexplicit referential information in one unit (refiner, pronoun) to a more informative "antecedent" elsewhere in the discourse. Also like in language, refiners can follow their antecedents (anaphoric) or precede them (cataphoric), along with having either proximal or distant connections. We here explore the constraints on visual narrative refiners created by modulating these features of order and distance. Experiment 1 examined participants' preferences for where refiners are placed in a sequence using a force-choice test, which revealed that refiners are preferred to follow their antecedents and have proximal distances from them. Experiment 2 then showed that distance dependencies lead to slower self-paced viewing times. Finally, measurements of event-related brain potentials (ERPs) in Experiment 3 revealed that these patterns evoke similar brain responses as referential dependencies in language (i.e., N400, LAN, Nref). Across all three studies, the constraints and (neuro)cognitive responses to refiners parallel those shown to anaphora in language, suggesting domain-general constraints on the sequencing of referential dependencies.

The effect of context congruency on fMRI repetition suppression for objects

The recognition of objects is strongly facilitated when they are presented in the context of other objects (Biederman, 1972). Such contexts facilitate perception and induce expectations of context-congruent objects (Trapp and Bar, 2015). The neural mechanisms underlying these facilitatory effects of context on object processing, however, are not yet fully understood. In the present study, we investigate how context-induced expectations affect subsequent object processing. We used functional magnetic resonance imaging and measured repetition suppression as a proxy for prediction error processing. Participants viewed pairs of alternating or repeated object images which were preceded by context-congruent, context-incongruent or neutral cues. We found a stronger repetition suppression in congruent as compared to incongruent or neutral cues in the object sensitive lateral occipital cortex. Interestingly, this stronger effect was driven by enhanced responses to alternating stimulus pairs in the congruent contexts, rather than by suppressed responses to repeated stimulus pairs, which emphasizes the contribution of surprise-related response enhancement for the context modulation on RS when expectations are violated. In addition, in the congruent condition, we discovered significant functional connectivity between object-responsive and frontal cortical regions, as well as between object-responsive regions and the fusiform gyrus. Our findings indicate that prediction errors, reflected in enhanced brain responses to violated contextual expectations, underlie the facilitating effect of context during object perception.

Semantic processing in children with Cochlear Implants: A review of current N400 studies and recommendations for future research

Deaf and hard of hearing children with cochlear implants (CI) often display impaired spoken language skills. While a large number of studies investigated brain responses to sounds in this population, relatively few focused on semantic processing. Here we summarize and discuss findings in four studies of the N400, a cortical response that reflects semantic processing, in children with CI. A study with auditory target stimuli found N400 effects at delayed latencies at 12 months after implantation, but at 18 and 24 months after implantation effects had typical latencies. In studies with visual target stimuli N400 effects were larger than or similar to controls in children with CI, despite lower semantic abilities. We propose that in children with CI, the observed large N400 effect reflects a stronger reliance on top-down predictions, relative to bottom-up language processing. Recent behavioral studies of children and adults with CI suggest that top-down processing is a common compensatory strategy, but with distinct limitations such as being effortful. A majority of the studies have small sample sizes (N < 20), and only responses to image targets were studied repeatedly in similar paradigms. This precludes strong conclusions. We give suggestions for future research and ways to overcome the scarcity of participants, including extending research to children with conventional hearing aids, an understudied group.

Unveiling the need of interactions for social N400s and supporting the N400 inhibition hypothesis

When participants (Pps) are presented with stimuli in the presence of another person, they may consider that person's perspective. Indeed, five recent ERP studies show that the amplitudes of their N400s are increased. The two most recent ones reveal that these social-N400 increases occur even when instructions do not require a focus on the other's perspective. These increases also happen when Pps know that this other person has the same stimulus information as they have. However, in all these works, Pps could see the other person. Here, we tested whether the interaction occurring with this sight is important or whether these social N400 increases also occur when the other person is seated a bit behind Pps, who are aware of it. All had to decide whether the word ending short stories was coherent, incoherent, or equivocal. No social N400 increase was observed: N400s elicited by those words in Pps who were with a confederate (n = 50) were similar to those of Pps who were alone (n = 51). On the other hand, equivocal endings did not elicit larger N400s than coherent ones but triggered larger late posterior positivities (LPPs), like in previous studies. The discussion focuses on the circumstances in which perspective-taking occurs and on the functional significance of the N400 and the LPP.

Interactions of lexical and conceptual representations: Evidence from EEG

We examined whether meanings automatically activate linguistic forms, and whether these forms affect semantic decisions. Participants were presented sequentially with pairs of pictures and decided whether the objects in the pictures were related. At no point did they name the pictures. The object names of the experimental stimuli were ambiguous either in orthography (homographs), phonology (homophones), or both (homonyms), or unambiguous. We show that the lexical characteristics of the name of the objects affect a semantic decision about real world relations, in an online measure (N400), in addition to offline behavioral measures. We show a dissociation between conceptual and lexical recognition, where an earlier component (N230), was affected by relatedness, but was not sensitive to the lexical characteristics. We interpret this as supporting the hypothesis that semantic recognition occurs before the automatic lexical activation of the object name, but that once linguistic representations are activated, they affect semantic integration.

Cinematographic continuity edits across shot scales and camera angles: an ERP analysis

Film editing has attracted great theoretical and practical interest since the beginnings of cinematography. In recent times, the neural correlates of visual transitions at edit cuts have been at the focus of attention in neurocinematics. Many Event Related Potential (ERP) studies studies have reported the consequences of cuts involving narrative discontinuities, and violations of standard montage rules. However, less is known about edits that are meant to induce continuity. Here, we addressed the neural correlates of continuity editing involving scale, and angle variations across the cut within the same scene, two of the most popular devices used for continuity editing. We recorded the electroencephalographic signal obtained from 20 viewers as they watched four different cinematographic excerpts to extract ERPs at edit points. First, we were able to reproduce the general time and scalp distribution of the typical ERPs to filmic cuts in prior studies. Second, we found significant ERP modulations triggered by scale changes (scale out, scale in, or maintaining the same scale). Edits involving an increase in scale (scale out) led to amplification of the ERP deflection, and scale reduction (scale in) led to decreases, compared to edits that kept scale across the cut. These modulations coincide with the time window of the N300 and N400 components and, according to previous findings, their amplitude has been associated with the likelihood of consciously detecting the edit. Third, we did not detect similar modulations as a function of angle variations across the cut. Based on these findings, we suggest that cuts involving reduction of scale are more likely to go unnoticed, than ones that scale out. This relationship between scale in/out and visibility is documented in film edition manuals. Specifically, in order to achieve fluidity in a scene, the edition is designed from the most opened shots to the most closed ones.

Neural Correlates and Perceived Attractiveness of Male and Female Shoulder-to-Hip Ratio in Men and Women: An EEG Study

While there are studies regarding the neural correlates of human facial attractiveness, there are few investigations considering neural responses for body form attractiveness. The most prominent physical feature defining men's attractiveness is their physical fitness and upper body strength. Shoulder-to-hip ratio (SHR), a sexually dimorphic trait in humans, is an indicator of men's attractiveness for both men and women. The current study is the first to report on the neurophysiological responses to male and female body forms varying in SHR in healthy heterosexual men and women observers. Electroencephalographic (EEG) signals were acquired while participants completed an oddball task as well as a subsequent attractiveness judgement task. Behavioral results showed larger SHRs were considered more attractive than smaller SHRs, regardless of stimuli and participants' sex. The electrophysiological results for both the oddball task and the explicit judgement of attractiveness showed that brain activity related to male SHR body stimuli differed depending on the specific ratios, both at early and late processing stages. For female avatars, SHR did not modulate neural activity. Collectively the data implicate posterior brain regions in the perception of body forms that differ in attractiveness vis-a-vis variation of SHR, and frontal brain regions when such perceptions are rated explicitly.

Processing different types of iconicity in Chinese transferred epithet comprehension: An ERP study

Transferred epithet can be regarded as a reflection of semantic markedness since the modifier and the modified conflict with each other and lead to semantic deviation; yet the corresponding processing mechanism is less studied. The present study examined the neurocognitive mechanism of Chinese transferred epithet comprehension by employing ERP technique from the perspective of Iconicity of Markedness. Participants were required to read materials with different types of semantic markedness, namely unmarked linguistic expression (literal sentences) and marked linguistic expression (transferred epithets), and then judge whether the targets were words or pseudo-words. In terms of semantic markedness, the targets are words reflecting the unmarked semantic meaning of literal sentences and marked semantic meaning of transferred epithets respectively. The target words after transferred epithets elicited a larger N400 and a smaller LPC than those in literal sentences. These results suggest that processing sentences with marked and unmarked iconicity involve different neural mechanisms, with the former requiring more cognitive efforts to extract the similarity features.

Pleasure of paying when using mobile payment: Evidence from EEG studies

Mobile payment has emerged as a popular payment method in many countries. While much research has focused on the antecedents of mobile payment adoption, limited research has investigated the consequences of mobile payment usage relating to how it would influence consumer behaviors (e.g., purchase intention or willingness to pay). Here, we propose that mobile payment not just reduces the “pain of paying,” a traditional view explaining why cashless payment stimulates spending, but it also evokes the “pleasure of paying,” raising from the enhanced processing fluency in completing transactions. We tested this new conceptualization of “pleasure of paying” using EEG, complementing other behavioral measures. In two studies, we found that mobile payment effectively enhanced purchase likelihood (study 1, N = 66) and such an enhancement is generalizable to both hedonic and utilitarian products (study 2, N = 29). By employing EEG measures, we provided the first neural evidence of “pleasure of paying” in addition to the signal of “pain of paying.” Critically, we demonstrated that the “pleasure of paying” is a distinctive psychological mechanism that is induced by mobile payment usage and that the “pleasure of paying” joins the “pain of paying” to mediate the increased purchase intention. We discuss the contributions and implications of these results to the ongoing evolution of cashless payment societies.

Three- and six-year-old children are sensitive to natural body expressions of emotion: An event-related potential emotional priming study

Body movements provide a rich source of emotional information during social interactions. Although the ability to perceive biological motion cues related to those movements begins to develop during infancy, processing those cues to identify emotions likely continues to develop into childhood. Previous studies used posed or exaggerated body movements, which might not reflect the kind of body expressions children experience. The current study used an event-related potential (ERP) priming paradigm to investigate the development of emotion recognition from more naturalistic body movements. Point-light displays (PLDs) of male adult bodies expressing happy or angry emotional movements while narrating a story were used as prime stimuli, whereas audio recordings of the words "happy" and "angry" spoken with an emotionally neutral prosody were used as targets. We recorded the ERPs time-locked to the onset of the auditory target from 3- and 6-year-old children, and we compared amplitude and latency of the N300 and N400 responses between the two age groups in the different prime-target conditions. There was an overall effect of prime for the N300 amplitude, with more negative-going responses for happy PLDs compared with angry PLDs. There was also an interaction between prime and target for the N300 latency, suggesting that all children were sensitive to the emotional congruency between body movements and words. For the N400 component, there was only an interaction among age, prime, and target for latency, suggesting an age-dependent modulation of this component when prime and target did not match in emotional information. Overall, our results suggest that the emergence of more complex emotion processing of body expressions occurs around 6 years of age, but it is not fully developed at this point in ontogeny.

The role of associative learning in healthy and sustainable food evaluations: An event-related potential study

Individuals in industrialized societies frequently include processed foods in their diet. However, overconsumption of heavily processed foods leads to imbalanced calorie intakes as well as negative health consequences and environmental impacts. In the present study, normal-weight healthy individuals were recruited in order to test whether associative learning (Evaluative Conditioning, EC) could strengthen the association between food-types (minimally processed and heavily processed foods) and concepts (e.g., healthiness), and whether these changes would be reflected at the implicit associations, at the explicit ratings and in behavioral choices. A Semantic Congruency task (SC) during electroencephalography recordings was used to examine the neural signature of newly acquired associations between foods and concepts. The accuracy after EC towards minimally processed food (MP-food) in the SC task significantly increased, indicating strengthened associations between MP-food and the concept of healthiness through EC. At the neural level, a more negative amplitude of the N400 waveform, which reflects semantic incongruency, was shown in response to MP-foods paired with the concept of unhealthiness in proximity of the dorsal lateral prefrontal cortex (DLPFC). This implied the possible role of the left DLPFC in changing food representations by integrating stimuli's features with existing food-relevant information. Finally, the N400 effect was modulated by individuals' attentional impulsivity as well as restrained eating behavior.

Toward P300-brain-computer interface access to contextual scene displays for AAC: An initial exploration of context and asymmetry processing in healthy adults

Brain-computer interfaces for augmentative and alternative communication (BCI-AAC) may help overcome physical barriers to AAC access. Traditionally, visually based P300-BCI-AAC displays utilize a symmetrical grid layout. Contextual scene displays are composed of context-rich images (e.g., photographs) and may support AAC success. However, contextual scene displays contrast starkly with the standard P300-grid approach. Understanding the neurological processes from which BCI-AAC devices function is crucial to human-centered computing for BCI-AAC. Therefore, the aim of this multidisciplinary investigation is to provide an initial exploration of contextual scene use for BCI-AAC.

METHODS

Participants completed three experimental conditions to evaluate the effects of item arrangement asymmetry and context on P300-based BCI-AAC signals and offline BCI-AAC accuracy, including 1) the full contextual scene condition, 2) asymmetrical item arraignment without context condition and 3) the grid condition. Following each condition, participants completed task-evaluation ratings (e.g., engagement). Offline BCI-AAC accuracy for each condition was evaluated using cross-validation.

RESULTS

Display asymmetry significantly decreased P300 latency in the centro-parietal cluster. P300 amplitudes in the frontal cluster were decreased, though nonsignificantly. Display context significantly increased N170 amplitudes in the occipital cluster, and N400 amplitudes in the centro-parietal and occipital clusters. Scenes were rated as more visually appealing and engaging, and offline BCI-AAC performance for the scene condition was not statistically different from the grid standard.

CONCLUSION

Findings support the feasibility of incorporating scene-based displays for P300-BCI-AAC development to help provide communication for individuals with minimal or emerging language and literacy skills.

EEG theta and N400 responses to congruent versus incongruent brand logos

Neuroimaging and behavioral studies have shown that brands convey meaning to consumers. To investigate the immediate reactions of the brain to brand logos, followed either by congruent or incongruent pictorial brand-related cues, can deepen understanding of the semantic processing of brands, and perhaps how consolidated the logo is in consumers’ minds. Participants were exposed to different brand-related image sets, that were either congruent (a match between brand-related images and brand logo) or incongruent (a mismatch between brand-related images and brand logo) while having their brain signals recorded. Event-related potential and EEG time–frequency domain features were extracted from the signals of the target image (brand logo). The results showed significantly larger N400 peak and relative theta power increase for incongruent compared to congruent logos, which could be attributed to an error-monitoring process. Thus, we argue that brands are encoded deeply in consumers’ minds, and cognitive processing of mismatched (vs matched) brand logos is more difficult, leading to greater error monitoring. The results were mostly consistent with previous studies investigating semantic incongruences in the linguistic field. Therefore, the error-monitoring process could be extended beyond linguistic forms, for example to images and brands.

Tracking the time course of sign recognition using ERP repetition priming

Repetition priming and event-related potentials (ERPs) were used to investigate the time course of sign recognition in deaf users of American Sign Language. Signers performed a go/no-go semantic categorization task to rare probe signs referring to people; critical target items were repeated and unrelated signs. In Experiment 1, ERPs were time-locked either to the onset of the video or to sign onset within the video; in Experiment 2, the same full videos were clipped so that video and sign onset were aligned (removing transitional movements), and ERPs were time-locked to video/sign onset. All analyses revealed an N400 repetition priming effect (less negativity for repeated than unrelated signs) but differed in the timing and/or duration of the N400 effect. Results from Experiment 1 revealed that repetition priming effects began before sign onset within a video, suggesting that signers are sensitive to linguistic information within the transitional movement to sign onset. The timing and duration of the N400 for clipped videos were more parallel to that observed previously for auditorily presented words and was 200 ms shorter than either time-locking analysis from Experiment 1. We conclude that time-locking to full video onset is optimal when early ERP components or sensitivity to transitional movements are of interest and that time-locking to the onset of clipped videos is optimal for priming studies with fluent signers.

Characteristics of N400 component elicited in patients who have migraine with aura

Background

This study aimed to examine the N400 effect and event-related potentials (ERPs) elicited from congruent and incongruent stimuli in patients who have migraines with aura (MwA).

Methods

A total of 33 MwA patients and 20 healthy controls (HCs) were studied. They were balanced in age (35.12 ± 8.94 vs 34.70 ± 9.59 years, p = 0.872) and sex (69.7 vs 75.0% females, p = 0.761). ERPs were measured in response to both stimuli, where pictures were preceded with an object name that either matched or mismatched with the object. Averaged amplitudes, peaks, peak latencies, difference waves and topography were compared between MwA and HCs.

Results

MwA patients had significantly lower averaged amplitudes at the Fz and F4 sites during incongruent stimuli, as well as reduced peaks at the C3 and Pz sites. Topography showed a more widespread N400 effect over scalp relative to HCs. The difference ERP waveforms did not differ in the N400 effect between groups, but the P600 effect was significantly stronger in the HCs group relative to the MwA group at the Pz (6.52 ± 2.57 vs. 3.50 ± 3.15, p = 0.001) and P4 (5.86 ± 2.79 vs. 3.95 ± 3.64, p = 0.040) sites.

Conclusions

Picture-word matching tasks could serve as a potential new method for the investigation of semantic processing in MwA patients.

Neural responses to rapidly selecting color names with different novelty

It is well known that different names of color can lead to distinct attractions to people. To study the neural mechanism underlying this phenomenon, an implicit association test task was designed for color names, in which participants were required to select the possible meanings of a Greek phrase from two color names (in Chinese). The behavioral results showed that the participants were more likely to select novel names for long Greek phrases and dates names for short Greek phrases. The EEG results showed that the mean amplitude of N1 was greater for selections of novel color names than selections of dates names for Greek phrases. Meanwhile, the mean amplitude of N3 for novel color names was more negative than that of dates color names. Significant interaction effect of N3 was also found for the four kinds of selections between Greek phrases and Chinese color names. Moreover, a frontal-positive and occipital-negative distribution for scalp topography of N1 was found, while the scalp topography of N3 was opposite as frontal-negative and occipital-positive distribution, suggesting the importance of visual cortex for perception of the color names and prefrontal cortex for integration and decision of selection. In summary, the results here indicated that colors with novel names could easily attract people's attention than colors with dates names, which might shed light on the usage of color names in real life.

The role of contextual materials in object recognition

While scene context is known to facilitate object recognition, little is known about which contextual "ingredients" are at the heart of this phenomenon. Here, we address the question of whether the materials that frequently occur in scenes (e.g., tiles in a bathroom) associated with specific objects (e.g., a perfume) are relevant for the processing of that object. To this end, we presented photographs of consistent and inconsistent objects (e.g., perfume vs. pinecone) superimposed on scenes (e.g., a bathroom) and close-ups of materials (e.g., tiles). In Experiment 1, consistent objects on scenes were named more accurately than inconsistent ones, while there was only a marginal consistency effect for objects on materials. Also, we did not find any consistency effect for scrambled materials that served as color control condition. In Experiment 2, we recorded event-related potentials and found N300/N400 responses-markers of semantic violations-for objects on inconsistent relative to consistent scenes. Critically, objects on materials triggered N300/N400 responses of similar magnitudes. Our findings show that contextual materials indeed affect object processing-even in the absence of spatial scene structure and object content-suggesting that material is one of the contextual "ingredients" driving scene context effects.

The influence of semantic relevance on the discernment of product appearance and function

Background

This study investigated the impact of semantic relevance on the ability to comprehend the appearance and function of a product, as presented in images.

Methods

The images used the constructs of Simile, Metaphor and Analogy to correspond to congruent, related and incongruent semantic structures, and measured the amplitude of Event-Related Potentials (ERPs) to compare these images with Landscape images. Sixteen participants with design-related educational backgrounds were invited to join in the ERP experiment.

Results

The results found that the image depicting the Metaphor showed a stronger N600 amplitude in the right anterior region of the brain than the Landscape image and the Analogy image induced a stronger N600 effect in the left anterior and right anterior part of the brain than the Landscape image. However, the Simile image did not trigger the N600. The N600 was triggered when the meaning of the Metaphor and Analogy being presented could not be understood. This indicates that a greater processing effort to comprehend them than was required for Simile. Analogy has a wider N600 distribution than Metaphor in the anterior area, suggesting that Analogy would require higher-level thinking processes and more complex semantic processing mechanisms than Metaphor.

Conclusions

The N600 implicated that an assessment method to detect the semantic relationship between appearance and function of a product would assist in determining whether a symbol was suitable to be associated with a product.

How to do Better N400 Studies: Reproducibility, Consistency and Adherence to Research Standards in the Existing Literature

Given the complexity of ERP recording and processing pipeline, the resulting variability of methodological options, and the potential for these decisions to influence study outcomes, it is important to understand how ERP studies are conducted in practice and to what extent researchers are transparent about their data collection and analysis procedures. The review gives an overview of methodology reporting in a sample of 132 ERP papers, published between January 1980 – June 2018 in journals included in two large databases: Web of Science and PubMed. Because ERP methodology partly depends on the study design, we focused on a well-established component (the N400) in the most commonly assessed population (healthy neurotypical adults), in one of its most common modalities (visual images). The review provides insights into 73 properties of study design, data pre-processing, measurement, statistics, visualization of results, and references to supplemental information across studies within the same subfield. For each of the examined methodological decisions, the degree of consistency, clarity of reporting and deviations from the guidelines for best practice were examined. Overall, the results show that each study had a unique approach to ERP data recording, processing and analysis, and that at least some details were missing from all papers. In the review, we highlight the most common reporting omissions and deviations from established recommendations, as well as areas in which there was the least consistency. Additionally, we provide guidance for a priori selection of the N400 measurement window and electrode locations based on the results of previous studies.

Effects of Design Aesthetics on the Perceived Value of a Product

Design aesthetics play a crucial role in product design. Stakeholders expect to develop highly valuable premium products by improving the design aesthetics of products. Nevertheless, the question of how to evaluate the value of design aesthetics has not been fully addressed. In this study, the effects of design aesthetics on the evaluation of the value of a product were investigated through a strictly controlled experiment in which the neural responses of the participants were measured. Forty participants completed the design aesthetics experiment in a laboratory setting. Images of products were divided into two categories: those representing high- and low-design-aesthetic stimuli. Both types of images were labeled with the same price. Overall, the images representing high design aesthetics elicited smaller N100 and lower P200 amplitudes than did the images representing low design aesthetics. This finding indicates that low design aesthetics attracted more attention than high design aesthetics did and that high design aesthetics triggered positive emotions. Low-design-aesthetic products elicited a larger N400 amplitude. This finding reveals the inconsistency between labeled and expected prices. The present study indicates that the N400 component can be used as an indicator for measuring the perceived value of a product in a future product design study. Our study provides event-related potential indicators that can be easily applied in decision making for measuring the perceived value of a product's design.

Does mobile payment change consumers' perception during payment process? -An ERP study

Innovative payment methods have been getting worldwide attention. Exploring the mechanisms behind consumers' purchase behaviors modulated by different payment methods was critical but challenging. In this paper, we proposed a 2 (payment methods: cash payment vs mobile payment) × 2 (price levels of products: high vs low) Event-Related Potentials (ERPs) experiment to study the difference of cash payment and mobile payment on consumers' purchase intention of products in different price level from a neuroscience perspective. Greater P200 amplitude was found in mobile payment condition, which meant that mobile payment captured more early attention resources than cash. Larger N270 amplitude was found in cash payment condition as the participants had to spend more cognitive resources and struggled more when using cash. Moreover, lower N270 amplitude was found in high-price product condition, which indicated that when there were limited cognitive resources, the affective process played a dominating role. Specifically, buying the high-price products with mobile payment, the consumers would experience the pleasure of consumption more rather than the pain of paying. The study offers insights on the cognitive process of consumers when they pay with different methods.

Regulation of Mindfulness-Based Music Listening on Negative Emotions Related to COVID-19: An ERP Study

The current study aimed to explore the behavioral and neural correlates of mindfulness-based music listening regulation of induced negative emotions related to COVID-19 using the face-word Stroop task. Eighty-five young adults visited the laboratory and were randomly assigned to three groups: a calm music group (CMG: n = 28), a happy music group (HMG: n = 30), and a sad music group (SMG: n = 27). Negative emotions were induced in all participants using a COVID-19 video, followed by the music intervention condition. Participants underwent the face-word Stroop tasks during which event-related potentials (ERPs) were recorded. The N2, N3, P3, and late positive component (LPC) were investigated. The results showed that calm music and happy music effectively regulate young adults' induced negative emotions, while young adults experienced more negative emotions when listening to sad music; the negative mood states at the post-induction phase inhibited the reaction of conflict control in face-word Stroop tasks, which manifested as lower accuracy (ACC) and slower reaction times (RTs). ERP results showed negative mood states elicited greater N2, N3, and LPC amplitudes and smaller P3 amplitudes. Further studies are needed to develop intervention strategies to enhance emotion regulation related to COVID-19 for other groups.

Which is better for concept visualization? Shape or spirit similarity: Evidence from event-related potentials

Shape and spirit similarity are two kinds of common artistic modes in concept visualization. The adoption depends on the designers' subjective preference and judgment, which may cause potential risks for semantic communication. This article used pairs of real image-concrete word as the roots, and contrasted four kinds of multimodal mappings such as shape similarity-concrete concept, shape similarity-abstract concept, spirit similarity-concrete concept, and spirit similarity-abstract concept to compare the matching difference through the S1(picture)-S2(word) paradigm. The behavioral results showed that shape similarity had advantages in both matching rate and reaction time over spirit similarity, but the difference was more significant to the concrete word than to the abstract word. The ERPs showed that the N1, P2, and N400 components had alike effects with the behavioral results, but the mappings of spirit similarity-concrete concept elicited the largest positivity of P600, suggesting the complicated mechanisms of semantic integration and concreteness effect in the multimodal mappings. This study proves that the concrete concept should be visualized according to its appearance, not the most striking feature or function; but the visulization of abstract concept shows less difference after a concreteness transition.

Novel Adjective Processing in Preschool Children: Evidence From Event-Related Brain Potentials

Purpose Recent findings in preschool children indicated novel adjective recall was enhanced when learned using repeated retrieval with contextual reinstatement (RRCR) compared to repeated study (RS). Recall was similar for learned pictures used during training and new (generalized) pictures with the same adjective features. The current study compared the effects of learning method and learned/generalized pictures on the neural processes mediating the recognition of novel adjectives. Method Twenty typically developing children aged 4;6-5;11 (years;months) learned four novel adjectives, two using RRCR and two using RS. Five-minute and 1-week tests assessed adjective recall using learned and generalized pictures. Also, at the 1-week visit, event-related potentials (ERPs) were recorded to assess children's processing of learned/generalized pictures, followed by naturally spoken novel adjectives in a match-mismatch paradigm. Results Naming recall and match-mismatch judgment accuracy were similar for the RS and RRCR conditions and across learned/generalized pictures. However, ERPs revealed more reliable condition effects in the phonological mapping negativity, indexing phonological expectations, and the late positive component, indexing semantic reanalysis, for the adjectives learned in the RRCR relative to the RS condition. Unfamiliar pictures (generalized) elicited larger amplitude N300 and N400 components relative to learned pictures. Conclusions Although behavioral accuracy measures suggest similar effects of the RS and RRCR learning conditions, subtle differences in the ERPs underlying novel adjective processing indicate advantages of RRCR for phonological processing and semantic reanalysis. While children readily generalized the novel adjectives, ERPs revealed greater cognitive resources for processing unfamiliar compared to learned pictures of the novel adjective characteristics. Supplemental Material https://doi.org/10.23641/asha.13683214.

A starring role for inference in the neurocognition of visual narratives

Research in verbal and visual narratives has often emphasized backward-looking inferences, where absent information is subsequently inferred. However, comics use conventions like star-shaped “action stars” where a reader knows events are undepicted at that moment, rather than omitted entirely. We contrasted the event-related brain potentials (ERPs) to visual narratives depicting an explicit event, an action star, or a “noise” panel of scrambled lines. Both action stars and noise panels evoked large N400s compared to explicit-events (300–500 ms), but action stars and noise panels then differed in their later effects (500–900 ms). Action stars elicited sustained negativities and P600s, which could indicate further interpretive processes and integration of meaning into a mental model, while noise panels evoked late frontal positivities possibly indexing that they were improbable narrative units. Nevertheless, panels following action stars and noise panels both evoked late sustained negativities, implying further inferential processing. Inference in visual narratives thus uses cascading mechanisms resembling those in language processing that differ based on the inferential techniques.

Motor Inhibition to Dangerous Objects: Electrophysiological Evidence for Task-dependent Aversive Affordances

Previous work suggests that perception of an object automatically facilitates actions related to object grasping and manipulation. Recently, the notion of automaticity has been challenged by behavioral studies suggesting that dangerous objects elicit aversive affordances that interfere with encoding of an object's motor properties; however, related EEG studies have provided little support for these claims. We sought EEG evidence that would support the operation of an inhibitory mechanism that interferes with the motor encoding of dangerous objects, and we investigated whether such mechanism would be modulated by the perceived distance of an object and the goal of a given task. EEGs were recorded by 24 participants who passively perceived dangerous and neutral objects in their peripersonal, boundary, or extrapersonal space and performed either a reachability judgment task or a categorization task. Our results showed that greater attention, reflected in the visual P1 potential, was drawn by dangerous and reachable objects. Crucially, a frontal N2 potential, associated with motor inhibition, was larger for dangerous objects only when participants performed a reachability judgment task. Furthermore, a larger parietal P3b potential for dangerous objects indicated the greater difficulty in linking a dangerous object to the appropriate response, especially when it was located in the participants' extrapersonal space. Taken together, our results show that perception of dangerous objects elicits aversive affordances in a task-dependent way and provides evidence for the operation of a neural mechanism that does not code affordances of dangerous objects automatically, but rather on the basis of contextual information.

Implicit Semantic Processing of Linguistic and Non-linguistic Stimuli in Adults with Autism Spectrum Disorder

While many individuals with autism spectrum disorder (ASD) experience difficulties with language processing, non-linguistic semantic processing may be intact. We examined neural responses to an implicit semantic priming task by comparing N400 responses—an event-related potential related to semantic processing—in response to semantically related or unrelated pairs of words or pictures. Adults with ASD showed larger N400 responses than typically developing adults for pictures, but no group differences occurred for words. However, we also observed complex modulations of N400 amplitude by age and by level of autistic traits. These results offer important implications for how groups are delineated and compared in autism research.

Equivalence relations do exist before they are tested: Confirmatory evidence revealed by EEG measurements

We examined equivalence-based N400 effects by comparing EEG data from participants with different experiences with equivalence testing. Before a priming task used in EEG measurement, Group 1 was given only matching-to-sample training trials whereas Group 2 was exposed to matching-to-sample training and equivalence probe trials. We asked whether exposure to the reinforcement contingency was sufficient to bring about an N400 outcome that might indicate potentially emergent equivalence relations or if such a response depended on experience with equivalence tests. Results showed robust N400 in both groups. Experience with equivalence tests did not further increase the N400 effects. Our findings add confirmatory evidence that equivalence relations may originate via the reinforcement contingency alone. Furthermore, complementary EEG data collected from priming tasks involving words from natural language showed functional overlap between laboratory-defined equivalence and natural word-based N400 effects.

Do Food Odors Differently Influence Cerebral Activity Depending on Weight Status? An Electroencephalography Study of Implicit Olfactory Priming Effects on the Processing of Food Pictures

Attentional automatic processes and cerebral activity may differ between individuals with different weight statuses in the presence of food stimuli (e.g. odors, pictures). In the present study, we used an implicit olfactory priming paradigm to test the influence of non-attentively perceived food odors on the cerebral activity underlying the processing of food pictures, in normal-weight, overweight, and obese adults. A pear odor and a pound cake odor were used as primes, respectively priming sweet low-energy-density foods and high-energy-density foods. Event-related potentials were recorded while the participants passively watched pictures of sweet low and high-energy-density foods, under the two priming conditions plus an odorless control condition. The amplitude and latency of several peaks were measured (P100, N100, P200, N400). As a major result, we found that weight status influences the cerebral activity underlying the processing of food cues outside of consciousness, as early as the first detectable P100 peak.

Parsing the Neural Mechanisms of Short-Term and Long-Term Associations in the Flanker Tasks: An ERP Analysis

The neural mechanisms of cognitive conflicts within various flanker tasks are still unclear, which may be mixed with different effects of short-term associations and long-term associations. We applied a perceptual (color) flanker task and a symbolic (arrow) flanker task to 25 healthy young adults, while the event-related potentials (ERP) and behavioral performance were recorded. The former contains stimulus-stimulus conflict (SSC) of short-term memory (STM) associations, and the latter contains stimulus-response conflict (SRC) of long-term memory (LTM) associations. Both flanker tasks included congruent and incongruent conditions. The reaction time demonstrated the stimulus-response conflict effect in the arrow flanker task without the stimulus-stimulus conflict effect in the color flanker task. The ERP results showed SSC enhanced the frontocentral N2b without behavioral effects. SRC increased the frontocentral P2 but decreased the centroparietal P3b with prolonged reaction time. In the comparison between both tasks, the color flanker task elicited both the centroparietal N2b/N300 and the frontocentral N400, and the arrow flanker task increased the occipital N1. Our findings provide new evidence that different neural mechanisms underlie conflict effects based on different types of memory associations.

Flexing Gender Perception: Brain Potentials Reveal the Cognitive Permeability of Grammatical Information

A growing body of recent research suggests that verbal categories, particularly labels, impact categorization and perception. These findings are commonly interpreted as demonstrating the involvement of language on cognition; however, whether these assumptions hold true for grammatical structures has yet to be investigated. In the present study, we investigated the extent to which linguistic information, namely, grammatical gender categories, structures cognition to subsequently influence categorical judgments and perception. In a nonverbal categorization task, French-English bilinguals and monolingual English speakers made gender-associated judgments about a set of image pairs while event-related potentials were recorded. The image sets were composed of an object paired with either a female or male face, wherein the object was manipulated for their conceptual gender relatedness and grammatical gender congruency to the sex of the following target face. The results showed that grammatical gender modulated the N1 and P2/VPP, as well as the N300 exclusively for the French-English bilinguals, indicating the inclusion of language in the mechanisms associated with attentional bias and categorization. In contrast, conceptual gender information impacted the monolingual English speakers in the later N300 time window given the absence of a comparable grammatical feature. Such effects of grammatical categories in the early perceptual stream have not been found before, and further provide grounds to suggest that language shapes perception.

Effects of Semantic Congruence on Sign Identification: An ERP Study

OBJECTIVE

The purpose of this study was to investigate the effects of semantic congruence and incongruence on sign identification by using event-related potentials (ERPs).

BACKGROUND

Sign systems have crucial roles in public spaces and traffic facilities. Poorly designed signs can easily confuse pedestrians and drivers and reduce the efficiency of public activities and urban administration.

METHOD

Thirty-one participants completed a sign identification experiment independently in a laboratory setting. Experimental materials were selected from GB/T 10001, a Chinese national recommendation standard that is officially named Public Information Graphical Symbols for Use on Signs. All ERP data were processed using MATLAB 13b, and behavioral data were analyzed using Stata 14.

RESULTS

N170, P200, N300, and N400 components were induced during semantic processing. Statistical analysis revealed that semantic congruence has a main effect on N300 in the frontal region and has a main effect on N400 at FZ in the frontal region, CPZ in the parietal-central region, and PZ in the parietal region. Amplitudes of N300 induced by picture-word matching were considerably different between the two experimental conditions at electrodes FZ and FCZ. Amplitudes of N400 were significantly larger in the incongruent condition than in the congruent condition.

CONCLUSION

The study demonstrated that N300 and N400 are promising indicators for measuring semantic congruence in future sign design.

APPLICATION

Our findings provide ERP indicators for measuring the semantic congruence of sign design, which can be easily applied to improve the efficiency of sign design and sign comprehension.

Predictability modulates neurocognitive semantic processing of non-verbal narratives

Predictability is known to modulate semantic processing in language, but it is unclear to what extent this applies for other modalities. Here we ask whether similar cognitive processes are at play in predicting upcoming events in a non-verbal visual narrative. Typically developing adults viewed comics sequences in which a target panel was highly predictable (“high cloze”), less predictable (“low cloze”), or incongruent with the preceding narrative context (“anomalous”) during EEG recording. High and low predictable sequences were determined by a pretest where participants assessed “what happened next?”, resulting in cloze probability scores for sequence outcomes comparable to those used to measure predictability in sentence processing. Through both factorial and correlational analyses, we show a significant modulation of neural responses by cloze such that N400 effects are diminished as a target panel in a comic sequence becomes more predictable. Predictability thus appears to play a similar role in non-verbal comprehension of sequential images as in language comprehension, providing further evidence for the domain generality of semantic processing in the brain.

A Facilitatory Effect of Perceptual Incongruity on Target-Source Matching in Pictorial Metaphors of Chinese Advertising: EEG Evidence

Using evoked response potentials, we investigated the implicit detection of incongruity during target-source matching in pictorial metaphors of Chinese advertising. Participants saw an image of a product (the target in a visual metaphorical relationship), and then made a same-different judgment in response to a second image (the source in a visual metaphorical relationship) which was (in)congruous to the first image in terms of shape and/or function. We collected behavioral (button-press reaction time and accuracy), and neural (N270, delta and theta band activity) measures. The time-frequency analysis showed faster processing of incongruous visual information. Moreover, shape and conceptual incongruity were associated with increased N270 amplitude as well as delta (1-3 Hz) and theta (4-8 Hz) band power. Noticeably, compared with conceptual incongruity, shape incongruity evoked a larger N270 amplitude and stronger delta and theta band oscillation. In addition, the average topographical analysis revealed a frontal and central distribution of the power activity. The analysis of attitudes towards the advertising metaphor pictures also proved the supportive role played by incongruity. In conclusion, incongruity facilitates target-source matching in pictorial metaphors of Chinese advertising. The findings obtained from the study are important to metaphor designs of advertising pictures.

The influence of sequential predictions on scene-gist recognition

Past research suggests that recognizing scene gist, a viewer's holistic semantic representation of a scene acquired within a single eye fixation, involves purely feed-forward mechanisms. We investigated whether expectations can influence scene categorization. To do this, we embedded target scenes in more ecologically valid, first-person-viewpoint image sequences, along spatiotemporally connected routes (e.g., an office to a parking lot). We manipulated the sequences' spatiotemporal coherence by presenting them either coherently or in random order. Participants identified the category of one target scene in a 10-scene-image rapid serial visual presentation. Categorization accuracy was greater for targets in coherent sequences. Accuracy was also greater for targets with more visually similar primes. In Experiment 2, we investigated whether targets in coherent sequences were more predictable and whether predictable images were identified more accurately in Experiment 1 after accounting for the effect of prime-to-target visual similarity. To do this, we removed targets and had participants predict the category of the missing scene. Images were more accurately predicted in coherent sequences, and both image predictability and prime-to-target visual similarity independently contributed to performance in Experiment 1. To test whether prediction-based facilitation effects were solely due to response bias, participants performed a two-alternative forced-choice task in which they indicated whether the target was an intact or a phase-randomized scene. Critically, predictability of the target category was irrelevant to this task. Nevertheless, results showed that sensitivity, but not response bias, was greater for targets in coherent sequences. Predictions made prior to viewing a scene facilitate scene-gist recognition.

Neural Signatures of Learning Novel Object-Scene Associations

Objects are perceived within rich visual contexts, and statistical associations may be exploited to facilitate their rapid recognition. Recent work using natural scene-object associations suggests that scenes can prime the visual form of associated objects, but it remains unknown whether this relies on an extended learning process. We asked participants to learn categorically structured associations between novel objects and scenes in a paired associate memory task while ERPs were recorded. In the test phase, scenes were first presented (2500 msec), followed by objects that matched or mismatched the scene; degree of contextual mismatch was manipulated along visual and categorical dimensions. Matching objects elicited a reduced N300 response, suggesting visuostructural priming based on recently formed associations. Amplitude of an extended positivity (onset ∼200 msec) was sensitive to visual distance between the presented object and the contextually associated target object, most likely indexing visual template matching. Results suggest recent associative memories may be rapidly recruited to facilitate object recognition in a top-down fashion, with clinical implications for populations with impairments in hippocampal-dependent memory and executive function.

Neurophysiological Correlates of Frequency, Concreteness, and Iconicity in American Sign Language

To investigate possible universal and modality-specific factors that influence the neurophysiological response during lexical processing, we recorded event-related potentials while a large group of deaf adults (n = 40) viewed 404 signs in American Sign Language (ASL) that varied in ASL frequency, concreteness, and iconicity. Participants performed a go/no-go semantic categorization task (does the sign refer to people?) to videoclips of ASL signs (clips began with the signer's hands at rest). Linear mixed-effects regression models were fit with per-participant, per-trial, and per-electrode data, allowing us to identify unique effects of each lexical variable. We observed an early effect of frequency (greater negativity for less frequent signs) beginning at 400 ms postvideo onset at anterior sites, which we interpreted as reflecting form-based lexical processing. This effect was followed by a more widely distributed posterior response that we interpreted as reflecting lexical-semantic processing. Paralleling spoken language, more concrete signs elicited greater negativities, beginning 600 ms postvideo onset with a wide scalp distribution. Finally, there were no effects of iconicity (except for a weak effect in the latest epochs; 1,000-1,200 ms), suggesting that iconicity does not modulate the neural response during sign recognition. Despite the perceptual and sensorimotoric differences between signed and spoken languages, the overall results indicate very similar neurophysiological processes underlie lexical access for both signs and words.

Your Brain on Comics: A Cognitive Model of Visual Narrative Comprehension

The past decade has seen a rapid growth of cognitive and brain research focused on visual narratives like comics and picture stories. This paper will summarize and integrate this emerging literature into the Parallel Interfacing Narrative‐Semantics Model (PINS Model)—a theory of sequential image processing characterized by an interaction between two representational levels: semantics and narrative structure. Ongoing semantic processes build meaning into an evolving mental model of a visual discourse. Updating of spatial, referential, and event information then incurs costs when they are discontinuous with the growing context. In parallel, a narrative structure organizes semantic information into coherent sequences by assigning images to categorical roles, which are then embedded within a hierarchic constituent structure. Narrative constructional schemas allow for specific predictions of structural sequencing, independent of semantics. Together, these interacting levels of representation engage in an iterative process of retrieval of semantic and narrative information, prediction of upcoming information based on those assessments, and subsequent updating based on discontinuity. These core mechanisms are argued to be domain‐general—spanning across expressive systems—as suggested by similar electrophysiological brain responses (N400, P600, anterior negativities) generated in response to manipulation of sequential images, music, and language. Such similarities between visual narratives and other domains thus pose fundamental questions for the linguistic and cognitive sciences.

Visual narratives like comics involve a range of complex cognitive operations in order to be understood. The Parallel Interfacing Narrative‐Semantics (PINS) Model integrates an emerging literature showing that comprehension of wordless image sequences balances two representational levels of semantic and narrative structure. The neurocognitive mechanisms that guide these processes are argued to overlap with other domains, such as language and music.