Retrieval and Monitoring Processes during Visual Working Memory: An ERP Study of the Benefit of Visual Semantics

Towards theoretically understanding how long-term memory semantics can support working memory performance

Working memory is the system that supports the temporary storage and processing of information. It is generally agreed that working memory is a mental workspace, with a combination of resources operating together to maintain information in mind for potential use in thought and action. Theories typically acknowledge the contributions of long-term memory to this system. One particular aspect of long-term memory, namely semantic long-term memory, can effectively supplement or "boost" working memory performance. This may be a relatively automatic process via the semantic properties of the stimuli or more active via strategy development and implementation. However, the precise mechanisms require greater theoretical understanding. In this review of the literature, we critically discuss theoretical models of working memory and their proposed links with long-term memory. We also explore empirical research that contributes to our understanding of the ways in which semantics can support performance of both verbal and visuospatial working memory tasks, with a view to potential intervention development. This includes the possibility of training people with lower performance (e.g., older adults) to use semantics during working memory tasks. We conclude that semantics may offer an opportunity to maximise working memory performance. However, to realise this potential, more research is needed, particularly in the visuospatial domain.

Neurophysiological responses during the binding process in working memory

Working memory is a limited-capacity system responsible for handling and temporarily maintaining information. The multicomponent model of working memory includes the episodic buffer, which encodes, retains, and integrates multimodal information from the visuospatial sketchpad and the phonological loop. Although the model is highly accepted, little research has been conducted to examine the binding process in working memory. This research aimed to examine the neurophysiological similarities and differences among three different types of bindings: verbal-verbal, visual-visual, and verbal-visual. Event-related potentials (ERPs) were recorded in 30 participants while two pairs of stimuli from the different types of bindings were presented followed by a single pair. Participants indicated whether the single pair was equal to one of the previous two pairs, even if the stimulus position was changed, or was not equal to any of them. Compared with crossmodal binding, unimodal binding enhanced the amplitude of the positive slow wave (PSW) during encoding and of the P300 component and PSW during retrieval. These ERPs have been linked to processes such as stimulus classification and association mechanisms. The present study demonstrated that different amounts of resources or underlying processes are required for crossmodal bindings than for unimodal bindings within working memory.

Autistic traits are associated with enhanced working memory capacity for abstract visual stimuli

We tested whether the association between autistic traits and enhanced performance in visual-perceptual tasks extends to visual working memory capacity. We predicted that any positive effect of autistic traits on visual working memory performance would be greatest during domain-specific tasks, in which visual resources must be relied upon. We used a visual 'matrix' task, involving recall of black-and-white chequered patterns which increased in size, to establish participants' capacity (span). We assessed 144 young adults' (M = 22.0 years, SD = 2.5) performance on abstract, 'low semantic' versus 'high semantic' task versions. The latter offered multimodal coding due to the availability of long-term memory resources that could supplement visual working memory. Participants also completed measures of autistic traits and trait anxiety. Autistic traits, especially Attention to Detail, Attention Switching, and Communication, positively predicted visual working memory capacity, specifically in the low semantic task, which relies on visual working memory resources. Autistic traits are therefore associated with enhanced processing and recall of visual information. The benefit is removed, however, when multimodal coding may be incorporated, emphasising the visual nature of the benefit. Strengths in focused attention to detail therefore appear to benefit domain-specific visual working memory task performance.

A Review of EEG and fMRI Measuring Aesthetic Processing in Visual User Experience Research

In human-computer interaction, the visual interaction of user experience (UX) and user interface (UI) plays an important role in enriching the quality of daily life. The purpose of our study analyzes the use of brain-computer interface (BCI), wearable technology, and functional magnetic resonance imaging (fMRI) to explore the aesthetic processing of visual neural response to UI and UX designs. Specifically, this review aims to understand neuroaesthetic processing knowledge, aesthetic appreciation models, and the ways in which visual brain studies can improve the quality of current and future UI and UX designs. Recent research has found that subjective evaluations of aesthetic appreciation produce different results for objective evaluations of brain research analysis. We applied SWOT analysis and examined the advantages and disadvantages of both evaluation methods. Furthermore, we conducted a traditional literature review on topics pertaining to the use of aesthetic processing knowledge in the visual interaction field in terms of art therapy, information visualization, website or mobile applications, and other interactive platforms. Our main research findings from current studies have helped and motivated researchers and designers to use convincing scientific knowledge of brain event-related potential, electroencephalography, and fMRI to understand aesthetic judgment. The key trend finds that many designers, artists, and engineers use artistic BCI technology in the visual interaction experience. Herein, the scientific methods applied in the aesthetic appreciation to human-computer interface are summarized, and the influence of the latest wearable brain technology on visual interaction design is discussed. Furthermore, current possible research entry points for aesthetics, usability, and creativity in UI and UX designs are explicated. The study results have implications for the visual user experience research domain as well as for interaction industries, which produce interactive projects to improve people's daily lives.

Does rehearsal matter? Left anterior temporal alpha and theta band changes correlate with the beneficial effects of rehearsal on working memory

Rehearsal during working memory (WM) maintenance is assumed to facilitate retrieval. Less is known about how rehearsal modulates WM delay activity. In the present study, 44 participants completed a Sternberg Task with either intact novel scenes or phase-scrambled scenes, which had similar color and spatial frequency but lacked semantic content. During the rehearsal condition participants generated a descriptive label during encoding and covertly rehearsed during the delay period. During the suppression condition participants did not generate a label during encoding and suppressed (repeated "the") during the delay period. This was easy in the former (novel scenes) but more difficult in the later condition (phase-scrambled scenes) where scenes lacked semantic content. Behavioral performance and EEG delay activity was analyzed as a function of maintenance strategy. Performance during WM revealed a benefit of rehearsal for phase-scrambled but not intact scenes. Examination of the absolute amplitude revealed three underlying sources of activity for rehearsal, including the left anterior temporal (ATL) and left and midline parietal regions. Increases in alpha and theta activity in ATL were correlated with improvement in performance on WM with rehearsal only when labeling was not automatic (e.g., phase-scrambled scenes), which may reflect differences in labeling and rehearsal (i.e., semantic associations vs. shallow labels). We conclude that rehearsal only benefits memory for visual stimuli that lack semantic information, and that this is correlated with changes in alpha and theta rhythms.

Unraveling the Electrophysiological Activity Behind Recognition Memory

Abstract. Traditionally, most event related potential (ERP) studies of memory retrieval have been reported during item-recognition tasks. Those studies lead to two well-known ERP memory components termed FN400 (familiarity) and LPC (recollection). Nevertheless, some critics have raised concerns regarding the actual meaning of that activity since it emerges as the result of contrasting two different memory traces (previously studied vs. seen for the first time), and it is registered after the target presentation. Therefore, they possibly depict operations not related to memory itself but some cognitive processes associated with recognition memory. Based on those critics, we propose an innovative approach to study electrophysiological activity underlying recognition memory. We compared two very similar tasks with only one of them requiring subjects to actively retrieve a “cue-target” pair of visual stimuli from memory, while the other task required subjects to recognize the target stimulus as equal/different to the cue. Because of this experimental manipulation, we assured that active memory retrieval processes take place between the presentation of the cue and the target stimuli for only one of the tasks. As a result, responses upon the targets can give us valuable information regarding ERP components associated with recognition based on memory retrieval. We found three components possibly related to brain computations necessary to achieve correct target recognition. A N200-like component linked to executive functions (inhibition) from frontal cortices, a P300-like component, related to the expectation of the target stimulus, and a P600-like component associated to recognition based on LTM retrieval. These results help us to understand the complexity behind ERP components associated with recognition memory.

Older Adults Benefit from Symmetry, but Not Semantic Availability, in Visual Working Memory

Visual working memory exhibits age effects that are amongst the largest observed in the cognitive aging literature. In this research we investigated whether or not older adults can benefit from visual symmetry and semantic availability, as young adults typically do. Visual matrix pattern tasks varied in terms of the perceptual factor of symmetry (Experiment 1), as well as the availability of visual semantics, or long-term memory (LTM; Experiment 2). In Experiment 1, within a visual memory span protocol, four matrix pattern sets were employed with discrete symmetry characteristics; random, vertical, horizontal, and diagonal symmetry. Encoding time was 3 s with a 2 s maintenance interval. The findings indicated a significant difference in span level across age groups for all of the symmetry variants. More importantly, both younger and older adults could take advantage of symmetry in the matrix array in order to significantly improve task performance. In Experiment 2, two visual matrix task sets were used, with visual arrays of either low or high semantic availability (i.e., they contained stimuli with recognizable shapes that allow for LTM support). Encoding duration was 3 s with a 1 s retention interval. Here, the older adult sample was significantly impaired in span performance with both variants of the task. However, only the younger adult participants could take advantage of visual semantics. These findings show that, in the context of overall impairment in individual task performance, older adults remain capable of employing the perceptual cue of symmetry in order to improve visual working memory task performance. However, they appear less able, within this protocol, to recruit visual semantics in order to scaffold performance.