Flexible cognitive resources: competitive content maps for attention and memory

Functional dissociation of multiple-object tracking mechanisms based on hemispheric asymmetries

BACKGROUND

A number of theoretical accounts have been put forward to explain the ability to simultaneously track multiple visually indistinguishable objects over a period of time. Serial processing models of visual tracking focus on the maintenance of the spatial locations of every single item over time. A more recent mechanism describes multiple object tracking as the ability to maintain a higher order representation of an abstract spatial configuration built by the illusory connection of the tracked items through their transition.

OBJECTIVE

The current study investigates the correspondence between these serial and parallel tracking accounts and the right hemispheric specialization for the space-based vs. left hemispheric for object-based attentional processing.

METHODS

Electrophysiological brain responses were recorded in two groups of patients with right- and left hemispheric lesions while performing in a multiple object tracking task.

RESULTS

The results suggest a failure to distinguish single item information for the right hemispheric patients accompanied by the absence of a known electrophysiological marker associated with single item tracking. Importantly, left hemispheric patients showed a graded behavioral and electrophysiological response to probe stimuli as a function of the congruence of the probe with the relevant target stimuli.

CONCLUSIONS

The current data suggest that the differential contribution of serial and parallel tracking mechanisms during object tracking can partly be explained by the different functional contributions of the right and left brain hemispheres.

Neural substrates of the interplay between cognitive load and emotional involvement in bilingual decision making

Prior work has reported that foreign language influences decision making by either reducing access to emotion or imposing additional cognitive demands. In this fMRI study, we employed a cross-task design to assess at the neural level whether and how the interaction between cognitive load and emotional involvement is affected by language (native L1 vs. foreign L2). Participants completed a Lexico-semantic task where in each trial they were presented with a neutrally or a negatively valenced word either in L1 or L2, either under cognitive load or not. We manipulated cognitive load by varying the difficulty of the task: to increase cognitive demands, we used traditional characters instead of simplified ones in L1 (Chinese), and words with capital letters instead of lowercase letters in L2 (English). After each trial, participants decided whether to take a risky decision in a gambling game. During the Gamling task, left amygdala and right insula were more activated after having processed a negative word under cognitive load in the Lexico-semantic task. However, this was true for L1 but not for L2. In particular, in L1, cognitive load facilitated rather than hindered access to emotion. Further suggesting that cognitive load can enhance emotional sensitivity in L1 but not in L2, we found that functional connectivity between reward-related striatum and right insula increased under cognitive load only in L1. Overall, results suggest that cognitive load in L1 can favor access to emotion and lead to impulsive decision making, whereas cognitive load in L2 can attenuate access to emotion and lead to more rational decisions.

Sense of Presence and Subjective Well-Being in Online Pet Watching: The Moderation Role of Loneliness and Perceived Stress

Watching videos/livestreams concerning pets is becoming an increasingly popular phenomenon among youth in China, thus the social dynamics and psychological impacts of this pet-centred online activities worth in-depth exploration. This study investigates the sensual experiences of the audiences who have watched pet videos/livestreams and examines how these online experiences influence their subjective well-being. We develop a conceptual model that bases on the relationships between telepresence, social presence, flow experience, and subjective well-being to comprehend this mechanism. The result of 439 samples indicates that both telepresence and social presence have significant positive effects on flow experience, and social presence also has a positive impact on subjective well-being. We also examine the role of loneliness and perceived stress in moderating the effects of online pet watching on subjective well-being, showing that loneliness has a significant positive moderating effect on the relationship between social presence and flow experience, while perceived pressure has a negative moderating effect on the relationship between telepresence and flow experience. This study not only demonstrates the positive effect of an online pet on subjective well-being and but also uses interview data to comprehend the social processes underlying this effect. We also discuss the theoretical and practical values of this study in improving public health in the digital age.

Foveal gravity: A robust illusion of color-location misbinding

Some types of object features, such as color, shape, or location, can be processed separately within the visual system, requiring that they be correctly "bound" to a single object via attentional selection of a subset of visual information. Forcing selection to spread too widely can cause an illusion where these features misbind to objects, creating illusory objects that were never present. Here, we present a novel display that produces a robust color-location misbinding illusion that we call foveal gravity (viewable at https://osf.io/2bndg/). When observers selected only a set of colored objects, colors were largely perceived in their correct locations. When observers additionally selected objects in the far periphery, colors in the near periphery migrated closer to the fovea on over 35% of trials. We speculate that foveal gravity occurs because locations closer to the fovea are more likely to defeat more peripheral locations in competitive interactions to "win" the task-relevant color.

Effects of visual short-term memory load and attentional demand on the contrast response function

Visual short-term memory (VSTM) load leads to impaired perception during maintenance. Here, we fitted the contrast response function to psychometric orientation discrimination data while also varying attention demand during maintenance to investigate: (1) whether VSTM load effects on perception are mediated by a modulation of the contrast threshold, consistent with contrast gain accounts, or by the function asymptote (1 lapse rate), consistent with response gain accounts; and (2) whether the VSTM load effects on the contrast response function depend on the availability of attentional resources. We manipulated VSTM load via the number of items in the memory set in a color and location VSTM task and assessed the contrast response function for an orientation discrimination task during maintenance. Attention demand was varied through spatial cuing of the orientation stimulus. Higher VSTM load increased the estimated contrast threshold of the contrast response function without affecting the estimated asymptote, but only when the discrimination task demanded attention. When attentional demand was reduced (in the cued conditions), the VSTM load effects on the contrast threshold were eliminated. The results suggest that VSTM load reduces perceptual sensitivity by increasing contrast thresholds, suggestive of a contrast gain modulation mechanism, as long as the perceptual discrimination task demands attention. These findings support recent claims that attentional resources are shared between perception and VSTM maintenance processes.

How Do Humans Perform in Multiple Object Tracking With Unstable Features

In real-world scenarios, objects’ surface features sometimes change as they move, impairing the continuity of objects. However, it is still unknown how our visual system adapts to this dynamic change. Hence, the present study investigated the role of feature changes in attentive tracking through a modified multiple object tracking (MOT) task. The feature heterogeneity and feature stability were manipulated in two experiments. The results from Experiment 1 showed that the tracking performance under feature-changed condition was lower than that under the feature-fixed condition only when the objects were four colors grouped or all unique, suggesting that the performance decrease was moderated by the feature heterogeneity. In Experiment 2, we further examined this effect by manipulating the frequency of feature change. The results showed that when the target set was one color or two colors grouped (the color grouping for the distractor set corresponded with it), the tracking performance decreased significantly as the feature-change frequency increased. However, this trend was not the case when the objects were of the same color or eight unique colors. In addition, a relatively consistent effect appeared both in Experiments 1 and 2. When objects have unique features, the tracking performance decreased significantly as the increase of feature heterogeneity in each frequency of feature changes. Taken together, we concluded that unstable features could be utilized in attentive tracking, and the extent to which the observers relied on surface feature information to assist tracking depended on the level of feature heterogeneity and the frequency of feature change.

Hemifield-specific control of spatial attention and working memory: Evidence from hemifield crossover costs

Attentional tracking and working memory tasks are often performed better when targets are divided evenly between the left and right visual hemifields, rather than contained within a single hemifield (Alvarez & Cavanagh, 2005; Delvenne, 2005). However, this bilateral field advantage does not provide conclusive evidence of hemifield-specific control of attention and working memory, because it can be explained solely from hemifield-limited spatial interference at early stages of visual processing. If control of attention and working memory is specific to each hemifield, maintaining target information should become more difficult as targets move between the two hemifields. Observers in the present study maintained targets that moved either within or between the left and right hemifields, using either attention (Experiment 1) or working memory (Experiment 2). Maintaining spatial information was more difficult when target items moved between the hemifields compared with when target items moved within their original hemifields, consistent with hemifield-specific control of spatial attention and working memory. However, this pattern was not found for maintaining identity information (e.g., color) in working memory (Experiment 3). Together, these results provide evidence that control of spatial attention and working memory is specific to each hemifield, and that hemifield-specific control is a unique signature of spatial processing.

Dissociable neural mechanisms underlie currently-relevant, future-relevant, and discarded working memory representations

In daily life, we use visual working memory (WM) to guide our actions. While attending to currently-relevant information, we must simultaneously maintain future-relevant information, and discard information that is no longer relevant. However, the neural mechanisms by which unattended, but future-relevant, information is maintained in working memory, and future-irrelevant information is discarded, are not well understood. Here, we investigated representations of these different information types, using functional magnetic resonance imaging in combination with multivoxel pattern analysis and computational modeling based on inverted encoding model simulations. We found that currently-relevant WM information in the focus of attention was maintained through representations in visual, parietal and posterior frontal brain regions, whereas deliberate forgetting led to suppression of the discarded representations in early visual cortex. In contrast, future-relevant information was neither inhibited nor actively maintained in these areas. These findings suggest that different neural mechanisms underlie the WM representation of currently- and future-relevant information, as compared to information that is discarded from WM.

Expansion and Contraction Modulate Visual Short-Term Memory

We examined the influence of expansion and contraction on visual short-term memory (VSTM) using a change-detection task. In each trial, several expanding/contracting line segments with various orientations were presented in two successive displays. The orientation of objects in the second display was either identical to, or different from, that of the first display. Observers were asked to judge the presence or absence of an orientation change in successive displays. Results showed that memory performance for expanding objects was higher than for contracting objects (expansion benefit: Experiments 1A and 1B). Further experiments focused on VSTM processing (encoding, storage, and retrieval). Regarding the retrieval stage, an expansion benefit was replicated only when the direction of motion was consistent between two successive displays (Experiment 2A). A cueing stimulus enhanced the memory performance for both expanding and contracting motions and eliminated the expansion benefit (Experiment 2B). Regarding the storage stage, we found the expansion benefit occurred only for shorter blank intervals between the two displays (Experiment 3). Regarding the encoding stage, the expansion benefit was observed regardless of presentation times (Experiment 4). These results indicate the possibility that expanding and contracting motions modulate VSTM.

Demystifying visual awareness: Peripheral encoding plus limited decision complexity resolve the paradox of rich visual experience and curious perceptual failures

Human beings subjectively experience a rich visual percept. However, when behavioral experiments probe the details of that percept, observers perform poorly, suggesting that vision is impoverished. What can explain this awareness puzzle? Is the rich percept a mere illusion? How does vision work as well as it does? This paper argues for two important pieces of the solution. First, peripheral vision encodes its inputs using a scheme that preserves a great deal of useful information, while losing the information necessary to perform certain tasks. The tasks rendered difficult by the peripheral encoding include many of those used to probe the details of visual experience. Second, many tasks used to probe attentional and working memory limits are, arguably, inherently difficult, and poor performance on these tasks may indicate limits on decision complexity. Two assumptions are critical to making sense of this hypothesis: (1) All visual perception, conscious or not, results from performing some visual task; and (2) all visual tasks face the same limit on decision complexity. Together, peripheral encoding plus decision complexity can explain a wide variety of phenomena, including vision's marvelous successes, its quirky failures, and our rich subjective impression of the visual world.

Affective Bias without Hemispheric Competition: Evidence for Independent Processing Resources in Each Cortical Hemisphere

We assessed the extent of neural competition for attentional processing resources in early visual cortex between foveally presented task stimuli and peripheral emotional distracter images. Task-relevant and distracting stimuli were shown in rapid serial visual presentation (RSVP) streams to elicit the steady-state visual evoked potential, which serves as an electrophysiological marker of attentional resource allocation in early visual cortex. A task-related RSVP stream of symbolic letters was presented centrally at 15 Hz while distracting RSVP streams were displayed at 4 or 6 Hz in the left and right visual hemifields. These image streams always had neutral content in one visual field and would unpredictably switch from neutral to unpleasant content in the opposite visual field. We found that the steady-state visual evoked potential amplitude was consistently modulated as a function of change in emotional valence in peripheral RSVPs, indicating sensory gain in response to distracting affective content. Importantly, the facilitated processing for emotional content shown in one visual hemifield was not paralleled by any perceptual costs in response to the task-related processing in the center or the neutral image stream in the other visual hemifield. Together, our data provide further evidence for sustained sensory facilitation in favor of emotional distracters. Furthermore, these results are in line with previous reports of a “different hemifield advantage” with low-level visual stimuli and are suggestive of independent processing resources in each cortical hemisphere that operate beyond low-level visual cues, that is, with complex images that impact early stages of visual processing via reentrant feedback loops from higher order processing areas.

Limitations of concurrently representing objects within view and in visual working memory

Representing visibly present stimuli is as limited in capacity as representing invisible stimuli in visual working memory (WM). In this study, we explored whether concurrently representing stimuli within view affects representing objects in visual WM, and if so, whether this effect is modulated by the storage states (active and silent state) of memory contents? In experiment 1, participants were asked to perform the change-detect task in a simultaneous-representing condition in which WM content and the continuously-visible stimuli in view were simultaneously represented, as well as a baseline condition in which only the representations of visual WM content were maintained. The results showed that the representations in visual WM would be impaired when the continuously-visible stimuli in view were concurrently represented, revealed by the reduced CDA amplitude and the lower behavior performance. In experiment 2, a dual-serial retro-cue paradigm was adopted to guide participants to maintain memory items in two different storage states, and the results revealed that simultaneously representing the continuously-visible stimuli and the WM content would only impair the WM representations in the active state. These evidences demonstrated that only the visual WM representations that were maintained in the active state would definitely share the limited resources with the representations of continuously-visible information, and further supported the dissociation between the active state and silent state of visual WM storage.

Shifts of Spatial Attention in Visual and Tactile Working Memory are Controlled by Independent Modality-Specific Mechanisms

The question whether the attentional control of working memory (WM) is shared across sensory modalities remains controversial. Here, we investigated whether attention shifts in visual and tactile WM are regulated independently. Participants memorized visual and tactile targets in a first memory sample set (S1) before encoding targets in a second sample set (S2). Importantly, visual or tactile S2 targets could appear on the same side as the corresponding S1 targets, or on opposite sides, thus, requiring shifts of spatial attention in visual or tactile WM. The activation of WM representations in modality-specific visual and somatosensory areas was tracked by recording visual and tactile contralateral delay activity (CDA/tCDA). CDA/tCDA components emerged contralateral to the side of visual or tactile S1 targets, and reversed polarity when S2 targets in the same modality appeared on the opposite side. Critically, the visual CDA was unaffected by the presence versus absence of concurrent attention shifts in tactile WM, and the tactile CDA remained insensitive to visual attention shifts. Visual and tactile WM performance was also not modulated by attention shifts in the other modality. These results show that the dynamic control of visual and tactile WM activation processes operates in an independent modality-specific fashion.

Differential Contribution of Cortical Thickness, Surface Area, and Gyrification to Fluid and Crystallized Intelligence

Human intelligence can be broadly subdivided into fluid (gf) and crystallized (gc) intelligence, each tapping into distinct cognitive abilities. Although neuroanatomical correlates of intelligence have been previously studied, differential contribution of cortical morphologies to gf and gc has not been fully delineated. Here, we tried to disentangle the contribution of cortical thickness, cortical surface area, and cortical gyrification to gf and gc in a large sample of healthy young subjects (n = 740, Human Connectome Project) with high-resolution MRIs, followed by replication in a separate data set with distinct cognitive measures indexing gf and gc. We found that while gyrification in distributed cortical regions had positive association with both gf and gc, surface area and thickness showed more regional associations. Specifically, higher performance in gf was associated with cortical expansion in regions related to working memory, attention, and visuo-spatial processing, while gc was associated with thinner cortex as well as higher cortical surface area in language-related networks. We discuss the results in a framework where "horizontal" cortical expansion enables higher resource allocation, computational capacity, and functional specificity relevant to gf and gc, while lower cortical thickness possibly reflects cortical pruning facilitating "vertical" intracolumnar efficiency in knowledge-based tasks relevant mostly to gc.

Distinct Neural Mechanisms Meet Challenges in Dynamic Visual Attention due to Either Load or Object Spacing

When engaged in dynamic visuospatial tasks, the brain copes with perceptual and cognitive processing challenges. During multiple-object tracking (MOT), the number of objects to be tracked (i.e., load) imposes attentional demands, but so does spatial interference from irrelevant objects (i.e., close encounters). Presently, it is not clear whether the effect of load on accuracy solely depends on the number of close encounters. If so, the same cognitive and physiological mechanisms deal with increasing load by preparing for and dealing with spatial interference. However, this has never been directly tested. Such knowledge is important to understand the neurophysiology of dynamic visual attention and resolve conflicting views within visual cognition concerning sources of capacity limitations. We varied the processing challenge in MOT task in two ways: the number of targets and the minimum spatial proximity between targets and distractors. In a first experiment, we measured task-induced pupil dilations and saccades during MOT. In a separate cohort, we measured fMRI activity. In both cohorts, increased load and close encounters (i.e., close spatial proximity) led to reduced accuracy in an additive manner. Load was associated with pupil dilations, whereas close encounters were not. Activity in dorsal attentional areas and frequency of saccades were proportionally larger both with higher levels of load and close encounters. Close encounters recruited additionally ventral attentional areas that may reflect orienting mechanisms. The activity in two brainstem nuclei, ventral tegmental area/substantia nigra and locus coeruleus, showed clearly dissociated patterns. Our results constitute convergent evidence indicating that different mechanisms underlie processing challenges due to load and object spacing.

POSTURAL ADJUSTMENTS OF ACTIVE YOUTHS IN PERTURBATION AND DUAL-TASK CONDITIONS

ABSTRACT Introduction Cognitive components are necessary to maintain posture during external perturbations. However, few studies have investigated postural control when external perturbations are associated with a concomitant cognitive task (DT). Objectives To analyze the behavior of reactive adjustments after perturbation with different intensities and displacements in active young adults; and to analyze the influence of DT on predictive and reactive adjustments in different perturbation conditions. Methods Twenty-eight physically active young adults stood on an item of equipment that produced displacements of the base. Four experimental conditions were introduced in a single task (ST) and DT (cognitive-report how many times a pre-established number appeared in the audio): 1 (5 cm and 10 cm/s); 2 (5 cm and 25 cm/s); 3 (12 cm and 10 cm/s) and 4 (12 cm and 25 cm/s). Three attempts were carried out for each condition (total=24). Center of pressure (CoP) parameters were analyzed considering the following windows: predictive (-250 to +50 ms), reactive 1 (+50 to +200 ms) and reactive 2 (+200 to +700 ms), in comparison to the start of the CoP activity. One-way ANOVAs were performed to analyze predictive adjustments, while two-way ANOVAs with factor for task (STxDT) and condition (1x2x3x4), with repeated measurements, were performed for the reactive adjustments. Results One-way ANOVA (predictive) indicated that the subjects had higher CoP parameters in ST vs DT. In reactive adjustments 1 and 2, ANOVA indicated greater CoP parameters in condition 2 and 4 when compared to 1 and 3, and in the ST vs DT. The subjects took longer to recover stable position in conditions 1 and 3 than in conditions 2 and 4. Conclusion Perturbation intensity has a greater influence on postural adjustments to maintain balance than on magnitude. Moreover, the association of cognitive tasks with external perturbation decreases CoP oscillation. Therefore, cognitive resources play an important role in postural control after perturbation. Level of evidence III; Study of nonconsecutive patients, with no “gold” standard applied uniformly.

Working memory prioritization impacts neural recovery from distraction

The ability to protect goal-relevant information from disruption over short intervals is a hallmark of working memory. Recent behavioral data suggest that high-priority items in working memory are more vulnerable to disruption. We used functional magnetic resonance imaging to evaluate the hypothesis that prioritization of working memories might impact the recovery of their neural representation(s) after distraction. A delay-period retrospective cue informed participants which of two memory items (a face or a scene) to prioritize during a first delay period. Consistent with prior work, and confirming successful prioritization, multivoxel pattern classifier evidence in perceptual brain regions was higher for cued versus uncued memory items. A distraction task was then imposed before a second retrospective cue informed participants to either "stay" remembering the previously cued item or "switch" to the previously uncued item. This allowed for the evaluation of recovery for high-priority items (on stay trials) and also low-priority items (on switch trials). Classifiers showed successful reinstatement of both high- and low-priority items after distraction, but only low-priority items recovered to their pre-distraction representational levels. Moreover, the degree of prioritization before distraction predicted the amount of disruption for high-priority items after distraction, suggesting that the more a participant prioritized the cued item, the greater the impact of distraction. Our data provide neural evidence that prioritizing working memory information in perceptual regions makes that information more vulnerable to disruption.

The offline stream of conscious representations

When do we become conscious of a stimulus after its presentation? We would all agree that this necessarily takes time and that it is not instantaneous. Here, I would like to propose not only that conscious access is delayed relative to the external stimulation, but also that it can flexibly desynchronize from external stimulation; it can process some information 'offline', if and when it becomes relevant. Thus, in contrast with initial sensory processing, conscious experience might not strictly follow the sequence of events in the environment. In this article, I will review gathering evidence in favour of this proposition. I will argue that it offers a coherent framework for explaining a great variety of observations in the domain of perception, sensory memory and working memory: the psychological refractory period, the attentional blink, post-dictive phenomena, iconic memory, latent working memory and the newly described retro-perception phenomenon. I will integrate this proposition to the global neuronal workspace model and consider possible underlying brain mechanisms. Finally, I will argue that this capacity to process information 'offline' might have made conscious processing evolutionarily advantageous in spite of its sluggishness and capacity limitations.This article is part of the theme issue 'Perceptual consciousness and cognitive access'.

Exploring Cognitive Flexibility With a Noninvasive BCI Using Simultaneous Steady-State Visual Evoked Potentials and Sensorimotor Rhythms

EEG-based brain-computer interface (BCI) technology creates non-biological pathways for conveying a user's mental intent solely through noninvasively measured neural signals. While optimizing the performance of a single task has long been the focus of BCI research, in order to translate this technology into everyday life, realistic situations, in which multiple tasks are performed simultaneously, must be investigated. In this paper, we explore the concept of cognitive flexibility, or multitasking, within the BCI framework by utilizing a 2-D cursor control task, using sensorimotor rhythms (SMRs), and a four-target visual attention task, using steady-state visual evoked potentials (SSVEPs), both individually and simultaneously. We found no significant difference between the accuracy of the tasks when executing them alone (SMR-57.9% ± 15.4% and SSVEP-59.0% ± 14.2%) and simultaneously (SMR-54.9% ± 17.2% and SSVEP-57.5% ± 15.4%). These modest decreases in performance were supported by similar, non-significant changes in the electrophysiology of the SSVEP and SMR signals. In this sense, we report that multiple BCI tasks can be performed simultaneously without a significant deterioration in performance; this finding will help drive these systems toward realistic daily use in which a user's cognition will need to be involved in multiple tasks at once.

Object Vision in a Structured World

In natural vision, objects appear at typical locations, both with respect to visual space (e.g., an airplane in the upper part of a scene) and other objects (e.g., a lamp above a table). Recent studies have shown that object vision is strongly adapted to such positional regularities. In this review we synthesize these developments, highlighting that adaptations to positional regularities facilitate object detection and recognition, and sharpen the representations of objects in visual cortex. These effects are pervasive across various types of high-level content. We posit that adaptations to real-world structure collectively support optimal usage of limited cortical processing resources. Taking positional regularities into account will thus be essential for understanding efficient object vision in the real world.

Item-specific delay activity demonstrates concurrent storage of multiple active neural representations in working memory

Persistent neural activity that encodes online mental representations plays a central role in working memory (WM). However, there has been debate regarding the number of items that can be concurrently represented in this active neural state, which is often called the “focus of attention.” Some models propose a strict single-item limit, such that just 1 item can be neurally active at once while other items are relegated to an activity-silent state. Although past studies have decoded multiple items stored in WM, these studies cannot rule out a switching account in which only a single item is actively represented at a time. Here, we directly tested whether multiple representations can be held concurrently in an active state. We tracked spatial representations in WM using alpha-band (8–12 Hz) activity, which encodes spatial positions held in WM. Human observers remembered 1 or 2 positions over a short delay while we recorded electroencephalography (EEG) data. Using a spatial encoding model, we reconstructed active stimulus-specific representations (channel-tuning functions [CTFs]) from the scalp distribution of alpha-band power. Consistent with past work, we found that the selectivity of spatial CTFs was lower when 2 items were stored than when 1 item was stored. Critically, data-driven simulations revealed that the selectivity of spatial representations in the two-item condition could not be explained by models that propose that only a single item can exist in an active state at once. Thus, our findings demonstrate that multiple items can be concurrently represented in an active neural state.

An electroencephalography-based study shows that working memory delay activity concurrently represents two items, posing a challenge for models which propose that only one item can be actively represented at a time.

Redefining Visual Working Memory: A Cognitive-Strategy, Brain-Region Approach

The ability to remember and manipulate visual information is pervasive and is associated with many cognitive abilities. Yet despite the importance of visual working memory (VWM), there is little consensus among researchers in the field as to which neural areas are necessary and sufficient and which models best describe its capacity. Here, we propose that an assumption that all people remember visual information in the same way has led to much contention and inconsistencies in the field. By accepting that there are multiple cognitive strategies and methods to perform a VWM task, we introduce an individual “precision” approach to the study of memory. We propose that VWM should be redefined, not by the type of stimuli used (e.g., visual) but rather by the specific mental processes (e.g., visual imagery, semantic, propositional, spatial) and the corresponding brain regions used to complete the mnemonic task. We further provide a short how-to guide for measuring different mnemonic strategies used for working memory.

Chunking as a rational strategy for lossy data compression in visual working memory

The nature of capacity limits for visual working memory has been the subject of an intense debate that has relied on models that assume items are encoded independently. Here we propose that instead, similar features are jointly encoded through a "chunking" process to optimize performance on visual working memory tasks. We show that such chunking can: (a) facilitate performance improvements for abstract capacity-limited systems, (b) be optimized through reinforcement, (c) be implemented by center-surround dynamics, and (d) increase effective storage capacity at the expense of recall precision. Human performance on a variant of a canonical working memory task demonstrated performance advantages, precision detriments, interitem dependencies, and trial-to-trial behavioral adjustments diagnostic of performance optimization through center-surround chunking. Models incorporating center-surround chunking provided a better quantitative description of human performance in our study as well as in a meta-analytic dataset, and apparent differences in working memory capacity across individuals were attributable to individual differences in the implementation of chunking. Our results reveal a normative rationale for center-surround connectivity in working memory circuitry, call for reevaluation of memory performance differences that have previously been attributed to differences in capacity, and support a more nuanced view of visual working memory capacity limitations: strategic tradeoff between storage capacity and memory precision through chunking contribute to flexible capacity limitations that include both discrete and continuous aspects. (PsycINFO Database Record

Executive Functions and Language: Their Differential Influence on Mono- vs. Multilingual Spelling in Primary School

We aimed at unveiling the role of executive functions (EFs) and language-related skills in spelling for mono- versus multilingual primary school children. We focused on EF and language-related skills, in particular lexicon size and phonological awareness (PA), because these factors were found to predict spelling in studies predominantly conducted with monolinguals, and because multilingualism can modulate these factors. There is evidence for (a) a bilingual advantage in EF due to constant high cognitive demands through language control, (b) a smaller mental lexicon in German and (c) possibly better PA. Multilinguals in Germany show on average poorer German language proficiency, what can influence performance on language-based tasks negatively. Thus, we included two spelling tasks to tease apart spelling based on lexical knowledge (i.e., word spelling) from spelling based on non-lexical strategies (i.e., non-word spelling). Our sample consisted of heterogeneous third graders from Germany: 69 monolinguals (age: M = 108 months) and 57 multilinguals (age: M = 111 months). On less language-dependent tasks (e.g., non-word spelling, PA, intelligence, short-term memory (STM) and three EF tasks testing switching, inhibition, and working memory) performance of both groups did not differ significantly. However, multilinguals performed significantly more poorly on tasks measuring German lexicon size and word spelling than monolinguals. Regression analyses revealed that for multilinguals, inhibition was related to spelling, whereas switching was the only EF component to influence word spelling in monolinguals and non-word spelling performance in both groups. By adding lexicon size and other language-related factors to the regression models, the influence of switching was reduced to insignificant effects, but inhibition remained significant for multilinguals. Language-related skills best predicted spelling and both language groups shared those variables: PA for word spelling, and STM for non-word spelling. Additionally, multilinguals' word spelling performance was also predicted by their German lexicon size, and non-word spelling performance by PA. This study offers an in-depth look at spelling acquisition at a certain point of literacy development. Mono- and multilinguals have the predominant factors for spelling in common, but probably due to superior language knowledge, monolinguals were already able to make use of EF during spelling. For multilinguals, German lexicon size was more important for spelling than EF. For multilinguals' spelling these functions might come into play only at a later stage.

Visuospatial short-term memory and dorsal visual gray matter volume

Visual short-term memory (VSTM) is an important cognitive capacity that varies across the healthy adult population and is affected in several neurodevelopmental disorders. It has been suggested that neuroanatomy places limits on this capacity through a map architecture that creates competition for cortical space. This suggestion has been supported by the finding that primary visual (V1) gray matter volume (GMV) is positively associated with VSTM capacity. However, evidence from neurodevelopmental disorders suggests that the dorsal visual stream is more broadly vulnerable and atypical volumes of other map-containing regions may therefore play a role. For example, Turner syndrome is associated with concomitantly reduced volume of the right intraparietal sulcus (IPS) and deficits in VSTM. As posterior IPS regions (IPS0-2) contain topographic maps, together this suggests that posterior IPS volumes may also associate with VSTM. In this study, we assessed VSTM using two tasks, as well as a composite score, and used voxel-based morphometry of T1-weighted magnetic resonance images to assess GMV in V1 and right IPS0-2 in 32 healthy young adults (16 female). For comparison with previous work, we also assessed associations between VSTM and voxel-wise GMV on a whole-brain basis. We found that total brain volume (TBV) significantly correlated with VSTM, and that correlations between VSTM and regional GMV were substantially reduced in strength when controlling for TBV. In our whole-brain analysis, we found that VSTM was associated with GMV of clusters centered around the right putamen and left Rolandic operculum, though only when TBV was not controlled for. Our results suggest that VSTM ability is unlikely to be accounted for by the volume of an individual cortical region, and may instead rely on distributed structural properties.

Binocular disparity can augment the capacity of vision without affecting subjective experience of depth

Binocular disparity results in a tangible subjective experience of three-dimensional world, but whether disparity also augments objective perceptual performance remains debated. We hypothesized that the improved coding of depth enabled by binocular disparity allows participants to individuate more objects at a glance as the objects can be more efficiently differentiated from each other and the background. We asked participants to enumerate objects in briefly presented naturalistic (Experiment 1) and artificial (Experiment 2) scenes in immersive virtual reality. This type of enumeration task yields well-documented capacity limits where up to 3-4 items can be enumerated rapidly and accurately, known as subitizing. Our results show that although binocular disparity did not yield a large general improvement in enumeration accuracy or reaction times, it improved participants' ability to process the items right after the limit of perceptual capacity. Binocular disparity also sped-up response times by 27 ms on average when artificial stimuli (cubes) were used. Interestingly, the influence of disparity on subjectively experienced depth revealed a clearly different pattern than the influence of disparity on objective performance. This suggests that the functional and subjective sides of stereopsis can be dissociated. Altogether our results suggest that binocular disparity may increase the number of items the visual system can simultaneously process. This may help animals to better resolve and track objects in complex, cluttered visual environments.

The Sources of Dual-task Costs in Multisensory Working Memory Tasks

We investigated the sources of dual-task costs arising in multisensory working memory (WM) tasks, where stimuli from different modalities have to be simultaneously maintained. Performance decrements relative to unimodal single-task baselines have been attributed to a modality-unspecific central WM store, but such costs could also reflect increased demands on central executive processes involved in dual-task coordination. To compare these hypotheses, we asked participants to maintain two, three, or four visual items. Unimodal trials, where only this visual task was performed, and bimodal trials, where a concurrent tactile WM task required the additional maintenance of two tactile items, were randomly intermixed. We measured the visual and tactile contralateral delay activity (CDA/tCDA components) as markers of WM maintenance in visual and somatosensory areas. There were reliable dual-task costs, as visual CDA components were reduced in size and visual WM accuracy was impaired on bimodal relative to unimodal trials. However, these costs did not depend on visual load, which caused identical CDA modulations in unimodal and bimodal trials, suggesting that memorizing tactile items did not reduce the number of visual items that could be maintained. Visual load did not also affect tCDA amplitudes. These findings indicate that bimodal dual-task costs do not result from a competition between multisensory items for shared storage capacity. Instead, these costs reflect generic limitations of executive control mechanisms that coordinate multiple cognitive processes in dual tasks. Our results support hierarchical models of WM, where distributed maintenance processes with modality-specific capacity limitations are controlled by a central executive mechanism.

Representation of affect in sensory cortex

Contemporary neuroscience suggests that perception is perhaps best understood as a dynamically iterative process that does not honor cleanly segregated "bottom-up" or "top-down" streams. We argue that there is substantial empirical support for the idea that affective influences infiltrate the earliest reaches of sensory processing and even that primitive internal affective dimensions (e.g., goodness-to-badness) are represented alongside physical dimensions of the external world.

A two-level hierarchical framework of visual short-term memory

Over the last couple of decades, a vast amount of research has been dedicated to understanding the nature and the architecture of visual short-term memory (VSTM), the mechanism by which currently relevant visual information is maintained. According to discrete-capacity models, VSTM is constrained by a limited number of discrete representations held simultaneously. In contrast, shared-resource models regard VSTM as limited in resources, which can be distributed flexibly between varying numbers of representations; and a new interference model posits that capacity is limited by interference among items. In this article, we begin by reviewing benchmark findings regarding the debate over VSTM limitations, focusing on whether VSTM storage is all-or-none and on whether object complexity affects capacity. After that, we put forward a hybrid framework of VSTM architecture, arguing that this system is composed of a two-level hierarchy of memory stores, each containing a different set of representations: (1) perceptual memory, a resourcelike level containing analog automatically formed representations of visual stimuli in varying degrees of activation, and (2) visual working memory, in which a subset of three to four items from perceptual memory are bound to conceptual representations and to their locations, thus conveying discrete (digital/symbolic) information which appears quantized. While perceptual memory has a large capacity and is relatively nonselective, visual working memory is restricted in the number of items that can be maintained simultaneously, and its content is regulated by a gating mechanism.

Dual Task Effects on Visual Attention Capacity in Normal Aging

Older adults show higher dual task performance decrements than younger adults. While this is assumed to be related to attentional capacity reductions, the precise affected functions are not specified. Such specification is, however, possible based on the "theory of visual attention" (TVA) which allows for modeling of distinct attentional capacity parameters. Furthermore, it is unclear whether older adults show qualitatively different attentional effects or whether they show the same effects as younger adults experience under more challenging conditions. By varying the complexity of the secondary task, it is possible to address this question. In our study, participants performed a verbal whole report of briefly presented letter arrays. TVA-based fitting of report performance delivered parameters of visual threshold t0, processing speed C, and visual short-term memory (VSTM) storage capacity K. Furthermore, participants performed a concurrent motor task consisting of continuous tapping of a (simple or complex) sequence. Both TVA and tapping tasks were performed under single and dual task conditions. Two groups of 30 younger adults each performed either the simple or complex tapping, and a group of 30 older adults performed the simple tapping condition. In older participants, VSTM storage capacity declined under dual task conditions. While no such effect was found in younger subjects performing the simple tapping sequence under dual task conditions, the younger group performing the complex tapping task under dual task conditions also showed a significant VSTM capacity reduction. Generally, no significant effect on other TVA parameters or on tapping accuracy was found. Comparable goodness-of-fit measures were obtained for the TVA modeling data in single and dual tasks, indicating that tasks were executed in a qualitatively similar, continuous manner, although quantitatively less efficiently under dual- compared to single-task conditions. Taken together, our results show that the age-specific effects of motor-cognitive dual task interference are reflected by a stronger decline of VSTM storage capacity. They support an interpretation of VSTM as central attentional capacity, which is shared across visual uptake and concurrent motor performance. Capacity limits are reached earlier, and already under lower motor task complexity, in older compared to younger adults.

Are stronger memories forgotten more slowly? No evidence that memory strength influences the rate of forgetting

Information stored in visual short-term memory is used ubiquitously in daily life; however, it is forgotten rapidly within seconds. When more items are to be remembered, they are forgotten faster, potentially suggesting that stronger memories are forgotten less rapidly. Here we tested this prediction with three experiments that assessed the influence of memory strength on the rate of forgetting of visual information without manipulating the number of items. Forgetting rate was assessed by comparing the accuracy of reports in a delayed-estimation task following relatively short and long retention intervals. In the first experiment, we compared the forgetting rate of items that were directly fixated, to items that were not. In Experiments 2 and 3 we manipulated memory strength by extending the exposure time of one item in the memory array. As expected, direct fixation and longer exposure led to better accuracy of reports, reflecting stronger memory. However, in all three experiments, we did not find evidence that increased memory strength moderated the forgetting rate.

Slot-like capacity and resource-like coding in a neural model of multiple-item working memory

For the past decade, research on the storage limitations of working memory has been dominated by two fundamentally different hypotheses. On the one hand, the contents of working memory may be stored in a limited number of "slots," each with a fixed resolution. On the other hand, any number of items may be stored but with decreasing resolution. These two hypotheses have been invaluable in characterizing the computational structure of working memory, but neither provides a complete account of the available experimental data or speaks to the neural basis of the limitations it characterizes. To address these shortcomings, we simulated a multiple-item working memory task with a cortical network model, the cellular resolution of which allowed us to quantify the coding fidelity of memoranda as a function of memory load, as measured by the discriminability, regularity, and reliability of simulated neural spiking. Our simulations account for a wealth of neural and behavioral data from human and nonhuman primate studies, and they demonstrate that feedback inhibition lowers both capacity and coding fidelity. Because the strength of inhibition scales with the number of items stored by the network, increasing this number progressively lowers fidelity until capacity is reached. Crucially, the model makes specific, testable predictions for neural activity on multiple-item working memory tasks. NEW & NOTEWORTHY Working memory is the ability to keep information in mind and is fundamental to cognition. It is actively debated whether the storage limitations of working memory reflect a small number of storage units (slots) or a decrease in coding resolution as a limited resource is allocated to more items. In a cortical model, we found that slot-like capacity and resource-like neural coding resulted from the same mechanism, offering an integrated explanation for storage limitations.

The contributions of visual and central attention to visual working memory

We investigated the role of two kinds of attention-visual and central attention-for the maintenance of visual representations in working memory (WM). In Experiment 1 we directed attention to individual items in WM by presenting cues during the retention interval of a continuous delayed-estimation task, and instructing participants to think of the cued items. Attending to items improved recall commensurate with the frequency with which items were attended (0, 1, or 2 times). Experiments 1 and 3 further tested which kind of attention-visual or central-was involved in WM maintenance. We assessed the dual-task costs of two types of distractor tasks, one tapping sustained visual attention and one tapping central attention. Only the central attention task yielded substantial dual-task costs, implying that central attention substantially contributes to maintenance of visual information in WM. Experiment 2 confirmed that the visual-attention distractor task was demanding enough to disrupt performance in a task relying on visual attention. We combined the visual-attention and the central-attention distractor tasks with a multiple object tracking (MOT) task. Distracting visual attention, but not central attention, impaired MOT performance. Jointly, the three experiments provide a double dissociation between visual and central attention, and between visual WM and visual object tracking: Whereas tracking multiple targets across the visual filed depends on visual attention, visual WM depends mostly on central attention.

Independent Attention Mechanisms Control the Activation of Tactile and Visual Working Memory Representations

Working memory (WM) is limited in capacity, but it is controversial whether these capacity limitations are domain-general or are generated independently within separate modality-specific memory systems. These alternative accounts were tested in bimodal visual/tactile WM tasks. In Experiment 1, participants memorized the locations of simultaneously presented task-relevant visual and tactile stimuli. Visual and tactile WM load was manipulated independently (one, two, or three items per modality), and one modality was unpredictably tested after each trial. To track the activation of visual and tactile WM representations during the retention interval, the visual contralateral delay activity (CDA) and tactile CDA (tCDA) were measured over visual and somatosensory cortex, respectively. CDA and tCDA amplitudes were selectively affected by WM load in the corresponding (tactile or visual) modality. The CDA parametrically increased when visual load increased from one to two and to three items. The tCDA was enhanced when tactile load increased from one to two items and showed no further enhancement for three tactile items. Critically, these load effects were strictly modality-specific, as substantiated by Bayesian statistics. Increasing tactile load did not affect the visual CDA, and increasing visual load did not modulate the tCDA. Task performance at memory test was also unaffected by WM load in the other (untested) modality. This was confirmed in a second behavioral experiment where tactile and visual loads were either two or four items, unimodal baseline conditions were included, and participants performed a color change detection task in the visual modality. These results show that WM capacity is not limited by a domain-general mechanism that operates across sensory modalities. They suggest instead that WM storage is mediated by distributed modality-specific control mechanisms that are activated independently and in parallel during multisensory WM.

Electrophysiological evidence for temporal dynamics associated with attentional processing in the zoom lens paradigm

Background

Visuospatial processing requires wide distribution or narrow focusing of attention to certain regions in space. This mechanism is described by the zoom lens model and predicts an inverse correlation between the efficiency of processing and the size of the attentional scope. Little is known, however, about the exact timing of the effects of attentional scaling on visual searching and whether or not additional processing phases are involved in this process.

Method

Electroencephalographic recordings were made while participants performed a visual search task under different attentional scaling conditions. Two concentric circles of different sizes, presented to the participants at the center of a screen modulated the attentional scopes, and search arrays were distributed in the space areas indicated by these concentric circles. To ensure consistent eccentricity of the search arrays across different conditions, we limited our studies to the neural responses evoked by the search arrays distributed in the overlapping region of different attentional scopes.

Results

Consistent with the prediction of the zoom lens model, our behavioral data showed that reaction times for target discrimination of search arrays decreased and the associated error rates also significantly decreased, with narrowing the attentional scope. Results of the event-related potential analysis showed that the target-elicited amplitude of lateral occipital N1, rather than posterior P1, which reflects the earliest visuospatial attentional processing, was sensitive to changes in the scaling of visuospatial attention, indicating that the modulation of the effect of changes in the spatial scale of attention on visual processing occurred after the delay period of P1. The N1 generator exhibited higher activity as the attentional scope narrowed, reflecting more intensive processing resources within the attentional focus. In contrast to N1, the amplitude of N2pc increased with the expansion of the attentional focus, suggesting that observers might further redistribute attentional resources according to the increased task difficulty.

Conclusion

These findings provide electrophysiological evidence that the neural activity of the N1 generator is the earliest marker of the zoom lens effect of visual spatial attention. Furthermore, evidence from N2pc shows that there is also a redistribution of attentional resources after the action of the zoom lens mechanism, which allows for better perform of the search task in the context of low attentional resolution. On the basis of the timing of P1, N1, and N2pc, our findings provide compelling evidence that visuospatial attention processing in the zoom lens paradigm involves multi-stage dynamic processing.

The Effects of Similarity on High-Level Visual Working Memory Processing

Similarity has been observed to have opposite effects on visual working memory (VWM) for complex images. How can these discrepant results be reconciled? To answer this question, we used a change-detection paradigm to test visual working memory performance for multiple real-world objects. We found that working memory for moderate similarity items was worse than that for either high or low similarity items. This pattern was unaffected by manipulations of stimulus type (faces vs. scenes), encoding duration (limited vs. self-paced), and presentation format (simultaneous vs. sequential). We also found that the similarity effects differed in strength in different categories (scenes vs. faces). These results suggest that complex real-world objects are represented using a centre-surround inhibition organization. These results support the category-specific cortical resource theory and further suggest that centre-surround inhibition organization may differ by category.

Transformation from independent to integrative coding of multi-object arrangements in human visual cortex

To optimize processing, the human visual system utilizes regularities present in naturalistic visual input. One of these regularities is the relative position of objects in a scene (e.g., a sofa in front of a television), with behavioral research showing that regularly positioned objects are easier to perceive and to remember. Here we use fMRI to test how positional regularities are encoded in the visual system. Participants viewed pairs of objects that formed minimalistic two-object scenes (e.g., a "living room" consisting of a sofa and television) presented in their regularly experienced spatial arrangement or in an irregular arrangement (with interchanged positions). Additionally, single objects were presented centrally and in isolation. Multi-voxel activity patterns evoked by the object pairs were modeled as the average of the response patterns evoked by the two single objects forming the pair. In two experiments, this approximation in object-selective cortex was significantly less accurate for the regularly than the irregularly positioned pairs, indicating integration of individual object representations. More detailed analysis revealed a transition from independent to integrative coding along the posterior-anterior axis of the visual cortex, with the independent component (but not the integrative component) being almost perfectly predicted by object selectivity across the visual hierarchy. These results reveal a transitional stage between individual object and multi-object coding in visual cortex, providing a possible neural correlate of efficient processing of regularly positioned objects in natural scenes.

Attention and the sense of agency: A review and some thoughts on the matter

Much is now known about the sense of agency and how it is produced. What is lacking, though, is an understanding of how it relates to other cognitive domains and operations. Here, the relationship between the sense of agency and attention is explored. A review of the literature suggests that attention is involved in the sense of agency in (at least) two key ways. First, agency processing is dependent on attentional resources. Second, attentional enhancement is necessary for extraneous information to have an influence over the sense of agency. These are discussed and some suggestions for future research are offered.