Limits to the usability of iconic memory

Serial dependence in facial identity perception and visual working memory

Serial dependence (SD) refers to the effect in which a person's current perceptual judgment is attracted toward recent stimulus history. Perceptual and memory processes, as well as response and decisional biases, are thought to contribute to SD effects. The current study examined the processing stages of SD facial identity effects in the context of task-related decision processes and how such effects may differ from visual working memory (VWM) interactions. In two experiments, participants were shown a series of two sequentially presented face images. In Experiment 1, the two faces were separated by an interstimulus interval (ISI) of 1, 3, 6, or 10 s, and participants were instructed to reproduce the second face after a varying response delay of 0, 1, 3, 6, or 10 s. Results showed that SD effects occurred most consistently at ISI of 1 s and response delays of 1 and 6 s consistent with early and late stages of processing. In Experiment 2, the ISI was held constant at 1 s, and to separate SD from VWM interactions participants were post-cued to reproduce either the first or the second face. When the second face was the target, SD effects again occurred at response delays of 1 and 6 s, but not when the first face was the target. Together, the results demonstrates that SD facial identity effects occur independently of task-related processes in a distinct temporal fashion and suggest that SD and VWM interactions may rely on separate underlying mechanisms.

mixtur: An R package for designing, analysing, and modelling continuous report visual short-term memory studies

Visual short-term memory (vSTM) is often measured via continuous-report tasks whereby participants are presented with stimuli that vary along a continuous dimension (e.g., colour) with the goal of memorising the stimulus features. At test, participants are probed to recall the feature value of one of the memoranda in a continuous manner (e.g., by clicking on a colour wheel). The angular deviation between the participant response and the true feature value provides an estimate of recall precision. Two prominent models of performance on such tasks are the two- and three-component mixture models (Bays et al., Journal of Vision, 9(10), Article 7, 2009; Zhang and Luck, Nature, 453(7192), 233–235, 2008). Both models decompose participant responses into probabilistic mixtures of: (1) responses to the true target value based on a noisy memory representation; (2) random guessing when memory fails. In addition, the three-component model proposes (3) responses to a non-target feature value (i.e., binding errors). Here we report the development of mixtur, an open-source package written for the statistical programming language R that facilitates the fitting of the two- and three-component mixture models to continuous report data. We also conduct simulations to develop recommendations for researchers on trial numbers, set sizes, and memoranda similarity, as well as parameter recovery and model recovery. In the Discussion, we discuss how mixtur can be used to fit the slots and the slots-plus-averaging models, as well as how mixtur can be extended to fit explanatory models of visual short-term memory. It is our hope that mixtur will lower the barrier of entry for utilising mixture modelling.

A neuronal basis of iconic memory in macaque primary visual cortex

After a briefly presented visual stimulus disappears, observers retain a detailed representation of this stimulus for a short period of time. This sensory storage is called iconic memory. We measured iconic memory in the perception of monkeys and its neuronal correlates in the primary visual cortex (area V1). We determined how many milliseconds extra viewing time iconic memory is worth and how it decays by varying the duration of a brief stimulus and the timing of a mask. The V1 activity that persists after the disappearance of a stimulus predicted accuracy, with a time course resembling the worth and decay of iconic memory. Finally, we examined how iconic memory interacts with attention. A cue presented after the stimulus disappears boosts attentional influences pertaining to a relevant part of the stimulus but only if it appears before iconic memory decayed. Our results relate iconic memory to neuronal activity in early visual cortex.

A sensory memory to preserve visual representations across eye movements

Saccadic eye movements (saccades) disrupt the continuous flow of visual information, yet our perception of the visual world remains uninterrupted. Here we assess the representation of the visual scene across saccades from single-trial spike trains of extrastriate visual areas, using a combined electrophysiology and statistical modeling approach. Using a model-based decoder we generate a high temporal resolution readout of visual information, and identify the specific changes in neurons' spatiotemporal sensitivity that underly an integrated perisaccadic representation of visual space. Our results show that by maintaining a memory of the visual scene, extrastriate neurons produce an uninterrupted representation of the visual world. Extrastriate neurons exhibit a late response enhancement close to the time of saccade onset, which preserves the latest pre-saccadic information until the post-saccadic flow of retinal information resumes. These results show how our brain exploits available information to maintain a representation of the scene while visual inputs are disrupted.

The effects of Botulinum toxin on the detection of gradual changes in facial emotion

When we feel sad or depressed, our face invariably “drops”. Conversely, when we try to cheer someone up, we might tell them “keep your smile up”, so presupposing that modifying the configuration of their facial muscles will enhance their mood. A crucial assumption that underpins this hypothesis is that mental states are shaped by information originating from the peripheral neuromotor system — a view operationalised as the Facial Feedback Hypothesis. We used botulinum toxin (BoNT-A) injected over the frown area to temporarily paralyse muscles necessary to express anger. Using a pre-post treatment design, we presented participants with gradually changing videos of a face morphing from neutral to full-blown expressions of either anger or happiness and asked them to press a button as soon as they had detected any change in the display. Results indicate that while all participants (control and BoNT-A) improved their reaction times from pre-test to post-test, the BoNT-A group did not when detecting anger in the post-test. We surmise that frown paralysis disadvantaged participants in their ability to improve the detection of anger. Our finding suggests that facial feedback causally affects perceptual awareness of changes in emotion, as well as people’s ability to use perceptual information to learn.

Using the flicker task to estimate visual working memory storage capacity

Studies of visual working memory (VWM) typically have used a "one-shot" change detection task to arrive at a capacity estimate of three to four objects, with additional limits imposed by the precision of the information needed for each object. Unlike the one-shot task, the flicker change detection task permits measurement of VWM capacity over time and with larger numbers of objects present in the scene, but it has rarely been used to assess the capacity of VWM. We used the flicker task to examine (a) whether capacity is close to the typical three to four items when using subtly different stimuli; (b) which dependent measure provides the most meaningful estimate of the capacity of VWM in the flicker task (response time or number of changes viewed); (c) whether capacity remains fixed at three to four items for displays containing many more objects; and (d) how VWM operates over time, with repeated opportunities to encode, retain, and compare elements in a display. Four experiments using grids of simple items varying only in luminance or color revealed a range for VWM capacity limits that was largely impervious to changes in display duration, interstimulus intervals, and array size. This estimate of VWM capacity was correlated with an estimate from the more typical one-shot task, further validating the flicker task as a tool for measuring the capacity of VWM.

Sensory Memory Is Allocated Exclusively to the Current Event-Segment

The Atkinson-Shiffrin modal model forms the foundation of our understanding of human memory. It consists of three stores (Sensory Memory (SM), also called iconic memory, Short-Term Memory (STM), and Long-Term Memory (LTM)), each tuned to a different time-scale. Since its inception, the STM and LTM components of the modal model have undergone significant modifications, while SM has remained largely unchanged, representing a large capacity system funneling information into STM. In the laboratory, visual memory is usually tested by presenting a brief static stimulus and, after a delay, asking observers to report some aspect of the stimulus. However, under ecological viewing conditions, our visual system receives a continuous stream of inputs, which is segmented into distinct spatio-temporal segments, called events. Events are further segmented into event-segments. Here we show that SM is not an unspecific general funnel to STM but is allocated exclusively to the current event-segment. We used a Multiple-Object Tracking (MOT) paradigm in which observers were presented with disks moving in different directions, along bi-linear trajectories, i.e., linear trajectories, with a single deviation in direction at the mid-point of each trajectory. The synchronized deviation of all of the trajectories produced an event stimulus consisting of two event-segments. Observers reported the pre-deviation or the post-deviation directions of the trajectories. By analyzing observers' responses in partial- and full-report conditions, we investigated the involvement of SM for the two event-segments. The hallmarks of SM hold only for the current event segment. As the large capacity SM stores only items involved in the current event-segment, the need for event-tagging in SM is eliminated, speeding up processing in active vision. By characterizing how memory systems are interfaced with ecological events, this new model extends the Atkinson-Shiffrin model by specifying how events are stored in the first stage of multi-store memory systems.

Visual percepts modify iconic memory in humans

Our visual system briefly retains a trace of a stimulus after it disappears. This phenomenon is known as iconic memory and its contents are thought to be temporally integrated with subsequent visual inputs to produce a single composite representation. However, there is little consensus on the temporal integration between iconic memory and subsequent visual inputs. Here, we show that iconic memory revises its contents depending upon the configuration of the newly produced single representation with particular temporal characteristics. The Poggendorff illusion, in which two collinear line segments are perceived as non-collinear by an intervening rectangle, was observed when the rectangle was presented during a period spanning from 50 ms before to 200 ms after the presentation of the line segments. The illusion was most prominent when the rectangle was presented approximately 100 to 150 ms after the line segments. Furthermore, the illusion was observed at the center of a moving object, but only when the line segments were presented before the rectangle. These results indicate that the contents of iconic memory are susceptible to the modulatory influence of subsequent visual inputs before being translated into conscious perception in a time-locked manner both in retinotopic and non-retinotopic, object-centered frames of reference.

To Have Seen or Not to Have Seen: A Look at Rensink, O'Regan, and Clark (1997)

Rensink, O'Regan, and Clark drew attention to the phenomenon of change blindness, in which even large changes can be difficult to notice if made during the appearance of motion transients elsewhere in the image. This article provides a sketch of the events that inspired that article as well as its subsequent impact on psychological science and on society at large.

It is magic! How impossible solutions prevent the discovery of obvious ones?

When confronted with an insight problem, some factors limit our capacity to discover the optimal solution. Previous research on problem solving has shown that the first idea that comes to participants’ minds can inhibit them from finding better alternative solutions. We used a magic trick to demonstrate that this mind fixing effect is more general than previously thought: a solution that participants knew to be incorrect and impossible inhibited the discovery of an easy alternative. We show that a simple exposure to an obvious false solution (e.g., the magician hides the card in the palm of his hand to secretly transfer it to his back pocket) can inhibit participants from finding the real secret of the trick (e.g., he used a duplicate card), even if the magician proves that this false solution is impossible (e.g., he shows his hand is empty). We discuss the psychological processes underlying this robust fixing effect.

Expectation creates something out of nothing: The role of attention in iconic memory reconsidered

Conscious experience is modulated by attention and expectation, yet is believed to be independent of attention. The experiments on iconic memory (IM) are usually taken as support for this claim. However, a recent experiment demonstrated that when attention is diverted away from the IM letter display subjects fail to see the absence of IM letters. Here we contribute to the ongoing debate by overcoming experimental shortcomings of this previous experiment, by measuring subjective visibility and by testing the effect of the post-cue. We were able to replicate these earlier findings and extend them by demonstrating that subjects who do not realize the absence of letters perceive illusory letters. This result means that there is still phenomenal consciousness, even when attention is diverted. Expectation creates illusory content that overwrites valid IM content. Taken together these findings suggest that the present experimental paradigm is not appropriate to make claims about IM content.

VQone MATLAB toolbox: A graphical experiment builder for image and video quality evaluations: VQone MATLAB toolbox

This article presents VQone, a graphical experiment builder, written as a MATLAB toolbox, developed for image and video quality ratings. VQone contains the main elements needed for the subjective image and video quality rating process. This includes building and conducting experiments and data analysis. All functions can be controlled through graphical user interfaces. The experiment builder includes many standardized image and video quality rating methods. Moreover, it enables the creation of new methods or modified versions from standard methods. VQone is distributed free of charge under the terms of the GNU general public license and allows code modifications to be made so that the program's functions can be adjusted according to a user's requirements. VQone is available for download from the project page (http://www.helsinki.fi/psychology/groups/visualcognition/).