The role of feedback in visual masking and visual processing

Metacontrast masking is ineffective in the first 6 months of life

Metacontrast masking is one of the most widely studied types of visual masking, in which a visual stimulus is rendered invisible by a subsequent mask that does not spatially overlap with the target. Metacontrast has been used for many decades as a tool to study visual processing and conscious perception in adults. However, there are so far no infant studies on metacontrast and it remains unknown even whether it occurs in infants. The present study examined metacontrast masking in 3- to 8-month-old infants (N = 168) using a habituation paradigm. We found that metacontrast is ineffective for infants under 7 months and that younger infants can perceive a masked stimulus that older infants cannot. Our results suggest that metacontrast is distinct from other simple types of masking that occur in early infancy, and would be consistent with the idea that metacontrast results from the disruption of recurrent processing.

Backward masking implicates cortico-cortical recurrent processes in convex figure context effects and cortico-thalamic recurrent processes in resolving figure-ground ambiguity

Introduction

Previous experiments purportedly showed that image-based factors like convexity were sufficient for figure assignment. Recently, however, we found that the probability of perceiving a figure on the convex side of a central border was only slightly higher than chance for two-region displays and increased with the number of display regions; this increase was observed only when the concave regions were homogeneously colored. These convex figure context effects (CEs) revealed that figure assignment in these classic displays entails more than a response to local convexity. A Bayesian observer replicated the convex figure CEs using both a convexity object prior and a new, homogeneous background prior and made the novel prediction that the classic displays in which both the convex and concave regions were homogeneous were ambiguous during perceptual organization.

Methods

Here, we report three experiments investigating the proposed ambiguity and examining how the convex figure CEs unfold over time with an emphasis on whether they entail recurrent processing. Displays were shown for 100 ms followed by pattern masks after ISIs of 0, 50, or 100 ms. The masking conditions were designed to add noise to recurrent processing and therefore to delay the outcome of processes in which they play a role. In Exp. 1, participants viewed two- and eight-region displays with homogeneous convex regions (homo-convex displays; the putatively ambiguous displays). In Exp. 2, participants viewed putatively unambiguous hetero-convex displays. In Exp. 3, displays and masks were presented to different eyes, thereby delaying mask interference in the thalamus for up to 100 ms.

Results and discussion

The results of Exps. 1 and 2 are consistent with the interpretation that recurrent processing is involved in generating the convex figure CEs and resolving the ambiguity of homo-convex displays. The results of Exp. 3 suggested that corticofugal recurrent processing is involved in resolving the ambiguity of homo-convex displays and that cortico-cortical recurrent processes play a role in generating convex figure CEs and these two types of recurrent processes operate in parallel. Our results add to evidence that perceptual organization evolves dynamically and reveal that stimuli that seem unambiguous can be ambiguous during perceptual organization.

Visual information processing through the interplay between fine and coarse signal pathways

Object recognition is often viewed as a feedforward, bottom-up process in machine learning, but in real neural systems, object recognition is a complicated process which involves the interplay between two signal pathways. One is the parvocellular pathway (P-pathway), which is slow and extracts fine features of objects; the other is the magnocellular pathway (M-pathway), which is fast and extracts coarse features of objects. It has been suggested that the interplay between the two pathways endows the neural system with the capacity of processing visual information rapidly, adaptively, and robustly. However, the underlying computational mechanism remains largely unknown. In this study, we build a two-pathway model to elucidate the computational properties associated with the interactions between two visual pathways. Specifically, we model two visual pathways using two convolution neural networks: one mimics the P-pathway, referred to as FineNet, which is deep, has small-size kernels, and receives detailed visual inputs; the other mimics the M-pathway, referred to as CoarseNet, which is shallow, has large-size kernels, and receives blurred visual inputs. We show that CoarseNet can learn from FineNet through imitation to improve its performance, FineNet can benefit from the feedback of CoarseNet to improve its robustness to noise; and the two pathways interact with each other to achieve rough-to-fine information processing. Using visual backward masking as an example, we further demonstrate that our model can explain visual cognitive behaviors that involve the interplay between two pathways. We hope that this study gives us insight into understanding the interaction principles between two visual pathways.

Does V1 response suppression initiate binocular rivalry?

During binocular rivalry (BR) only one eye's view is perceived. Neural underpinnings of BR are debated. Recent studies suggest that primary visual cortex (V1) initiates BR. One trigger might be response suppression across most V1 neurons at the onset of BR. Here, we utilize a variant of BR called binocular rivalry flash suppression (BRFS) to test this hypothesis. BRFS is identical to BR, except stimuli are shown with a ∼1s delay. If V1 response suppression was required to initiate BR, it should occur during BRFS as well. To test this, we compared V1 spiking in two macaques observing BRFS. We found that BRFS resulted in response facilitation rather than response suppression across V1 neurons. However, BRFS still reduces responses in a subset of V1 neurons due to the adaptive effects of asynchronous stimulus presentation. We argue that this selective response suppression could serve as an alternate initiator of BR.

Perceptual bias contextualized in visually ambiguous stimuli

The visual appearance of an object is a function of stimulus properties as well as perceptual biases imposed by the observer. The context-specific trade-off between both can be measured accurately in a perceptual judgment task, involving grouping by proximity in ambiguous dot lattices. Such grouping depends lawfully on a stimulus parameter of the dot lattices known as their aspect ratio (AR), whose effect is modulated by a perceptual bias representing the preference for a cardinal orientation. In two experiments, we investigated how preceding context can lead to bias modulation, either in a top-down fashion via visual working memory (VWM) or bottom-up via sensory priming. In Experiment 1, we embedded the perceptual judgment task in a change detection paradigm and studied how the factors of VWM load (complexity of the memory array) and content (congruency in orientation to the ensuing dot lattice) affect the prominence of perceptual bias. A robust vertical orientation bias was observed, which was increased by VWM load and modulated by congruent VWM content. In Experiment 2, dot lattices were preceded by oriented primes. Here, primes regardless of orientation elicited a vertical orientation bias in dot lattices compared to a neutral baseline. Taken together, the two experiments demonstrate that top-down context (VWM load and content) effectively controls orientation bias modulation, while bottom-up context (i.e., priming) merely acts as an undifferentiated trigger to perceptual bias. These findings characterize the temporal context sensitivity of Gestalt perception, shed light on the processes responsible for different perceptual outcomes of ambiguous stimuli, and identify some of the mechanisms controlling perceptual bias.

Difficulty with the preceding visual search affects brain activity in the following resting period

It has been well-documented that brain regions related to a task are activated during the task performance. We investigated whether brain activity and functional connectivity during the rest period are affected by the preceding task. Participants performed visual search tasks with three search conditions, which were followed by a rest period. During the rest period, participants were asked to look at the display that did not show any visual stimuli. In the result, brain activity in occipital and superior parietal regions would be deactivated by the preceding task during the rest period after visual search tasks. However, the activity of the inferior frontal gyrus during the rest period, which is also part of the attention network, was not affected by the brain activity during the preceding visual search task. We proposed a new model for explaining how the cognitive demands of the preceding visual search task regulate the attention network during the rest period after the task. In this model, the cognitive demand changes with task difficulty, which affects the brain activity even after removing the visual search task in the rest phase.

The central-peripheral dichotomy and metacontrast masking

According to the central-peripheral dichotomy (CPD), feedback from higher to lower cortical areas along the visual pathway to aid recognition is weaker in the more peripheral visual field. Metacontrast masking is predominantly a reduced visibility of a brief target by a brief and spatially adjacent mask when the mask succeeds rather than precedes or coincides with the target. If this masking works mainly by interfering with the feedback mechanisms for target recognition, then, by the CPD, this masking should be weaker at more peripheral visual locations. We extended the metacontrast masking at fovea by Enns and Di Lollo to visual field eccentricities 1∘, 3∘, and 9∘. Relative to the target’s onset, the mask appeared at a stimulus onset asynchrony (SOA) of −50, 0, 50, 92, or 142 milliseconds (ms). Enlarged stimuli were used for larger eccentricities to equalize target discrimination performance across eccentricities as best as possible for zero SOA and when SOA was too long for substantial masking. At each eccentricity, the masking was weakest at 0 or −50 ms SOA, strongest at 50 ms SOA, and weakened with larger (positive) SOAs. Consistent with the CPD, larger eccentricities presented weaker maskings at all nonzero, and particularly the positive, SOAs.

Deep neural network models of sound localization reveal how perception is adapted to real-world environments

Mammals localize sounds using information from their two ears. Localization in real-world conditions is challenging, as echoes provide erroneous information, and noises mask parts of target sounds. To better understand real-world localization we equipped a deep neural network with human ears and trained it to localize sounds in a virtual environment. The resulting model localized accurately in realistic conditions with noise and reverberation. In simulated experiments, the model exhibited many features of human spatial hearing: sensitivity to monaural spectral cues and interaural time and level differences, integration across frequency, biases for sound onsets, and limits on localization of concurrent sources. But when trained in unnatural environments without either reverberation, noise, or natural sounds, these performance characteristics deviated from those of humans. The results show how biological hearing is adapted to the challenges of real-world environments and illustrate how artificial neural networks can reveal the real-world constraints that shape perception.

Perception of invisible masked objects in early infancy

Recurrent loops in the visual cortex play a critical role in visual perception, which is likely not mediated by purely feed-forward pathways. However, the development of recurrent loops is poorly understood. The role of recurrent processing has been studied using visual backward masking, a perceptual phenomenon in which a visual stimulus is rendered invisible by a following mask, possibly because of the disruption of recurrent processing. Anatomical studies have reported that recurrent pathways are immature in early infancy. This raises the possibility that younger infants process visual information mainly in a feed-forward manner, and thus, they might be able to perceive visual stimuli that adults cannot see because of backward masking. Here, we show that infants under 7 mo of age are immune to visual backward masking and that masked stimuli remain visible to younger infants while older infants cannot perceive them. These results suggest that recurrent processing is immature in infants under 7 mo and that they are able to perceive objects even without recurrent processing. Our findings indicate that the algorithm for visual perception drastically changes in the second half of the first year of life.

The influence of facial expression at perceptual threshold on electrodermal activity and social comfort distance

Interpersonal distance, an essential component of social interaction, is modulated by the emotion conveyed by others and associated physiological response. However, in modern societies with overcrowded and hyperstimulating environments, we can only surreptitiously glimpse the faces of others in order to quickly make behavioral adjustments. How this impacts social interactions is not yet well understood. In the present study, we investigated this issue by testing whether facial expressions that are difficult to identify modify the physiological response (Electrodermal Activity, EDA) and subsequent judgment of interpersonal comfort distance. We recorded participants' EDA while they provided comfort judgments to interpersonal distances with a Point-Light Walker (PLW). The PLW, with an emotionally neutral gait, moved toward and crossed participants at various distances after the latter were exposed to a negative (anger), positive (happiness) or neutral facial expression presented at the perceptual threshold. Bayesian analyses of the data revealed an increase versus decrease of interpersonal comfort distance with the PLW depending on the negative versus positive emotional valence of the facial expression. They also showed an increase in EDA when the approaching PLW violated interpersonal comfort distance after participants were exposed to an angry facial expression. These effects correlated with the subjective assessment of the arousal of facial expressions. Thus, previous exposure to barely visible facial expressions can alter the representation of social comfort space and the physiological response associated with a violation of interpersonal comfort distances, depending on the valence and arousal of the emotional social stimuli.

Beyond the feedforward sweep: feedback computations in the visual cortex

Visual perception involves the rapid formation of a coarse image representation at the onset of visual processing, which is iteratively refined by late computational processes. These early versus late time windows approximately map onto feedforward and feedback processes, respectively. State-of-the-art convolutional neural networks, the main engine behind recent machine vision successes, are feedforward architectures. Their successes and limitations provide critical information regarding which visual tasks can be solved by purely feedforward processes and which require feedback mechanisms. We provide an overview of recent work in cognitive neuroscience and machine vision that highlights the possible role of feedback processes for both visual recognition and beyond. We conclude by discussing important open questions for future research.

Late disruption of central visual field disrupts peripheral perception of form and color

Evidence from neuroimaging and brain stimulation studies suggest that visual information about objects in the periphery is fed back to foveal retinotopic cortex in a separate representation that is essential for peripheral perception. The characteristics of this phenomenon have important theoretical implications for the role fovea-specific feedback might play in perception. In this work, we employed a recently developed behavioral paradigm to explore whether late disruption to central visual space impaired perception of color. In the first experiment, participants performed a shape discrimination task on colored novel objects in the periphery while fixating centrally. Consistent with the results from previous work, a visual distractor presented at fixation ~100ms after presentation of the peripheral stimuli impaired sensitivity to differences in peripheral shapes more than a visual distractor presented at other stimulus onset asynchronies. In a second experiment, participants performed a color discrimination task on the same colored objects. In a third experiment, we further tested for this foveal distractor effect with stimuli restricted to a low-level feature by using homogenous color patches. These two latter experiments resulted in a similar pattern of behavior: a central distractor presented at the critical stimulus onset asynchrony impaired sensitivity to peripheral color differences, but, importantly, the magnitude of the effect was stronger when peripheral objects contained complex shape information. These results show a behavioral effect consistent with disrupting feedback to the fovea, in line with the foveal feedback suggested by previous neuroimaging studies.

Influence of Artificially Generated Interocular Blur Difference on Fusion Stability Under Vergence Stress

The stability of fusion was evaluated by its breakage when interocular blur differences were presented under vergence demand to healthy subjects. We presumed that these blur differences cause suppression of the more blurred image (interocular blur suppression, IOBS), disrupt binocular fusion and suppressed eye leaves its forced vergent position. During dichoptic presentation of static grayscale images of natural scenes, the luminance contrast (mode B) or higher-spatial frequency content (mode C) or luminance contrast plus higher-spatial frequency content (mode A) were stepwise reduced in the image presented to the non-dominant eye. We studied the effect of these types of blur on fusion stability at various levels of the vergence demand. During the divergence demand, the fusion was disrupted with approximately half blur than during convergence. Various modes of blur influenced fusion differently. The mode C (isolated reduction of higher-spatial frequency content) violated fusion under the lowest vergence demand significantly more than either isolated or combined reduction of luminance contrast (mode B and A). According to our results, the image´s details (i.e. higher-spatial frequency content) protects binocular fusion from disruption by the lowest vergence demand.

Saccade-based termination responses in macaque V1 and visual perception

Neurons in visual areas of the brain are generally characterized by the increase in firing rate that occurs when a stimulus is flashed on in the receptive field (RF). However, neurons also increase their firing rate when a stimulus is turned off. These "termination responses" or "after-discharges" that occur with flashed stimuli have been observed in area V1 and they may be important for vision as stimulus terminations have been shown to influence visual perception. The goal of the present study was to determine the strength of termination responses in the more natural situation in which eye movements move a stimulus out of an RF. We find that termination responses do occur in macaque V1 when termination results from a saccadic eye movement, but they are smaller in amplitude compared to flashed-off stimuli. Furthermore, there are termination responses even in the absence of visual stimulation. These findings demonstrate that termination responses are a component of naturalistic vision. They appear to be based on both visual and nonvisual signals in visual cortex. We speculate that the weakening of termination responses might be a neural correlate of saccadic suppression, the loss of perceptual sensitivity around the time of saccades.

Beyond core object recognition: Recurrent processes account for object recognition under occlusion

Core object recognition, the ability to rapidly recognize objects despite variations in their appearance, is largely solved through the feedforward processing of visual information. Deep neural networks are shown to achieve human-level performance in these tasks, and explain the primate brain representation. On the other hand, object recognition under more challenging conditions (i.e. beyond the core recognition problem) is less characterized. One such example is object recognition under occlusion. It is unclear to what extent feedforward and recurrent processes contribute in object recognition under occlusion. Furthermore, we do not know whether the conventional deep neural networks, such as AlexNet, which were shown to be successful in solving core object recognition, can perform similarly well in problems that go beyond the core recognition. Here, we characterize neural dynamics of object recognition under occlusion, using magnetoencephalography (MEG), while participants were presented with images of objects with various levels of occlusion. We provide evidence from multivariate analysis of MEG data, behavioral data, and computational modelling, demonstrating an essential role for recurrent processes in object recognition under occlusion. Furthermore, the computational model with local recurrent connections, used here, suggests a mechanistic explanation of how the human brain might be solving this problem.

In recent years, deep-learning-based computer vision algorithms have been able to achieve human-level performance in several object recognition tasks. This has also contributed in our understanding of how our brain may be solving these recognition tasks. However, object recognition under more challenging conditions, such as occlusion, is less characterized. Temporal dynamics of object recognition under occlusion is largely unknown in the human brain. Furthermore, we do not know if the previously successful deep-learning algorithms can similarly achieve human-level performance in these more challenging object recognition tasks. By linking brain data with behavior, and computational modeling, we characterized temporal dynamics of object recognition under occlusion, and proposed a computational mechanism that explains both behavioral and the neural data in humans. This provides a plausible mechanistic explanation for how our brain might be solving object recognition under more challenging conditions.

Perceptual Errors Support the Notion of Masking by Object Substitution

Two experiments examined the effect of object substitution masking (OSM) on the perceptual errors in reporting the orientation of a target. In Experiment 1, a four-dot trailing mask was compared with a simultaneous-noise mask. In Experiment 2, the four-dot and noise masks were factorially varied. Responses were modelled using a mixture regression model and Bayesian inference to deduce whether the relative impacts of OSM on guessing and precision were the same as those of a noise mask, and thus whether the mechanism underpinning OSM is based on increasing noise rather than a substitution process. Across both experiments, OSM was associated with an increased guessing rate when the mask trailed target offset and a reduction in the precision of the target representation (although the latter was less reliable across the two experiments). Importantly, the noise mask also influenced both guessing and precision, but in a different manner, suggesting that OSM is not simply caused by increasing noise. In Experiment 2, the effects of OSM and simultaneous-noise interacted, suggesting the two manipulations involve common mechanisms. Overall results suggest that OSM is often a consequence of a substitution process, but there is evidence that the mask increases noise levels on trials where substitution does not occur.

Anisotropic visual awareness of shapes

While object perception may feel instantaneous, it is an iterative process in which information is accumulated until ambiguity about identity and location is resolved. In theory, awareness of an object should depend on how efficiently this process occurs. Therefore, objects with inherently weak visual representations should be more susceptible to perceptual disruption. We tested this hypothesis by examining the perception of aspect ratio, a 2D feature of shapes with anisotropic representation (circular shapes are less robustly represented than elongated shapes in high-level visual areas). Observers viewed a target shape shown for 20-ms within an array of ellipses. The target, which varied from flat to tall, was either masked or unmasked. Observers indicated the target's aspect ratio and if it was visible. Observers reported seeing elongated shapes far more often than circular shapes, but only on trials with object-substitution masking. This effect replicated across five control experiments, even though the shapes were identical in basic image attributes (e.g., contrast, area). Our findings demonstrate that shapes with extreme aspect ratios are more readily available to awareness than shapes with ambiguous dimensionality. More generally, this work supports theories of object processing which suggest that strength of visual representation gates access to awareness.

Human-like perceptual masking is difficult to observe in rats performing an orientation discrimination task

Visual masking occurs when the perception of a brief target stimulus is affected by a preceding or succeeding mask. The uncoupling of the target and its perception allows an opportunity to investigate the neuronal mechanisms involved in sensory representation and visual perception. To determine whether rats are a suitable model for subsequent studies of the neuronal basis of visual masking, we first demonstrated that decoding of neuronal responses recorded in the primary visual cortex (V1) of anaesthetized rats predicted that orientation discrimination performance should decline when masking stimuli are presented immediately before or after oriented target stimuli. We then trained Long-Evans rats (n = 7) to discriminate between horizontal and vertical target Gabors or gratings. In some trials, a plaid mask was presented at varying stimulus onset asynchronies (SOAs) relative to the target. Spatially, the masks were presented either overlapping or surrounding the target location. In the absence of a mask, all animals could reliably discriminate orientation when stimulus durations were 16 ms or longer. In the presence of a mask, discrimination performance was impaired, but did not systematically vary with SOA as is typical of visual masking. In humans performing a similar task, we found visual masking impaired perception of the target at short SOAs regardless of the spatial or temporal configuration of stimuli. Our findings indicate that visual masking may be difficult to observe in rats as the stimulus parameters necessary to quantify masking will make the task so difficult that it prevents robust measurement of psychophysical performance. Thus, our results suggest that rats may not be an ideal model to investigate the effects of visual masking on perception.

Spatial vision in older adults: perceptual changes and neural bases

PURPOSE

The number of older adults is rapidly increasing internationally, leading to a significant increase in research on how healthy ageing impacts vision. Most clinical assessments of spatial vision involve simple detection (letter acuity, grating contrast sensitivity, perimetry). However, most natural visual environments are more spatially complicated, requiring contrast discrimination, and the delineation of object boundaries and contours, which are typically present on non-uniform backgrounds. In this review we discuss recent research that reports on the effects of normal ageing on these more complex visual functions, specifically in the context of recent neurophysiological studies.

RECENT FINDINGS

Recent research has concentrated on understanding the effects of healthy ageing on neural responses within the visual pathway in animal models. Such neurophysiological research has led to numerous, subsequently tested, hypotheses regarding the likely impact of healthy human ageing on specific aspects of spatial vision.

SUMMARY

Healthy normal ageing impacts significantly on spatial visual information processing from the retina through to visual cortex. Some human data validates that obtained from studies of animal physiology, however some findings indicate that rethinking of presumed neural substrates is required. Notably, not all spatial visual processes are altered by age. Healthy normal ageing impacts significantly on some spatial visual processes (in particular centre-surround tasks), but leaves contrast discrimination, contrast adaptation, and orientation discrimination relatively intact. The study of older adult vision contributes to knowledge of the brain mechanisms altered by the ageing process, can provide practical information regarding visual environments that older adults may find challenging, and may lead to new methods of assessing visual performance in clinical environments.

Perceptual averaging of facial expressions requires visual awareness and attention

Humans, as highly social animals, are regularly exposed to the faces of conspecifics-often more than one at a time. This feature of social living is important for understanding face perception, not just because it means that information from faces is available in bulk, but also because it changes the way individuals are perceived. For instance, when two faces are seen nearby one another, they tend to look like each other. This phenomenon of perceptual averaging is robust when both faces are seen and attended. But in everyday life, some faces may not receive the full benefit of attention, or they may not be visible at all. We evaluated whether perceptual averaging of relatively complex and simple information on faces, including facial expression and head orientation, can still occur even in these circumstances. In particular, we used object-substitution masking (OSM) and a dual-task designed to disrupt visual awareness and attention, respectively, during evaluations of briefly presented face pairs. Disruptions of awareness or attention eliminated averaging of facial expression, whereas orientation averaging persisted in spite of these challenges. These results demonstrate boundary conditions for the process of perceptual averaging. More generally, they provide insight into how the visual system processes multitudes of objects, both simple and complex, both with and without attention and awareness.

Binocular response modulation in the lateral geniculate nucleus

The dorsal lateral geniculate nucleus of the thalamus (LGN) receives the main outputs of both eyes and relays those signals to the visual cortex. Each retina projects to separate layers of the LGN so that each LGN neuron is innervated by a single eye. In line with this anatomical separation, visual responses of almost all of LGN neurons are driven by one eye only. Nonetheless, many LGN neurons are sensitive to what is shown to the other eye as their visual responses differ when both eyes are stimulated compared to when the driving eye is stimulated in isolation. This, predominantly suppressive, binocular modulation of LGN responses might suggest that the LGN is the first location in the primary visual pathway where the outputs from the two eyes interact. Indeed, the LGN features several anatomical structures that would allow for LGN neurons responding to one eye to modulate neurons that respond to the other eye. However, it is also possible that binocular response modulation in the LGN arises indirectly as the LGN also receives input from binocular visual structures. Here we review the extant literature on the effects of binocular stimulation on LGN spiking responses, highlighting findings from cats and primates, and evaluate the neural circuits that might mediate binocular response modulation in the LGN.

Slow and steady, not fast and furious: Slow temporal modulation strengthens continuous flash suppression

Continuous flash suppression (CFS) involves the presentation of a rapidly changing Mondrian sequence to one eye and a static target in the other eye. Targets presented in this manner remain suppressed for several seconds at a time, and this has seen the prevalent use of CFS in studies of unconscious visual processes. However, the mechanisms behind CFS remain unclear, complicating its use and the comprehension of results obtained with the paradigm. For example, some studies report observations indicative of faster, visual masking processes whereas others suggest slower, rivalry processes. To reconcile this discrepancy, this study investigates the effect of temporal frequency content and Mondrian pattern structure on CFS suppression. Our results show predominant influences of spatial edges and low temporal-frequency content, which are similar to binocular rivalry, affording a parsimonious alternative in unifying the two paradigms.

Failure to detect meaning in RSVP at 27 ms per picture

The human visual system has the remarkable ability to rapidly detect meaning from visual stimuli. Potter, Wyble, Hagmann, and McCourt (Attention, Perception, & Psychophysics, 76, 270-279, 2014) tested the minimum viewing time required to obtain meaning from a stream of pictures shown in a rapid serial visual presentation (RSVP) sequence containing either six or 12 pictures. They reported that observers could detect the presence of a target picture specified by name (e.g., smiling couple) even when the pictures in the sequence were presented for just 13 ms each. Potter et al. claimed that this was insufficient time for feedback processing to occur, so feedforward processing alone must be able to generate conscious awareness of the target pictures. A potential confound in their study is that the pictures in the RSVP sequence sometime contained areas with no high-contrast edges, and so may not have adequately masked each other. Consequently, iconic memories of portions of the target pictures may have persisted in the visual system, thereby increasing the effective presentation time. Our study addressed this issue by redoing the Potter et al. study, but using four different types of masks. We found that when adequate masking was used, no evidence emerged that observers could detect the presence of a specific target picture, even when each picture in the RSVP sequence was presented for 27 ms. On the basis of these findings, we cannot rule out the possibility that feedback processing is necessary for individual pictures to be recognized.

Object-substitution masking weakens but does not eliminate shape interactions

At any moment, some objects in the environment are seen clearly, whereas others go unnoticed. Whether or not these gaps in awareness are actually problematic may depend on the extent that information about unseen objects is lost. Determining when and how visual awareness and visual processing become linked is thus of great importance. Previous research using object-substitution masking (OSM) demonstrated that relatively simple visual features, such as size or orientation, are still processed even when they are not visible. Yet this does not appear to be the case for more complex features like faces. This suggests that, during OSM, disruptions of visual processing and awareness may tend to co-occur beginning at some intermediate stage along the ventral pathway. We tested this hypothesis by evaluating the extent to which OSM disrupted the perception and processing of two-dimensional objects. Specifically, we evaluated whether an unseen shape's aspect ratio would influence the appearance of another shape that was briefly visible nearby. As expected, the aspect ratios of two shapes appeared to be more similar to each other when both were visible. This averaging effect was weakened, but not eliminated, when one ellipse in each pair received OSM. These shape interactions persisted even when one ellipse from each pair was invisible. When combined with previous work, these results suggest that during object-substitution masking, disruptions of visual processing tend to strengthen with increases in stimulus complexity, becoming more tightly bound to the mechanisms of visual awareness at intermediate stages of visual analysis.

Masking reduces orientation selectivity in rat visual cortex

In visual masking the perception of a target stimulus is impaired by a preceding (forward) or succeeding (backward) mask stimulus. The illusion is of interest because it allows uncoupling of the physical stimulus, its neuronal representation, and its perception. To understand the neuronal correlates of masking, we examined how masks affected the neuronal responses to oriented target stimuli in the primary visual cortex (V1) of anesthetized rats (n = 37). Target stimuli were circular gratings with 12 orientations; mask stimuli were plaids created as a binarized sum of all possible target orientations. Spatially, masks were presented either overlapping or surrounding the target. Temporally, targets and masks were presented for 33 ms, but the stimulus onset asynchrony (SOA) of their relative appearance was varied. For the first time, we examine how spatially overlapping and center-surround masking affect orientation discriminability (rather than visibility) in V1. Regardless of the spatial or temporal arrangement of stimuli, the greatest reductions in firing rate and orientation selectivity occurred for the shortest SOAs. Interestingly, analyses conducted separately for transient and sustained target response components showed that changes in orientation selectivity do not always coincide with changes in firing rate. Given the near-instantaneous reductions observed in orientation selectivity even when target and mask do not spatially overlap, we suggest that monotonic visual masking is explained by a combination of neural integration and lateral inhibition.

A computational investigation of feedforward and feedback processing in metacontrast backward masking

In human perception studies, visual backward masking has been used to understand the temporal dynamics of subliminal vs. conscious perception. When a brief target stimulus is followed by a masking stimulus after a short interval of <100 ms, performance on the target is impaired when the target and mask are in close spatial proximity. While the psychophysical properties of backward masking have been studied extensively, there is still debate on the underlying cortical dynamics. One prevailing theory suggests that the impairment of target performance due to the mask is the result of lateral inhibition between the target and mask in feedforward processing. Another prevailing theory suggests that this impairment is due to the interruption of feedback processing of the target by the mask. This computational study demonstrates that both aspects of these theories may be correct. Using a biophysical model of V1 and V2, visual processing was modeled as interacting neocortical attractors, which must propagate up the visual stream. If an activating target attractor in V1 is quiesced enough with lateral inhibition from a mask, or not reinforced by recurrent feedback, it is more likely to burn out before becoming fully active and progressing through V2 and beyond. Results are presented which simulate metacontrast backward masking with an increasing stimulus interval and with the presence and absence of feedback activity. This showed that recurrent feedback diminishes backward masking effects and can make conscious perception more likely. One model configuration presented a metacontrast noise mask in the same hypercolumns as the target, and produced type-A masking. A second model configuration presented a target line with two parallel adjacent masking lines, and produced type-B masking. Future work should examine how the model extends to more complex spatial mask configurations.

Probing feedforward and feedback contributions to awareness with visual masking and transcranial magnetic stimulation

A number of influential theories posit that visual awareness relies not only on the initial, stimulus-driven (i.e., feedforward) sweep of activation but also on recurrent feedback activity within and between brain regions. These theories of awareness draw heavily on data from masking paradigms in which visibility of one stimulus is reduced due to the presence of another stimulus. More recently transcranial magnetic stimulation (TMS) has been used to study the temporal dynamics of visual awareness. TMS over occipital cortex affects performance on visual tasks at distinct time points and in a manner that is comparable to visual masking. We draw parallels between these two methods and examine evidence for the neural mechanisms by which visual masking and TMS suppress stimulus visibility. Specifically, both methods have been proposed to affect feedforward as well as feedback signals when applied at distinct time windows relative to stimulus onset and as a result modify visual awareness. Most recent empirical evidence, moreover, suggests that while visual masking and TMS impact stimulus visibility comparably, the processes these methods affect may not be as similar as previously thought. In addition to reviewing both masking and TMS studies that examine feedforward and feedback processes in vision, we raise questions to guide future studies and further probe the necessary conditions for visual awareness.

Phosphene-guided transcranial magnetic stimulation of occipital but not parietal cortex suppresses stimulus visibility

Transcranial magnetic stimulation (TMS) applied over the occipital lobe approximately 100 ms after the onset of a stimulus decreases its visibility if it appears in the location of the phosphene. Because phosphenes can also be elicited by stimulation of the parietal regions, we asked if the same procedure that is used to reduce visibility of stimuli with occipital TMS will lead to decreased stimulus visibility when TMS is applied to parietal regions. TMS was randomly applied at 0-130 ms after the onset of the stimulus in steps of 10 ms in occipital and parietal regions. Participants responded to the orientation of the line stimulus and rated its visibility. We replicate previous reports of phosphenes from both occipital and parietal TMS. As previously reported, we also observed visual suppression around the classical 100 ms window both in the objective line orientation and subjective visibility responses with occipital TMS. Parietal stimulation, on the other hand, did not consistently reduce stimulus visibility in any time window.

Perception-action dissociations depend on the luminance contrast of the stimuli

The observation that near-threshold low-contrast visual distractors can equally influence perceptual state and goal-directed motor responses was recently taken as an argument against a sharp separation between a conscious vision for perception and an unconscious vision for action. However, data supporting the dual visual system theory have principally involved high-contrast stimuli. In the present study, we assessed the effect of varying the contrast of a near-threshold visual distractor while keeping its visibility constant with backward noise masks. Eight participants performed fast manual reaching movements toward a highly visible target while subsequently reporting the presence/absence of a near-threshold distractor appearing at the opposite location with respect to the body midline. For all distractor contrasts, hand trajectory deviations toward the distractor were observed when the distractor was present and detected. When the distractor remained undetected deviations also occurred, but for higher contrasts. The subliminal motor effect traditionally observed in visual masking studies may therefore primarily depend on the luminance contrast of the interfering stimuli. These results suggest that dissociations between perceptual and motor responses can be explained by a single-signal model involving differential thresholds for perception and action that are specifically modulated as a function of both the requirements of the task and the contrast level of the stimuli. Such modulation is compatible with neurophysiological accounts of visual masking in which feedforward activation to—and feedback activation from—higher visual areas are correlated with the actual presence of the stimulation and its conscious perception, respectively.

Unconscious Priming Requires Early Visual Cortex at Specific Temporal Phases of Processing

Although examples of unconscious shape priming have been well documented, whether such priming requires early visual cortex (V1/V2) has not been established. In the current study, we used TMS of V1/V2 at varying temporal intervals to suppress the visibility of preceding shape primes while the interval between primes and targets was kept constant. Our results show that, although conscious perception requires V1/V2, unconscious priming can occur without V1/V2 at an intermediate temporal interval but not at early (5–25 msec) or later (65–125 msec) stages of processing. Because the later time window of unconscious priming suppression has been proposed to interfere with feedback processing, our results further suggest that feedback processing is also essential for unconscious priming and may not be a sufficient condition for conscious vision.

Contextual influences in texture-segmentation: distinct effects from elements along the edge and in the texture-region

Both neurophysiological and psychophysical evidence suggest a strong influence of context on texture-segmentation. Here we extend and further analyse this issue, with a particular focus on the underlying mechanism. Specifically, we use a texture-edge discrimination task and separately investigate the effect of elements far from and along the edge. Consistent with previous studies, we report both an iso-near contextual effect - whereby performance is better if elements along the edge are iso-oriented compared to ortho-oriented to the edge - as well as an ortho-far effect - whereby discrimination is higher when elements far from the edge are orthogonal to the edge. We found that backward mask, which is known to interrupt re-entrant processing from extrastriate areas, only interferes with the iso-near effect whereas perturbing orientation, position or contrast polarity of elements far from the edge only abolishes the ortho-far effect. This suggests that feedback processes may be involved in the iso-near effect. Instead, the ortho-far effect may be accounted for by recurrent interactions among 1st order filters.

Optimizing the temporal dynamics of light to human perception

No previous research has tuned the temporal characteristics of light-emitting devices to enhance brightness perception in human vision, despite the potential for significant power savings. The role of stimulus duration on perceived contrast is unclear, due to contradiction between the models proposed by Bloch and by Broca and Sulzer over 100 years ago. We propose that the discrepancy is accounted for by the observer's "inherent expertise bias," a type of experimental bias in which the observer's life-long experience with interpreting the sensory world overcomes perceptual ambiguities and biases experimental outcomes. By controlling for this and all other known biases, we show that perceived contrast peaks at durations of 50-100 ms, and we conclude that the Broca-Sulzer effect best describes human temporal vision. We also show that the plateau in perceived brightness with stimulus duration, described by Bloch's law, is a previously uncharacterized type of temporal brightness constancy that, like classical constancy effects, serves to enhance object recognition across varied lighting conditions in natural vision-although this is a constancy effect that normalizes perception across temporal modulation conditions. A practical outcome of this study is that tuning light-emitting devices to match the temporal dynamics of the human visual system's temporal response function will result in significant power savings.

Reading a standing wave: figure-ground-alternation masking of primes in evaluative priming

We propose a new masking technique for masking word stimuli. Drawing on the phenomena of metacontrast and paracontrast, we alternately presented two prime displays of the same word with the background color in one display matching the font color in the other display and vice versa. The sequence of twenty alterations (spanning approx. 267 ms) was sandwich-masked by structure masks. Using this masking technique, we conducted evaluative priming experiments with positive and negative target and prime words. Significant priming effects were found - for primes and targets drawn from the same as well as from different word sets. Priming effects were independent of prime discrimination performance in direct tests and they were still significant after the sample was restricted to those participants who showed random responding in the direct test.

Masking of figure-ground texture and single targets by surround inhibition: a computational spiking model

A visual stimulus can be made invisible, i.e. masked, by the presentation of a second stimulus. In the sensory cortex, neural responses to a masked stimulus are suppressed, yet how this suppression comes about is still debated. Inhibitory models explain masking by asserting that the mask exerts an inhibitory influence on the responses of a neuron evoked by the target. However, other models argue that the masking interferes with recurrent or reentrant processing. Using computer modeling, we show that surround inhibition evoked by ON and OFF responses to the mask suppresses the responses to a briefly presented stimulus in forward and backward masking paradigms. Our model results resemble several previously described psychophysical and neurophysiological findings in perceptual masking experiments and are in line with earlier theoretical descriptions of masking. We suggest that precise spatiotemporal influence of surround inhibition is relevant for visual detection.

The effects of spatial and temporal cueing on metacontrast masking

We studied the effects of selective attention on metacontrast masking with 3 different cueing experiments. Experiments 1 and 2 compared central symbolic and peripheral spatial cues. For symbolic cues, we observed small attentional costs, that is, reduced visibility when the target appeared at an unexpected location, and attentional costs as well as benefits for peripheral cues. All these effects occurred exclusively at the late, ascending branch of the U-shaped metacontrast masking function, although the possibility exists that cueing effects at the early branch were obscured by a ceiling effect due to almost perfect visibility at short stimulus onset asynchronies (SOAs). In Experiment 3, we presented temporal cues that indicated when the target was likely to appear, not where. Here, we also observed cueing effects in the form of higher visibility when the target appeared at the expected point in time compared to when it appeared too early. However, these effects were not restricted to the late branch of the masking function, but enhanced visibility over the complete range of the masking function. Given these results we discuss a common effect for different types of spatial selective attention on metacontrast masking involving neural subsystems that are different from those involved in temporal attention.

Temporal integration of movement: the time-course of motion streaks revealed by masking

Temporal integration in the visual system causes fast-moving objects to leave oriented ‘motion streaks’ in their wake, which could be used to facilitate motion direction perception. Temporal integration is thought to occur over 100 ms in early cortex, although this has never been tested for motion streaks. Here we compare the ability of fast-moving (‘streaky’) and slow-moving fields of dots to mask briefly flashed gratings either parallel or orthogonal to the motion trajectory. Gratings were presented at various asynchronies relative to motion onset (from to ms) to sample the time-course of the accumulating streaks. Predictions were that masking would be strongest for the fast parallel condition, and would be weak at early asynchronies and strengthen over time as integration rendered the translating dots more streaky and grating-like. The asynchrony where the masking function reached a plateau would correspond to the temporal integration period. As expected, fast-moving dots caused greater masking of parallel gratings than orthogonal gratings, and slow motion produced only modest masking of either grating orientation. Masking strength in the fast, parallel condition increased with time and reached a plateau after 77 ms, providing an estimate of the temporal integration period for mechanisms encoding motion streaks. Interestingly, the greater masking by fast motion of parallel compared with orthogonal gratings first reached significance at 48 ms before motion onset, indicating an effect of backward masking by motion streaks.

Neuro-cognitive mechanisms of conscious and unconscious visual perception: From a plethora of phenomena to general principles

Psychological and neuroscience approaches have promoted much progress in elucidating the cognitive and neural mechanisms that underlie phenomenal visual awareness during the last decades. In this article, we provide an overview of the latest research investigating important phenomena in conscious and unconscious vision. We identify general principles to characterize conscious and unconscious visual perception, which may serve as important building blocks for a unified model to explain the plethora of findings. We argue that in particular the integration of principles from both conscious and unconscious vision is advantageous and provides critical constraints for developing adequate theoretical models. Based on the principles identified in our review, we outline essential components of a unified model of conscious and unconscious visual perception. We propose that awareness refers to consolidated visual representations, which are accessible to the entire brain and therefore globally available. However, visual awareness not only depends on consolidation within the visual system, but is additionally the result of a post-sensory gating process, which is mediated by higher-level cognitive control mechanisms. We further propose that amplification of visual representations by attentional sensitization is not exclusive to the domain of conscious perception, but also applies to visual stimuli, which remain unconscious. Conscious and unconscious processing modes are highly interdependent with influences in both directions. We therefore argue that exactly this interdependence renders a unified model of conscious and unconscious visual perception valuable. Computational modeling jointly with focused experimental research could lead to a better understanding of the plethora of empirical phenomena in consciousness research.