Overt visual attention as a causal factor of perceptual awareness

Motor Strategies: The Role of Active Behavior in Spatial Hearing Research

When completing a task, the ability to implement behavioral strategies to solve it in an effective and cognitively less-demanding way is extremely adaptive for humans. This behavior makes it possible to accumulate evidence and test one's own predictions about the external world. In this work, starting from examples in the field of spatial hearing research, I analyze the importance of considering motor strategies in perceptual tasks, and I stress the urgent need to create ecological experimental settings, which are essential in allowing the implementation of such behaviors and in measuring them. In particular, I will consider head movements as an example of strategic behavior implemented to solve acoustic space-perception tasks.

Gestalt's Perspective on Insight: A Recap Based on Recent Behavioral and Neuroscientific Evidence

The Gestalt psychologists' theory of insight problem-solving was based on a direct parallelism between perceptual experience and higher-order forms of cognition (e.g., problem-solving). Similarly, albeit not exclusively, to the sudden recognition of bistable figures, these psychologists contended that problem-solving involves a restructuring of one's initial representation of the problem's elements, leading to a sudden leap of understanding phenomenologically indexed by the "Aha!" feeling. Over the last century, different scholars have discussed the validity of the Gestalt psychologists' perspective, foremost using the behavioral measures available at the time. However, in the last two decades, scientists have gained a deeper understanding of insight problem-solving due to the advancements in cognitive neuroscience. This review aims to provide a retrospective reading of Gestalt theory based on the knowledge accrued by adopting novel paradigms of research and investigating their neurophysiological correlates. Among several key points that the Gestalt psychologists underscored, we focus specifically on the role of the visual system in marking a discrete switch of knowledge into awareness, as well as the perceptual experience and holistic standpoints. While the main goal of this paper is to read the previous theory in light of new evidence, we also hope to initiate an academic discussion and encourage further research about the points we raise.

Pupil dilation response elicited by violations of auditory regularities is a promising but challenging approach to probe consciousness at the bedside

Pupil dilation response (PDR) has been proposed as a physiological marker of conscious access to a stimulus or its attributes, such as novelty. In a previous study on healthy volunteers, we adapted the auditory "local global" paradigm and showed that violations of global regularity elicited a PDR. Notably without instructions, this global effect was present only in participants who could consciously report violations of global regularities. In the present study, we used a similar approach in 24 non-communicating patients affected with a Disorder of Consciousness (DoC) and compared PDR to ERPs regarding diagnostic and prognostic performance. At the group level, global effect could not be detected in DoC patients. At the individual level, the only patient with a PDR global effect was in a MCS and recovered consciousness at 6 months. Contrasting the most regular trials to the most irregular ones improved PDR's diagnostic and prognostic power in DoC patients. Pupillometry is a promising tool but requires several methodological improvements to enhance the signal-to-noise ratio and make it more robust for probing consciousness and cognition in DoC patients.

Active vision in sight recovery individuals with a history of long-lasting congenital blindness

What we see is intimately linked to how we actively and systematically explore the world through eye movements. However, it is unknown to what degree visual experience during early development is necessary for such systematic visual exploration to emerge. The present study investigated visual exploration behavior in ten human participants whose sight had been restored only in childhood or adulthood, after a period of congenital blindness due to dense bilateral congenital cataracts. Participants freely explored real-world images while their eye movements were recorded. Despite severe residual visual impairments and gaze instability (nystagmus), visual exploration patterns were preserved in individuals with reversed congenital cataract. Modelling analyses indicated that similar to healthy controls, visual exploration in individuals with reversed congenital cataract was based on the low-level (luminance contrast) and high-level (object components) visual content of the images. Moreover, participants used visual short-term memory representations for narrowing down the exploration space. More systematic visual exploration in individuals with reversed congenital cataract was associated with better object recognition, suggesting that active vision might be a driving force for visual system development and recovery. The present results argue against a sensitive period for the development of neural mechanisms associated with visual exploration.SIGNIFICANCE STATEMENTHumans explore the visual world with systematic patterns of eye movements, but it is unknown whether early visual experience is necessary for the acquisition of visual exploration. Here, we show that sight recovery individuals who had been born blind demonstrate highly systematic eye movements while exploring real-world images, despite visual impairments and pervasive gaze instability. In fact, their eye movement patterns were predicted by those of normally sighted controls and models calculating eye movements based on low- and high-level visual features, and they moreover took memory information into account. Since object recognition performance was associated with systematic visual exploration it was concluded that eye movements might be a driving factor for the development of the visual system.

Apical amplification-a cellular mechanism of conscious perception?

We present a theoretical view of the cellular foundations for network-level processes involved in producing our conscious experience. Inputs to apical synapses in layer 1 of a large subset of neocortical cells are summed at an integration zone near the top of their apical trunk. These inputs come from diverse sources and provide a context within which the transmission of information abstracted from sensory input to their basal and perisomatic synapses can be amplified when relevant. We argue that apical amplification enables conscious perceptual experience and makes it more flexible, and thus more adaptive, by being sensitive to context. Apical amplification provides a possible mechanism for recurrent processing theory that avoids strong loops. It makes the broadcasting hypothesized by global neuronal workspace theories feasible while preserving the distinct contributions of the individual cells receiving the broadcast. It also provides mechanisms that contribute to the holistic aspects of integrated information theory. As apical amplification is highly dependent on cholinergic, aminergic, and other neuromodulators, it relates the specific contents of conscious experience to global mental states and to fluctuations in arousal when awake. We conclude that apical dendrites provide a cellular mechanism for the context-sensitive selective amplification that is a cardinal prerequisite of conscious perception.

Oculometric signature of switch into awareness? Pupil size predicts sudden insight whereas microsaccades predict problem-solving via analysis

According to the Gestalt theorists, restructuring is an essential component of insight problem-solving, contributes to the "Aha!" experience, and is similar to the perceptual switch experienced when reinterpreting ambiguous figures. Previous research has demonstrated that pupil diameter increases during the perceptual switch of ambiguous figures, and indexes norepeinephrine functioning mediated by the locus coeruleus. In this study, we investigated if pupil diameter similarly predicts the switch into awareness people experience when solving a problem via insight. Additionally, we explored eye movement dynamics during the same task to investigate if the problem-solving strategies used are linked to specific oculomotor behaviors. In 38 participants, pupil diameter increased about 500 msec prior to solution only in trials for which subjects report having an insight. In contrast, participants increased their microsaccade rate only prior to non-insight solutions. Pupil dilation and microsaccades were not reliably related, but both appear to be robust markers of how people solve problems (with or without insight). The pupil size change seen when people have an "Aha!" moment represents an indicator of the switch into awareness of unconscious processes humans depend upon for insight, and suggests important involvement of norepinephrine, via the locus coeruleus, in sudden insight.

Natural visual behavior in individuals with peripheral visual-field loss

Retinitis pigmentosa (RP) is an inherited disease that causes progressive peripheral visual-field loss. In this study, we investigated how such loss affects visual exploration of natural images. Individuals with varying degrees of visual-field loss and healthy control participants freely observed images of different sizes while eye movements were recorded. We examined whether visual behavior differed when the scene content was shown in various extents of the visual field, and investigated the spatial bias, saccade amplitudes, and number and duration of fixations. We found that the healthy control group showed a central spatial bias during image viewing. The RP group showed similar biases on the group level, but with reproducible individual exploration patterns. For saccade amplitudes, the healthy control group and the RP group showed similar behavior throughout all image sizes. The RP group with severe loss of peripheral vision thus tended to target saccades toward blind areas of their visual field. The number of fixations did not change between the two groups, although fixation durations decreased in the RP group. In conclusion, the RP group scanned the images surprisingly similarly to the healthy control group; however, they showed individual idiosyncratic explorative strategies when the observed scene exceeded their visible field. Thus, although RP leads to a severe loss of the visual field, there is no general adaptive mechanism to change visual exploration. Instead, individuals rely on individual strategies, leading to high heterogeneity in the RP group.

Association between pupil dilation and implicit processing prior to object recognition via insight

Insight refers to the sudden conscious shift in the perception of a situation following a period of unconscious processing. The present study aimed to investigate the implicit neural mechanisms underlying insight-based recognition, and to determine the association between these mechanisms and the extent of pupil dilation. Participants were presented with ambiguous, transforming images comprised of dots, following which they were asked to state whether they recognized the object and their level of confidence in this statement. Changes in pupil dilation were not only characterized by the recognition state into the ambiguous object but were also associated with prior awareness of object recognition, regardless of meta-cognitive confidence. Our findings indicate that pupil dilation may represent the level of implicit integration between memory and visual processing, despite the lack of object awareness, and that this association may involve noradrenergic activity within the locus coeruleus-noradrenergic (LC-NA) system.

Functional dissociation of stimulus intensity encoding and predictive coding of pain in the insula

The computational principles by which the brain creates a painful experience from nociception are still unknown. Classic theories suggest that cortical regions either reflect stimulus intensity or additive effects of intensity and expectations, respectively. By contrast, predictive coding theories provide a unified framework explaining how perception is shaped by the integration of beliefs about the world with mismatches resulting from the comparison of these beliefs against sensory input. Using functional magnetic resonance imaging during a probabilistic heat pain paradigm, we investigated which computations underlie pain perception. Skin conductance, pupil dilation, and anterior insula responses to cued pain stimuli strictly followed the response patterns hypothesized by the predictive coding model, whereas posterior insula encoded stimulus intensity. This novel functional dissociation of pain processing within the insula together with previously observed alterations in chronic pain offer a novel interpretation of aberrant pain processing as disturbed weighting of predictions and prediction errors.

An extensive dataset of eye movements during viewing of complex images

We present a dataset of free-viewing eye-movement recordings that contains more than 2.7 million fixation locations from 949 observers on more than 1000 images from different categories. This dataset aggregates and harmonizes data from 23 different studies conducted at the Institute of Cognitive Science at Osnabrück University and the University Medical Center in Hamburg-Eppendorf. Trained personnel recorded all studies under standard conditions with homogeneous equipment and parameter settings. All studies allowed for free eye-movements, and differed in the age range of participants (~7-80 years), stimulus sizes, stimulus modifications (phase scrambled, spatial filtering, mirrored), and stimuli categories (natural and urban scenes, web sites, fractal, pink-noise, and ambiguous artistic figures). The size and variability of viewing behavior within this dataset presents a strong opportunity for evaluating and comparing computational models of overt attention, and furthermore, for thoroughly quantifying strategies of viewing behavior. This also makes the dataset a good starting point for investigating whether viewing strategies change in patient groups.

Differential Contribution of Low- and High-level Image Content to Eye Movements in Monkeys and Humans

Oculomotor selection exerts a fundamental impact on our experience of the environment. To better understand the underlying principles, researchers typically rely on behavioral data from humans, and electrophysiological recordings in macaque monkeys. This approach rests on the assumption that the same selection processes are at play in both species. To test this assumption, we compared the viewing behavior of 106 humans and 11 macaques in an unconstrained free-viewing task. Our data-driven clustering analyses revealed distinct human and macaque clusters, indicating species-specific selection strategies. Yet, cross-species predictions were found to be above chance, indicating some level of shared behavior. Analyses relying on computational models of visual saliency indicate that such cross-species commonalities in free viewing are largely due to similar low-level selection mechanisms, with only a small contribution by shared higher level selection mechanisms and with consistent viewing behavior of monkeys being a subset of the consistent viewing behavior of humans.

Pupil Sizes Scale with Attentional Load and Task Experience in a Multiple Object Tracking Task

Previous studies have related changes in attentional load to pupil size modulations. However, studies relating changes in attentional load and task experience on a finer scale to pupil size modulations are scarce. Here, we investigated how these changes affect pupil sizes. To manipulate attentional load, participants covertly tracked between zero and five objects among several randomly moving objects on a computer screen. To investigate effects of task experience, the experiment was conducted on three consecutive days. We found that pupil sizes increased with each increment in attentional load. Across days, we found systematic pupil size reductions. We compared the model fit for predicting pupil size modulations using attentional load, task experience, and task performance as predictors. We found that a model which included attentional load and task experience as predictors had the best model fit while adding performance as a predictor to this model reduced the overall model fit. Overall, results suggest that pupillometry provides a viable metric for precisely assessing attentional load and task experience in visuospatial tasks.

Combined effects of expectations and visual uncertainty upon detection and identification of a target in the fog

Detecting a pedestrian while driving in the fog is one situation where the prior expectation about the target presence is integrated with the noisy visual input. We focus on how these sources of information influence the oculomotor behavior and are integrated within an underlying decision-making process. The participants had to judge whether high-/low-density fog scenes displayed on a computer screen contained a pedestrian or a deer by executing a mouse movement toward the response button (mouse-tracking). A variable road sign was added on the scene to manipulate expectations about target identity. We then analyzed the timing and amplitude of the deviation of mouse trajectories toward the incorrect response and, using an eye tracker, the detection time (before fixating the target) and the identification time (fixations on the target). Results revealed that expectation of the correct target results in earlier decisions with less deviation toward the alternative response, this effect being partially explained by the facilitation of target identification.

Revisiting the empirical case against perceptual modularity

Some theorists hold that the human perceptual system has a component that receives input only from units lower in the perceptual hierarchy. This thesis, that we shall here refer to as the encapsulation thesis, has been at the center of a continuing debate for the past few decades. Those who deny the encapsulation thesis often rely on the large body of psychological findings that allegedly suggest that perception is influenced by factors such as the beliefs, desires, goals, and the expectations of the perceiver. Proponents of the encapsulation thesis, however, often argue that, when correctly interpreted, these psychological findings are compatible with the thesis. In our view, the debate over the significance and the correct interpretation of these psychological findings has reached an impasse. We hold that this impasse is due to the methodological limitations over psychophysical experiments, and it is very unlikely that such experiments, on their own, could yield results that would settle the debate. After defending this claim, we argue that integrating data from cognitive neuroscience resolves the debate in favor of those who deny the encapsulation thesis.

Dynamic competition and binding of concepts through time and space

Models of implicit stereotypes (e.g., association of male with math or female with language) usually explain the faster responses observed for stereotype-congruent trials in the Implicit Association Test (IAT) by requiring a fundamental opposition between the male and female concepts (or math-language), limiting the decision-making dynamics to abstract dimensions. This paper introduces alternate models exploiting the sensorimotor dimensions of the IAT, which naturally account for the opposition between concepts, because typically mapped on opposite corners of the screen space and on different response actions. In addition to the emergence of the IAT effect, dynamic characteristics of the decision-making process within these models are tested against human data, obtained with a mouse-tracking adapted IAT procedure.

Using the past to anticipate the future in human foraging behavior

Humans engage in many tasks that involve gathering multiple targets from their environment (e.g., picking berries from a patch). Such foraging tasks raise questions about how observers maximize target collection - e.g., how long should one spend at one berry patch before moving to the next patch. Classic optimal foraging theories propose a simple decision rule: People move on when current intake drops below the average rate. Previous studies of foraging often assume this average is fixed and predict no strong relationship between the contents of the immediately preceding patch or patches and the current patch. In contrast to this prediction, we found evidence of temporal effects in a laboratory analog of a berry-picking task. Observers stayed longer when previous patches were better. This result is the opposite of what would be predicted by a model in which the assessment of the average rate is biased in favor of recent patches. This result was found when patch quality varied systematically over the course of the experiment (Experiment 1). Smaller effects were seen when patch quality was randomized (Experiment 2). Together, these results suggest that optimal foraging theories must account for the recent history to explain current behavior.

Effects of contextual information and stimulus ambiguity on overt visual sampling behavior

The sampling of our visual environment through saccadic eye movements is an essential function of the brain, allowing us to overcome the limits of peripheral vision. Understanding which parts of a scene attract overt visual attention is subject to intense research, and considerable progress has been made in unraveling the underlying cortical mechanisms. In contrast to spatial aspects, however, relatively little is understood about temporal aspects of overt visual sampling. At every fixation, the oculomotor system faces the decision whether to keep exploring different aspects of an object or scene or whether to remain fixated to allow for in-depth cortical processing - a situation that can be understood in terms of an exploration-exploitation dilemma. To improve our understanding of the factors involved in these decisions, we here investigate how the level of visual information, experimentally manipulated by scene context and stimulus ambiguity, changes the sampling behavior preceding the recognition of centrally presented ambiguous and disambiguated objects. Behaviorally, we find that context, although only presented until the first voluntary saccade, biases the perceptual outcome and significantly reduces reaction times. Importantly, we find that increased information about an object significantly alters its visual exploration, as evident through increased fixation durations and reduced saccade amplitudes. These results demonstrate that the initial sampling of an object, preceding its recognition, is subject to change based on the amount of information available in the system: increased evidence for its identity biases the exploration-exploitation strategy towards in-depth analyses.

Competition with and without priority control: linking rivalry to attention through winner-take-all networks with memory

Competition is ubiquitous in perception. For example, items in the visual field compete for processing resources, and attention controls their priority (biased competition). The inevitable ambiguity in the interpretation of sensory signals yields another form of competition: distinct perceptual interpretations compete for access to awareness. Rivalry, where two equally likely percepts compete for dominance, explicates the latter form of competition. Building upon the similarity between attention and rivalry, we propose to model rivalry by a generic competitive circuit that is widely used in the attention literature-a winner-take-all (WTA) network. Specifically, we show that a network of two coupled WTA circuits replicates three common hallmarks of rivalry: the distribution of dominance durations, their dependence on input strength ("Levelt's propositions"), and the effects of stimulus removal (blanking). This model introduces a form of memory by forming discrete states and explains experimental data better than competitive models of rivalry without memory. This result supports the crucial role of memory in rivalry specifically and in competitive processes in general. Our approach unifies the seemingly distinct phenomena of rivalry, memory, and attention in a single model with competition as the common underlying principle.

How active perception and attractor dynamics shape perceptual categorization: a computational model

We propose a computational model of perceptual categorization that fuses elements of grounded and sensorimotor theories of cognition with dynamic models of decision-making. We assume that category information consists in anticipated patterns of agent-environment interactions that can be elicited through overt or covert (simulated) eye movements, object manipulation, etc. This information is firstly encoded when category information is acquired, and then re-enacted during perceptual categorization. The perceptual categorization consists in a dynamic competition between attractors that encode the sensorimotor patterns typical of each category; action prediction success counts as "evidence" for a given category and contributes to falling into the corresponding attractor. The evidence accumulation process is guided by an active perception loop, and the active exploration of objects (e.g., visual exploration) aims at eliciting expected sensorimotor patterns that count as evidence for the object category. We present a computational model incorporating these elements and describing action prediction, active perception, and attractor dynamics as key elements of perceptual categorizations. We test the model in three simulated perceptual categorization tasks, and we discuss its relevance for grounded and sensorimotor theories of cognition.

Pupil dilation co-varies with memory strength of individual traces in a delayed response paired-associate task

Studies on cognitive effort have shown that pupil dilation is a reliable indicator of memory load. However, it is conceivable that there are other sources of effort involved in memory that also affect pupil dilation. One of these is the ease with which an item can be retrieved from memory. Here, we present the results of an experiment in which we studied the way in which pupil dilation acts as an online marker for memory processing during the retrieval of paired associates while reducing confounds associated with motor responses. Paired associates were categorized into sets containing either 4 or 7 items. After learning the paired associates once, pupil dilation was measured during the presentation of the retrieval cue during four repetitions of each set. Memory strength was operationalized as the number of repetitions (frequency) and set-size, since having more items per set results in a lower average recency. Dilation decreased with increased memory strength, supporting the hypothesis that the amplitude of the evoked pupillary response correlates positively with retrieval effort. Thus, while many studies have shown that “memory load” influences pupil dilation, our results indicate that the task-evoked pupillary response is also sensitive to the experimentally manipulated memory strength of individual items. As these effects were observed well before the response had been given, this study also suggests that pupil dilation can be used to assess an item’s memory strength without requiring an overt response.