Shape perception simultaneously up- and downregulates neural activity in the primary visual cortex

Visual hallucinations originating in the retinofugal pathway under clinical and psychedelic conditions

Psychedelics like LSD (Lysergic acid diethylamide) and psilocybin are known to modulate perceptual modalities due to the activation of mostly serotonin receptors in specific cortical (e.g., visual cortex) and subcortical (e.g., thalamus) regions of the brain. In the visual domain, these psychedelic modulations often result in peculiar disturbances of viewed objects and light and sometimes even in hallucinations of non-existent environments, objects, and creatures. Although the underlying processes are poorly understood, research conducted over the past twenty years on the subjective experience of psychedelics details theories that attempt to explain these perceptual alterations due to a disruption of communication between cortical and subcortical regions. However, rare medical conditions in the visual system like Charles Bonnet syndrome that cause perceptual distortions may shed new light on the additional importance of the retinofugal pathway in psychedelic subjective experiences. Interneurons in the retina called amacrine cells could be the first site of visual psychedelic modulation and aid in disrupting the hierarchical structure of how humans perceive visual information. This paper presents an understanding of how the retinofugal pathway communicates and modulates visual information in psychedelic and clinical conditions. Therefore, we elucidate a new theory of psychedelic modulation in the retinofugal pathway.

Self-Regulation of the Posterior-Frontal Brain Activity with Real-Time fMRI Neurofeedback to Influence Perceptual Discrimination

The Global Neuronal Workspace (GNW) hypothesis states that the visual percept is available to conscious awareness only if recurrent long-distance interactions among distributed brain regions activate neural circuitry extending from the posterior areas to prefrontal regions above a certain excitation threshold. To directly test this hypothesis, we trained 14 human participants to increase blood oxygenation level-dependent (BOLD) signals with real-time functional magnetic resonance imaging (rtfMRI)-based neurofeedback simultaneously in four specific regions of the occipital, temporal, insular and prefrontal parts of the brain. Specifically, we hypothesized that the up-regulation of the mean BOLD activity in the posterior-frontal brain regions lowers the perceptual threshold for visual stimuli, while down-regulation raises the threshold. Our results showed that participants could perform up-regulation (Wilcoxon test, session 1: p = 0.022; session 4: p = 0.041) of the posterior-frontal brain activity, but not down-regulation. Furthermore, the up-regulation training led to a significant reduction in the visual perceptual threshold, but no substantial change in perceptual threshold was observed after the down-regulation training. These findings show that the up-regulation of the posterior-frontal regions improves the perceptual discrimination of the stimuli. However, further questions as to whether the posterior-frontal regions can be down-regulated at all, and whether down-regulation raises the perceptual threshold, remain unanswered.

Systematic review and meta-analysis of the visual mismatch negativity in schizophrenia

Mismatch negativity (MMN) is an event-related potential component automatically elicited by events that violate predictions based on prior events. To elicit this component, researchers use stimulus repetition to induce predictions, and the MMN is obtained by subtracting the brain response to rare or unpredicted stimuli from that of frequent stimuli. Under the Predictive Processing framework, one increasingly popular interpretation of the mismatch response postulates that MMN represents a prediction error. In this context, the reduced MMN amplitude to auditory stimuli has been considered a potential biomarker of Schizophrenia, representing a reduced prediction error and the inability to update the mental model of the world based on the sensory signals. It is unclear, however, whether this amplitude reduction is specific for auditory events or if the visual MMN reveals a similar pattern in schizophrenia spectrum disorder. This review and meta-analysis aimed to summarise the available literature on the vMMN in schizophrenia. A systematic literature search resulted in 10 eligible studies that resulted in a combined effect size of g = -.63, CI [-.86, -.41], reflecting lower vMMN amplitudes in patients. These results are in line with the findings in the auditory domain. This component offers certain advantages, such as less susceptibility to overlap with components generated by attentional demands. Future studies should use vMMN to explore abnormalities in the Predictive Processing framework in different stages and groups of the SSD and increase the knowledge in the search for biomarkers in schizophrenia.

Visual working memories are abstractions of percepts

During perception, decoding the orientation of gratings depends on complex interactions between the orientation of the grating, aperture edges, and topographic structure of the visual map. Here, we aimed to test how aperture biases described during perception affect working memory (WM) decoding. For memoranda, we used gratings multiplied by radial and angular modulators to generate orthogonal aperture biases for identical orientations. Therefore, if WM representations are simply maintained sensory representations, they would have similar aperture biases. If they are abstractions of sensory features, they would be unbiased and the modulator would have no effect on orientation decoding. Neural patterns of delay period activity while maintaining the orientation of gratings with one modulator (e.g. radial) were interchangeable with patterns while maintaining gratings with the other modulator (e.g. angular) in visual and parietal cortex, suggesting that WM representations are insensitive to aperture biases during perception. Then, we visualized memory abstractions of stimuli using models of visual field map properties. Regardless of aperture biases, WM representations of both modulated gratings were recoded into a single oriented line. These results provide strong evidence that visual WM representations are abstractions of percepts, immune to perceptual aperture biases, and compel revisions of WM theory.

Emerging Theories of Allostatic-Interoceptive Overload in Neurodegeneration

Recent integrative multilevel models offer novel insights into the etiology and course of neurodegenerative conditions. The predictive coding of allostatic-interoception theory posits that the brain adapts to environmental demands by modulating internal bodily signals through the allostatic-interoceptive system. Specifically, a domain-general allostatic-interoceptive network exerts adaptive physiological control by fine-tuning initial top-down predictions and bottom-up peripheral signaling. In this context, adequate adaptation implies the minimization of prediction errors thereby optimizing energy expenditure. Abnormalities in top-down interoceptive predictions or peripheral signaling can trigger allostatic overload states, ultimately leading to dysregulated interoceptive and bodily systems (endocrine, immunological, circulatory, etc.). In this context, environmental stress, social determinants of health, and harmful exposomes (i.e., the cumulative life-course exposition to different environmental stressors) may interact with physiological and genetic factors, dysregulating allostatic interoception and precipitating neurodegenerative processes. We review the allostatic-interoceptive overload framework across different neurodegenerative diseases, particularly in the behavioral variant frontotemporal dementia (bvFTD). We describe how concepts of allostasis and interoception could be integrated with principles of predictive coding to explain how the brain optimizes adaptive responses, while maintaining physiological stability through feedback loops with multiple organismic systems. Then, we introduce the model of allostatic-interoceptive overload of bvFTD and discuss its implications for the understanding of pathophysiological and neurocognitive abnormalities in multiple neurodegenerative conditions.

Incorporating uncertainty within dynamic interoceptive learning

Introduction

Interoception, the perception of the internal state of the body, has been shown to be closely linked to emotions and mental health. Of particular interest are interoceptive learning processes that capture associations between environmental cues and body signals as a basis for making homeostatically relevant predictions about the future. One method of measuring respiratory interoceptive learning that has shown promising results is the Breathing Learning Task (BLT). While the original BLT required binary predictions regarding the presence or absence of an upcoming inspiratory resistance, here we extended this paradigm to capture continuous measures of prediction (un)certainty.

Methods

Sixteen healthy participants completed the continuous version of the BLT, where they were asked to predict the likelihood of breathing resistances on a continuous scale from 0.0 to 10.0. In order to explain participants' responses, a Rescorla-Wagner model of associative learning was combined with suitable observation models for continuous or binary predictions, respectively. For validation, we compared both models against corresponding null models and examined the correlation between observed and modeled predictions. The model was additionally extended to test whether learning rates differed according to stimuli valence. Finally, summary measures of prediction certainty as well as model estimates for learning rates were considered against interoceptive and mental health questionnaire measures.

Results

Our results demonstrated that the continuous model fits closely captured participant behavior using empirical data, and the binarised predictions showed excellent replicability compared to previously collected data. However, the model extension indicated that there were no significant differences between learning rates for negative (i.e. breathing resistance) and positive (i.e. no breathing resistance) stimuli. Finally, significant correlations were found between fatigue severity and both prediction certainty and learning rate, as well as between anxiety sensitivity and prediction certainty.

Discussion

These results demonstrate the utility of gathering enriched continuous prediction data in interoceptive learning tasks, and suggest that the updated BLT is a promising paradigm for future investigations into interoceptive learning and potential links to mental health.

Visual working memories are abstractions of percepts

Pioneering studies demonstrating that the contents of visual working memory (WM) can be decoded from the patterns of multivoxel activity in early visual cortex transformed not only how we study WM, but theories of how memories are stored. For instance, the ability to decode the orientation of memorized gratings is hypothesized to depend on the recruitment of the same neural encoding machinery used for perceiving orientations. However, decoding evidence cannot be used to test the so-called sensory recruitment hypothesis without understanding the underlying nature of what is being decoded. Although unknown during WM, during perception decoding the orientation of gratings does not simply depend on activities of orientation tuned neurons. Rather, it depends on complex interactions between the orientation of the grating, the aperture edges, and the topographic structure of the visual map. Here, our goals are to 1) test how these aperture biases described during perception may affect WM decoding, and 2) leverage carefully manipulated visual stimulus properties of gratings to test how sensory-like are WM codes. For memoranda, we used gratings multiplied by radial and angular modulators to generate orthogonal aperture biases despite having identical orientations. Therefore, if WM representations are simply maintained sensory representations, they would have similar aperture biases. If they are abstractions of sensory features, they would be unbiased and the modulator would have no effect on orientation decoding. Results indicated that fMRI patterns of delay period activity while maintaining the orientation of a grating with one modulator (eg, radial) were interchangeable with patterns while maintaining a grating with the other modulator (eg, angular). We found significant cross-classification in visual and parietal cortex, suggesting that WM representations are insensitive to aperture biases during perception. Then, we visualized memory abstractions of stimuli using a population receptive field model of the visual field maps. Regardless of aperture biases, WM representations of both modulated gratings were recoded into a single oriented line. These results provide strong evidence that visual WM representations are abstractions of percepts, immune to perceptual aperture biases, and compel revisions of WM theory.

A hole in a piece of cardboard and predictive brain: the incomprehension of modern art in the light of the predictive coding paradigm

Incomprehension of and resistance to contemporaneous art have been constant features in the development of modern art. The predictive coding framework can be used to analyse this response by outlining the difference between the misunderstanding of (i) contemporary conceptual/minimalist art and (ii) early modern avant-garde art and by elucidating their underlying cognitive mechanisms. In both of these cases, incomprehension and its behavioural consequences are tied to the failure of the optimal prediction error (PE) minimization that is involved in the perception of such art works. In the case of contemporary conceptual/minimalist art the failure stems from the fact that the encounter results in non-salient visual sensations and generates no PE. In early modern avant-garde art, the occasional inability of viewers to recognize pictorial content using new pictorial conventions reflected the absence of suitable priors to explain away ambiguous sensory data. The capacity to recognize pictorial content in modernist painting, as a prerequisite for a satisfying encounter with such works and ultimately a wider acceptance of new artistic styles, required an updating of a number of expectations in order to optimize the fit between priors and sensations, from low-level perceptual priors to the development of higher-level, culturally determined expectations. This article is part of the theme issue 'Art, aesthetics and predictive processing: theoretical and empirical perspectives'.

The promises and pitfalls of seizure phenomenology

The typical adult patient presenting with a first seizure has a normal clinical examination, uninformative investigations, and often has no witness to their episode. The assessing clinician, therefore, has one primary source of information to guide their assessment; the patient's experience. However, seizure phenomenology - the subjective seizure experience - has received relatively less attention by researchers than objective semiology or investigations. This essay reviews the clinical importance of seizure phenomenology, and the challenges clinicians face in eliciting accurate and clinically relevant descriptions of ictal experience. I conclude by discussing tools that clinicians may use to support the clinical application of seizure phenomenology, and exploring the subjectivity of epilepsy more broadly.

Reconstructing visual illusory experiences from human brain activity

Visual illusions provide valuable insights into the brain's interpretation of the world given sensory inputs. However, the precise manner in which brain activity translates into illusory experiences remains largely unknown. Here, we leverage a brain decoding technique combined with deep neural network (DNN) representations to reconstruct illusory percepts as images from brain activity. The reconstruction model was trained on natural images to establish a link between brain activity and perceptual features and then tested on two types of illusions: illusory lines and neon color spreading. Reconstructions revealed lines and colors consistent with illusory experiences, which varied across the source visual cortical areas. This framework offers a way to materialize subjective experiences, shedding light on the brain's internal representations of the world.

A selection and targeting framework of cortical locations for line-scanning fMRI

Depth-resolved functional magnetic resonance imaging (fMRI) is an emerging field growing in popularity given the potential of separating signals from different computational processes in cerebral cortex. Conventional acquisition schemes suffer from low spatial and temporal resolutions. Line-scanning methods allow depth-resolved fMRI by sacrificing spatial coverage to sample blood oxygenated level-dependent (BOLD) responses at ultra-high temporal and spatial resolution. For neuroscience applications, it is critical to be able to place the line accurately to (1) sample the right neural population and (2) target that neural population with tailored stimuli or tasks. To this end, we devised a multi-session framework where a target cortical location is selected based on anatomical and functional properties. The line is then positioned according to this information in a separate second session, and we tailor the experiment to focus on the target location. Anatomically, the precision of the line placement was confirmed by projecting a nominal representation of the acquired line back onto the surface. Functional estimates of neural selectivities in the line, as quantified by a visual population-receptive field model, resembled the target selectivities well for most subjects. This functional precision was quantified in detail by estimating the distance between the visual field location of the targeted vertex and the location in visual cortex (V1) that most closely resembled the line-scanning estimates; this distance was on average ~5.5 mm. Given the dimensions of the line, differences in acquisition, session, and stimulus design, this validates that line-scanning can be used to probe local neural sensitivities across sessions. In summary, we present an accurate framework for line-scanning MRI; we believe such a framework is required to harness the full potential of line-scanning and maximize its utility. Furthermore, this approach bridges canonical fMRI experiments with electrophysiological experiments, which in turn allows novel avenues for studying human physiology non-invasively.

Linguistic Priors for Perception

In this commentary, we approach the topic of linguistic relativity from a predictive coding perspective. Discussing the role of "priors" in shaping perception, we argue that language creates an important set of priors for humans, which can affect how sensory information is processed and interpreted. Namely, languages create conventionalized conceptual systems for their speakers, mirroring and reinforcing what is behaviorally important in a society. As such, they create collective conceptual convergence on how to categorize the world and thus "streamline" what people rely on to guide their perception.

Local minimization of prediction errors drives learning of invariant object representations in a generative network model of visual perception

The ventral visual processing hierarchy of the cortex needs to fulfill at least two key functions: perceived objects must be mapped to high-level representations invariantly of the precise viewing conditions, and a generative model must be learned that allows, for instance, to fill in occluded information guided by visual experience. Here, we show how a multilayered predictive coding network can learn to recognize objects from the bottom up and to generate specific representations via a top-down pathway through a single learning rule: the local minimization of prediction errors. Trained on sequences of continuously transformed objects, neurons in the highest network area become tuned to object identity invariant of precise position, comparable to inferotemporal neurons in macaques. Drawing on this, the dynamic properties of invariant object representations reproduce experimentally observed hierarchies of timescales from low to high levels of the ventral processing stream. The predicted faster decorrelation of error-neuron activity compared to representation neurons is of relevance for the experimental search for neural correlates of prediction errors. Lastly, the generative capacity of the network is confirmed by reconstructing specific object images, robust to partial occlusion of the inputs. By learning invariance from temporal continuity within a generative model, the approach generalizes the predictive coding framework to dynamic inputs in a more biologically plausible way than self-supervised networks with non-local error-backpropagation. This was achieved simply by shifting the training paradigm to dynamic inputs, with little change in architecture and learning rule from static input-reconstructing Hebbian predictive coding networks.

Neural population dynamics of human working memory

The activity of neurons in macaque prefrontal cortex (PFC) persists during working memory (WM) delays, providing a mechanism for memory.1,2,3,4,5,6,7,8,9,10,11 Although theory,11,12 including formal network models,13,14 assumes that WM codes are stable over time, PFC neurons exhibit dynamics inconsistent with these assumptions.15,16,17,18,19 Recently, multivariate reanalyses revealed the coexistence of both stable and dynamic WM codes in macaque PFC.20,21,22,23 Human EEG studies also suggest that WM might contain dynamics.24,25 Nonetheless, how WM dynamics vary across the cortical hierarchy and which factors drive dynamics remain unknown. To elucidate WM dynamics in humans, we decoded WM content from fMRI responses across multiple cortical visual field maps.26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48 We found coexisting stable and dynamic neural representations of WM during a memory-guided saccade task. Geometric analyses of neural subspaces revealed that early visual cortex exhibited stronger dynamics than high-level visual and frontoparietal cortex. Leveraging models of population receptive fields, we visualized and made the neural dynamics interpretable. We found that during WM delays, V1 population initially encoded a narrowly tuned bump of activation centered on the peripheral memory target. Remarkably, this bump then spread inward toward foveal locations, forming a vector along the trajectory of the forthcoming memory-guided saccade. In other words, the neural code transformed into an abstraction of the stimulus more proximal to memory-guided behavior. Therefore, theories of WM must consider both sensory features and their task-relevant abstractions because changes in the format of memoranda naturally drive neural dynamics.

Extrastriate activity reflects the absence of local retinal input

The physiological blind spot corresponds to the optic disc where the retina contains no light-detecting photoreceptor cells. Our perception seemingly fills in this gap in input. Here we suggest that rather than an active process, such perceptual filling-in could instead be a consequence of the integration of visual inputs at higher stages of processing discounting the local absence of retinal input. Using functional brain imaging, we resolved the retinotopic representation of the physiological blind spot in early human visual cortex and measured responses while participants perceived filling-in. Responses in early visual areas simply reflected the absence of visual input. In contrast, higher extrastriate regions responded more to stimuli in the eye containing the blind spot than the fellow eye. However, this signature was independent of filling-in. We argue that these findings agree with philosophical accounts that posit that the concept of filling-in of absent retinal input is unnecessary.

The role of awareness in shaping responses in human visual cortex

The visual cortex contains information about stimuli even when they are not consciously perceived. However, it remains unknown whether the visual system integrates local features into global objects without awareness. Here, we tested this by measuring brain activity in human observers viewing fragmented shapes that were either visible or rendered invisible by fast counterphase flicker. We then projected measured neural responses to these stimuli back into visual space. Visible stimuli caused robust responses reflecting the positions of their component fragments. Their neural representations also strongly resembled one another regardless of local features. By contrast, representations of invisible stimuli differed from one another and, crucially, also from visible stimuli. Our results demonstrate that even the early visual cortex encodes unconscious visual information differently from conscious information, presumably by only encoding local features. This could explain previous conflicting behavioural findings on unconscious visual processing.

Predictive neural representations of naturalistic dynamic input

Adaptive behavior such as social interaction requires our brain to predict unfolding external dynamics. While theories assume such dynamic prediction, empirical evidence is limited to static snapshots and indirect consequences of predictions. We present a dynamic extension to representational similarity analysis that uses temporally variable models to capture neural representations of unfolding events. We applied this approach to source-reconstructed magnetoencephalography (MEG) data of healthy human subjects and demonstrate both lagged and predictive neural representations of observed actions. Predictive representations exhibit a hierarchical pattern, such that high-level abstract stimulus features are predicted earlier in time, while low-level visual features are predicted closer in time to the actual sensory input. By quantifying the temporal forecast window of the brain, this approach allows investigating predictive processing of our dynamic world. It can be applied to other naturalistic stimuli (e.g., film, soundscapes, music, motor planning/execution, social interaction) and any biosignal with high temporal resolution.

Backward masking reveals coarse-to-fine dynamics in human V1

Natural images exhibit luminance variations aligned across a broad spectrum of spatial frequencies (SFs). It has been proposed that, at early stages of processing, the coarse signals carried by the low SF (LSF) of the visual input are sent rapidly from primary visual cortex (V1) to ventral, dorsal and frontal regions to form a coarse representation of the input, which is later sent back to V1 to guide the processing of fine-grained high SFs (i.e., HSF). We used functional resonance imaging (fMRI) to investigate the role of human V1 in the coarse-to-fine integration of visual input. We disrupted the processing of the coarse and fine content of full-spectrum human face stimuli via backward masking of selective SF ranges (LSFs: <1.75cpd and HSFs: >1.75cpd) at specific times (50, 83, 100 or 150ms). In line with coarse-to-fine proposals, we found that (1) the selective masking of stimulus LSF disrupted V1 activity in the earliest time window, and progressively decreased in influence, while (2) an opposite trend was observed for the masking of stimulus' HSF. This pattern of activity was found in V1, as well as in ventral (i.e. the Fusiform Face area, FFA), dorsal and orbitofrontal regions. We additionally presented subjects with contrast negated stimuli. While contrast negation significantly reduced response amplitudes in the FFA, as well as coupling between FFA and V1, coarse-to-fine dynamics were not affected by this manipulation. The fact that V1 response dynamics to strictly identical stimulus sets differed depending on the masked scale adds to growing evidence that V1 role goes beyond the early and quasi-passive transmission of visual information to the rest of the brain. It instead indicates that V1 may yield a 'spatially registered common forum' or 'blackboard' that integrates top-down inferences with incoming visual signals through its recurrent interaction with high-level regions located in the inferotemporal, dorsal and frontal regions.

Severely Attenuated Visual Feedback Processing in Children on the Autism Spectrum

Individuals on the autism spectrum often exhibit atypicality in their sensory perception, but the neural underpinnings of these perceptual differences remain incompletely understood. One proposed mechanism is an imbalance in higher-order feedback re-entrant inputs to early sensory cortices during sensory perception, leading to increased propensity to focus on local object features over global context. We explored this theory by measuring visual evoked potentials during contour integration as considerable work has revealed that these processes are largely driven by feedback inputs from higher-order ventral visual stream regions. We tested the hypothesis that autistic individuals would have attenuated evoked responses to illusory contours compared with neurotypical controls. Electrophysiology was acquired while 29 autistic and 31 neurotypical children (7-17 years old, inclusive of both males and females) passively viewed a random series of Kanizsa figure stimuli, each consisting of four inducers that were aligned either at random rotational angles or such that contour integration would form an illusory square. Autistic children demonstrated attenuated automatic contour integration over lateral occipital regions relative to neurotypical controls. The data are discussed in terms of the role of predictive feedback processes on perception of global stimulus features and the notion that weakened "priors" may play a role in the visual processing anomalies seen in autism.SIGNIFICANCE STATEMENT Children on the autism spectrum differ from typically developing children in many aspects of their processing of sensory stimuli. One proposed mechanism for these differences is an imbalance in higher-order feedback to primary sensory regions, leading to an increased focus on local object features rather than global context. However, systematic investigation of these feedback mechanisms remains limited. Using EEG and a visual illusion paradigm that is highly dependent on intact feedback processing, we demonstrated significant disruptions to visual feedback processing in children with autism. This provides much needed experimental evidence that advances our understanding of the contribution of feedback processing to visual perception in autism spectrum disorder.

Brain connectivity modulation by Bayesian surprise in relation to control demand drives cognitive flexibility via control engagement

Human control is characterized by its flexibility and adaptability in response to the conditional probability in the environment. Previous studies have revealed that efficient conflict control could be attained by predicting and adapting to the changing control demand. However, it is unclear whether cognitive flexibility could also be gained by predicting and adapting to the changing control demand. The present study aimed to explore this issue by combining the model-based analyses of behavioral and neuroimaging data with a probabilistic cued task switching paradigm. We demonstrated that the Bayesian surprise (i.e. unsigned precision-weighted prediction error [PE]) negatively modulated the connections among stimulus processing brain regions and control regions/networks. The effect of Bayesian surprise modulation on these connections guided control engagement as reflected by the control PE effect on behavior, which in turn facilitated cognitive flexibility. These results bridge a gap in the literature by illustrating the neural and behavioral effect of control demand prediction (or PE) on cognitive flexibility and offer novel insights into the source of switch cost and the mechanism of cognitive flexibility.

Contextual drive of neuronal responses in mouse V1 in the absence of feedforward input

Neurons in the primary visual cortex (V1) respond to stimuli in their receptive field (RF), which is defined by the feedforward input from the retina. However, V1 neurons are also sensitive to contextual information outside their RF, even if the RF itself is unstimulated. Here, we examined the cortical circuits for V1 contextual responses to gray disks superimposed on different backgrounds. Contextual responses began late and were strongest in the feedback-recipient layers of V1. They differed between the three main classes of inhibitory neurons, with particularly strong contextual drive of VIP neurons, indicating a contribution of disinhibitory circuits to contextual drive. Contextual drive was strongest when the gray disk was perceived as figure, occluding its background, rather than a hole. Our results link contextual drive in V1 to perceptual organization and provide previously unknown insight into how recurrent processing shapes the response of sensory neurons to facilitate figure perception.

Awareness-independent gradual spread of object-based attention

Although attention can be directed at certain objects, how object-based attention spreads within an object and whether this spread interacts with awareness remain unclear. Using a modified spatial cuing paradigm with backward masking, we addressed these issues with either visible or invisible displays presenting the real (Experiment 1) and illusory (Experiment 2) U-shaped objects (UOs), whose ends and middles, the possible locations of the cue and target, have iso-eccentric distances from the fixation. These equidistant ends and middles of UOs offered us a unique opportunity to examine whether attention gradually spreads within a given object, i.e., within an UO, attention spreads from its cued-end to uncued-end via the uncued-middle. Despite the visibility (visible or invisible) of UOs, both experiments supported this gradual spread manner by showing a faster response of human participants (male and female) to the target in the uncued-middle than that in the uncued-end. Our results thus indicate a gradual spread of object-based attention and further reveal that this gradual spread is independent of both the “visual objectness” (whether the object is defined as the real or illusory boundaries) and conscious access to objects.

Statistical Learning in Vision

Vision and learning have long been considered to be two areas of research linked only distantly. However, recent developments in vision research have changed the conceptual definition of vision from a signal-evaluating process to a goal-oriented interpreting process, and this shift binds learning, together with the resulting internal representations, intimately to vision. In this review, we consider various types of learning (perceptual, statistical, and rule/abstract) associated with vision in the past decades and argue that they represent differently specialized versions of the fundamental learning process, which must be captured in its entirety when applied to complex visual processes. We show why the generalized version of statistical learning can provide the appropriate setup for such a unified treatment of learning in vision, what computational framework best accommodates this kind of statistical learning, and what plausible neural scheme could feasibly implement this framework. Finally, we list the challenges that the field of statistical learning faces in fulfilling the promise of being the right vehicle for advancing our understanding of vision in its entirety. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Unveiling the abstract format of mnemonic representations

Working memory (WM) enables information storage for future use, bridging the gap between perception and behavior. We hypothesize that WM representations are abstractions of low-level perceptual features. However, the neural nature of these putative abstract representations has thus far remained impenetrable. Here, we demonstrate that distinct visual stimuli (oriented gratings and moving dots) are flexibly recoded into the same WM format in visual and parietal cortices when that representation is useful for memory-guided behavior. Specifically, the behaviorally relevant features of the stimuli (orientation and direction) were extracted and recoded into a shared mnemonic format that takes the form of an abstract line-like pattern. We conclude that mnemonic representations are abstractions of percepts that are more efficient than and proximal to the behaviors they guide.

Attention impedes neural representation of interpolated orientation during perceptual completion

One of the most remarkable functional feats accomplished by visual system is the interpolation of missing retinal inputs based on surrounding information, a process known as perceptual completion. Perceptual completion enables the active construction of coherent, vivid percepts from spatially discontinuous visual information that is prevalent in real-life visual scenes. Despite mounting evidence linking sensory activity enhancement and perceptual completion, surprisingly little is known about whether and how attention, a fundamental modulator of sensory activities, affects perceptual completion. Using EEG-based time-resolved inverted encoding model (IEM), we reconstructed the moment-to-moment representation of the illusory grating that resulted from spatially interpolating the orientation of surrounding inducers. We found that, despite manipulation of observers' attentional focus, the illusory grating representation unfolded in time in a similar manner. Critically, attention to the surrounding inducers simultaneously attenuated the illusory grating representation and delayed its temporal development. Our findings disclosed, for the first time, the suppressive role of selective attention in perceptual completion and were suggestive of a fast, automatic neural machinery that implements the interpolation of missing visual information.

Coding strategy for surface luminance switches in the primary visual cortex of the awake monkey

Both surface luminance and edge contrast of an object are essential features for object identification. However, cortical processing of surface luminance remains unclear. In this study, we aim to understand how the primary visual cortex (V1) processes surface luminance information across its different layers. We report that edge-driven responses are stronger than surface-driven responses in V1 input layers, but luminance information is coded more accurately by surface responses. In V1 output layers, the advantage of edge over surface responses increased eight times and luminance information was coded more accurately at edges. Further analysis of neural dynamics shows that such substantial changes for neural responses and luminance coding are mainly due to non-local cortical inhibition in V1’s output layers. Our results suggest that non-local cortical inhibition modulates the responses elicited by the surfaces and edges of objects, and that switching the coding strategy in V1 promotes efficient coding for luminance.

How brightness is encoded in the visual cortex remains incompletely understood. By recording from macaque V1, the authors revealed a switch from surface to edge encoding that is mediated by widespread inhibition in the output layers of the cortex.

Behavioral Prioritization Enhances Working Memory Precision and Neural Population Gain

Humans allocate visual working memory (WM) resource according to behavioral relevance, resulting in more precise memories for more important items. Theoretically, items may be maintained by feature-tuned neural populations, where the relative gain of the populations encoding each item determines precision. To test this hypothesis, we compared the amplitudes of delay period activity in the different parts of retinotopic maps representing each of several WM items, predicting the amplitudes would track behavioral priority. Using fMRI, we scanned participants while they remembered the location of multiple items over a WM delay and then reported the location of one probed item using a memory-guided saccade. Importantly, items were not equally probable to be probed (0.6, 0.3, 0.1, 0.0), which was indicated with a precue. We analyzed fMRI activity in 10 visual field maps in occipital, parietal, and frontal cortex known to be important for visual WM. In early visual cortex, but not association cortex, the amplitude of BOLD activation within voxels corresponding to the retinotopic location of visual WM items increased with the priority of the item. Interestingly, these results were contrasted with a common finding that higher-level brain regions had greater delay period activity, demonstrating a dissociation between the absolute amount of activity in a brain area and the activity of different spatially selective populations within it. These results suggest that the distribution of WM resources according to priority sculpts the relative gains of neural populations that encode items, offering a neural mechanism for how prioritization impacts memory precision.

OUP accepted manuscript

Working memory (WM) allows goal-relevant information to be encoded and maintained in mind, even when the contents of WM are incongruent with the immediate environment. While regions of heteromodal cortex are important for WM, the neural mechanisms that relate to individual differences in the encoding and maintenance of goal-relevant information remain unclear. Here, we used behavioral correlates of two large-scale heteromodal networks at rest, the default mode (DMN) and frontoparietal (FPN) networks, to understand their contributions to distinct features of WM. We assessed each individual’s ability to resist distracting information during the encoding and maintenance phases of a visuospatial WM task. Individuals with stronger connectivity of DMN with medial visual and retrosplenial cortex were less affected by encoding distraction. Conversely, weaker connectivity of both DMN and FPN with visual regions was associated with better WM performance when target information was no longer in the environment and distractors were presented in the maintenance phase. Our study suggests that stronger coupling between heteromodal cortex and visual–spatial regions supports WM encoding by reducing the influence of concurrently presented distractors, while weaker visual coupling is associated with better maintenance of goal-relevant information because it relates to the capacity to ignore task-irrelevant changes in the environment.

Feedback from lateral occipital cortex to V1/V2 triggers object completion: Evidence from functional magnetic resonance imaging and dynamic causal modeling

Illusory figures demonstrate the visual system's ability to integrate disparate parts into coherent wholes. We probed this object integration process by either presenting an integrated diamond shape or a comparable ungrouped configuration that did not render a complete object. Two tasks were used that either required localization of a target dot (relative to the presented configuration) or discrimination of the dot's luminance. The results showed that only when the configuration was task relevant (in the localization task), performance benefited from the presentation of an integrated object. Concurrent functional magnetic resonance imaging was performed and analyzed using dynamic causal modeling to investigate the (causal) relationship between regions that are associated with illusory figure completion. We found object‐specific feedback connections between the lateral occipital cortex (LOC) and early visual cortex (V1/V2). These modulatory connections persisted across task demands and hemispheres. Our results thus provide direct evidence that interactions between mid‐level and early visual processing regions engage in illusory figure perception. These data suggest that LOC first integrates inputs from multiple neurons in lower‐level cortices, generating a global shape representation while more fine‐graded object details are then determined via feedback to early visual areas, independently of the current task demands.

Optimizing the strength of the Bourdon effect by varying the triangle arrangement

The Bourdon illusion refers to the perceived bentness of the straight collinear edges when two right-angled triangles are placed apex to apex. We studied this illusion using a cancellation method. In the first of three experiments, we manipulated the apex angle, with six different angles ranging from 4° to 45°. Results indicated that the Bourdon illusion is strongest when the angle is around 12°. In the second experiment, we compared four scalene triangles with a right-angled triangle. The angular shift was most salient when the shape corresponded to a right-angled triangle. In the third experiment, the patterns were created by varying the size of one right-angled triangle while holding the size of the second right-angled triangle constant. Results indicated that the Bourdon illusion was strongest when both right-angled triangles were of equal size. Our data suggest that the Bourdon illusion depends critically upon the specific arrangement of shapes in the display.

Gestalts at threshold could reveal Gestalts as predictions

We review and revisit the predictive processing inspired “Gestalts as predictions” hypothesis. The study of Gestalt phenomena at and below threshold can help clarify the role of higher-order object selective areas and feedback connections in mid-level vision. In two psychophysical experiments assessing manipulations of contrast and configurality we showed that: (1) Gestalt phenomena are robust against saliency manipulations across the psychometric function even below threshold (with the accuracy gains and higher saliency associated with Gestalts being present even around chance performance); and (2) peak differences between Gestalt and control conditions happened around the time where responses to Gestalts are starting to saturate (mimicking the differential contrast response profile of striate vs. extra-striate visual neurons). In addition, Gestalts are associated with steeper psychometric functions in all experiments. We propose that these results reflect the differential engagement of object-selective areas in Gestalt phenomena and of information- or percept-based processing, as opposed to energy- or stimulus-based processing, more generally. In addition, the presence of nonlinearities in the psychometric functions suggest differential top-down modulation of the early visual cortex. We treat this as a proof of principle study, illustrating that classic psychophysics can help assess possible involvement of hierarchical predictive processing in Gestalt phenomena.

A bias in saccadic suppression of shape change

Processing of visual information in the central (foveal) and peripheral visual field is vastly different. To achieve a homogeneous representation of the visual world across eye movements, the visual system needs to compensate for these differences. By introducing subtle changes between peripheral and foveal inputs across saccades, one can test this compensation. We morphed shapes between a triangle and a circle and presented two different change directions (circularity decrease or increase) at varying magnitudes across a saccade. In a change-discrimination task, observers disproportionally often reported percepts of circularity increase. To test the relationship with visual-field differences, we measured perception when shapes were exclusively presented either in the periphery (before a saccade), or in the fovea (after a saccade). We found that overall shapes were perceived as more circular before than after a saccade and the more pronounced this difference was for a participant, the smaller was their circularity-increase bias in the change-discrimination task. We propose that visual-field differences have a direct and an indirect influence on transsaccadic perception of shape change. The direct influence is based on the distinct appearance of shape in the central and peripheral visual field in a trial, causing an increase of the perceptual magnitude of circularity-decrease changes. The indirect influence is based on long-term build-up of transsaccadic expectations; if a change is opposite (circularity increase) to the expectation (circularity decrease), it should elicit a strong error signal facilitating change detection. We discuss the concept of transsaccadic expectations and theoretical implications for transsaccadic perception of other feature changes.

Amodal completion instead of predictive coding can explain activity suppression of early visual cortex during illusory shape perception

A set of recent neuroimaging studies observed that the perception of an illusory shape can elicit both positive and negative feedback modulations in different parts of the early visual cortex. When three Pac-Men shapes were aligned in such a way that they created an illusory triangle (i.e., the Kanizsa illusion), neural activity in early visual cortex was enhanced in those neurons that had receptive fields that overlapped with the illusory shape but suppressed in neurons whose receptive field overlapped with the Pac-Men inducers. These results were interpreted as congruent with the predictive coding framework, in which neurons in early visual cortex enhance or suppress their activity depending on whether the top-down predictions match the bottom-up sensory inputs. However, there are several plausible alternative explanations for the activity modulations. Here we tested a recent proposal (Moors, 2015) that the activity suppression in early visual cortex during illusory shape perception reflects neural adaptation to perceptually stable input. Namely, the inducers appear perceptually stable during the illusory shape condition (discs on which a triangle is superimposed), but not during the control condition (discs that change into Pac-Men). We examined this hypothesis by manipulating the perceptual stability of inducers. When the inducers could be perceptually interpreted as persistent circles, we replicated the up- and downregulation pattern shown in previous studies. However, when the inducers could not be perceived as persistent circles, we still observed enhanced activity in neurons representing the illusory shape but the suppression of activity in neurons representing the inducers was absent. Thus our results support the hypothesis that the activity suppression in neurons representing the inducers during the Kanizsa illusion is better explained by neural adaptation to perceptually stable input than by reduced prediction error.

Brain network mechanisms of visual shape completion

Visual shape completion recovers object shape, size, and number from spatially segregated edges. Despite being extensively investigated, the process’s underlying brain regions, networks, and functional connections are still not well understood. To shed light on the topic, we scanned (fMRI) healthy adults during rest and during a task in which they discriminated pac-man configurations that formed or failed to form completed shapes (illusory and fragmented condition, respectively). Task activation differences (illusory-fragmented), resting-state functional connectivity, and multivariate patterns were identified on the cortical surface using 360 predefined parcels and 12 functional networks composed of such parcels. Brain activity flow mapping (ActFlow) was used to evaluate the likely involvement of resting-state connections for shape completion. We identified 36 differentially-active parcels including a posterior temporal region, PH, whose activity was consistent across 95% of observers. Significant task regions primarily occupied the secondary visual network but also incorporated the frontoparietal dorsal attention, default mode, and cingulo-opercular networks. Each parcel’s task activation difference could be modeled via its resting-state connections with the remaining parcels (r=.62, p<10⁻⁹), suggesting that such connections undergird shape completion. Functional connections from the dorsal attention network were key in modelling task activation differences in the secondary visual network. Dorsal attention and frontoparietal connections could also model activations in the remaining networks. Taken together, these results suggest that shape completion relies upon a sparsely distributed but densely interconnected network coalition that is centered in the secondary visual network, coordinated by the dorsal attention network, and inclusive of at least three other networks.

Zooming-in on higher-level vision: High-resolution fMRI for understanding visual perception and awareness

One of the central questions in visual neuroscience is how the sparse retinal signals leaving our eyes are transformed into a rich subjective visual experience of the world. Invasive physiology studies, which offers the highest spatial resolution, have revealed many facts about the processing of simple visual features like contrast, color, and orientation, focusing on the early visual areas. At the same time, standard human fMRI studies with comparably coarser spatial resolution have revealed more complex, functionally specialized, and category-selective responses in higher visual areas. Although the visual system is the best understood among the sensory modalities, these two areas of research remain largely segregated. High-resolution fMRI opens up a possibility for linking them. On the one hand, it allows studying how the higher-level visual functions affect the fine-scale activity in early visual areas. On the other hand, it allows discovering the fine-scale functional organization of higher visual areas and exploring their functional connectivity with visual areas lower in the hierarchy. In this review, I will discuss examples of successful work undertaken in these directions using high-resolution fMRI and discuss where this method could be applied in the future to advance our understanding of the complexity of higher-level visual processing.

Cooperation and Social Rules Emerging From the Principle of Surprise Minimization

The surprise minimization principle has been applied to explain various cognitive processes in humans. Originally describing perceptual and active inference, the framework has been applied to different types of decision making including long-term policies, utility maximization and exploration. This analysis extends the application of surprise minimization (also known as free energy principle) to a multi-agent setup and shows how it can explain the emergence of social rules and cooperation. We further show that in social decision-making and political policy design, surprise minimization is superior in many aspects to the classical approach of maximizing utility. Surprise minimization shows directly what value freedom of choice can have for social agents and why, depending on the context, they enter into cooperation, agree on social rules, or do nothing of the kind.

Shape coding in occipito-temporal cortex relies on object silhouette, curvature, and medial axis

Object recognition relies on different transformations of the retinal input, carried out by the visual system, that range from local contrast to object shape and category. While some of those transformations are thought to occur at specific stages of the visual hierarchy, the features they represent are correlated (e.g., object shape and identity) and selectivity for the same feature overlaps in many brain regions. This may be explained either by collinearity across representations or may instead reflect the coding of multiple dimensions by the same cortical population. Moreover, orthogonal and shared components may differently impact distinctive stages of the visual hierarchy. We recorded functional MRI activity while participants passively attended to object images and employed a statistical approach that partitioned orthogonal and shared object representations to reveal their relative impact on brain processing. Orthogonal shape representations (silhouette, curvature, and medial axis) independently explained distinct and overlapping clusters of selectivity in the occitotemporal and parietal cortex. Moreover, we show that the relevance of shared representations linearly increases moving from posterior to anterior regions. These results indicate that the visual cortex encodes shared relations between different features in a topographic fashion and that object shape is encoded along different dimensions, each representing orthogonal features.NEW & NOTEWORTHY There are several possible ways of characterizing the shape of an object. Which shape description better describes our brain responses while we passively perceive objects? Here, we employed three competing shape models to explain brain representations when viewing real objects. We found that object shape is encoded in a multidimensional fashion and thus defined by the interaction of multiple features.

Object Selection by Automatic Spreading of Top-Down Attentional Signals in V1

What is selected when attention is directed to a specific location of the visual field? Theories of object-based attention have suggested that when spatial attention is directed to part of an object, attention does not simply enhance the attended location but automatically spreads to enhance all locations that comprise the object. Here, we tested this hypothesis by reconstructing the distribution of attention from primary visual cortex (V1) population neuronal activity patterns in 24 human adults (17 female) using functional magnetic resonance imaging (fMRI) and population-based receptive field (prf) mapping. We find that attention spreads from a spatially cued location to the underlying object, and enhances all spatial locations that comprise the object. Importantly, this spreading was also evident when the object was not task relevant. These data suggest that attentional selection automatically operates at an object level, facilitating the reconstruction of coherent objects from fragmented representations in early visual cortex.SIGNIFICANCE STATEMENT Object perception is an astonishing feat of the visual system. When visual information about orientation, shape, and color enters through our eyes, it has yet to be integrated into a coherent representation of an object. But which visual features constitute a single object and which features belong to the background? The brain mechanisms underpinning object perception are yet to be understood. We now demonstrate that one candidate mechanism, the successive activation of all parts of an object, occurs in early visual cortex and results in a detailed representation of the object following Gestalt principles. Furthermore, our results suggest that object selection occurs automatically, without involving voluntary control.

Aberrant Cortical Connectivity During Ambiguous Object Recognition Is Associated With Schizophrenia

BACKGROUND

Dysfunctional connectivity within the perceptual hierarchy is proposed to be an integral component of psychosis. The fragmented ambiguous object task was implemented to investigate neural connectivity during object recognition in patients with schizophrenia (SCZ) and bipolar disorder and first-degree relatives of patients with SCZ (SREL).

METHODS

We analyzed 3T functional magnetic resonance imaging data collected from 27 patients with SCZ, 23 patients with bipolar disorder, 24 control subjects, and 19 SREL during the administration of the fragmented ambiguous object task. Fragmented ambiguous object task stimuli were line-segmented versions of objects and matched across a number of low-level features. Images were categorized as meaningful or meaningless based on ratings assigned by the participants.

RESULTS

An a priori region of interest was defined in the primary visual cortex (V1). In addition, the lateral occipital complex/ventral visual areas, intraparietal sulcus (IPS), and middle frontal gyrus (MFG) were identified functionally via the contrast of cortical responses to stimuli judged as meaningful or meaningless. SCZ was associated with altered neural activations at V1, IPS, and MFG. Psychophysiological interaction analyses revealed negative connectivity between V1 and MFG in patient groups and altered modulation of connectivity between conditions from right IPS to left IPS and right IPS to left MFG in patients with SCZ and SREL.

CONCLUSIONS

Results provide evidence that SCZ is associated with inefficient processing of ambiguous visual objects at V1, which is likely attributable to altered feedback from higher-level visual areas. We also observed distinct patterns of aberrant connectivity among low-level, mid-level, and high-level visual areas in patients with SCZ, patients with bipolar disorder, and SREL.

Feedback contribution to surface motion perception in the human early visual cortex

Human visual surface perception has neural correlates in early visual cortex, but the role of feedback during surface segmentation in human early visual cortex remains unknown. Feedback projections preferentially enter superficial and deep anatomical layers, which provides a hypothesis for the cortical depth distribution of fMRI activity related to feedback. Using ultra-high field fMRI, we report a depth distribution of activation in line with feedback during the (illusory) perception of surface motion. Our results fit with a signal re-entering in superficial depths of V1, followed by a feedforward sweep of the re-entered information through V2 and V3. The magnitude and sign of the BOLD response strongly depended on the presence of texture in the background, and was additionally modulated by the presence of illusory motion perception compatible with feedback. In summary, the present study demonstrates the potential of depth-resolved fMRI in tackling biomechanical questions on perception.

Topographic signatures of global object perception in human visual cortex

Our visual system readily groups dynamic fragmented input into global objects. How the brain represents global object perception remains however unclear. To address this question, we recorded brain responses using functional magnetic resonance imaging whilst observers viewed a dynamic bistable stimulus that could either be perceived globally (i.e., as a grouped and coherently moving shape) or locally (i.e., as ungrouped and incoherently moving elements). We further estimated population receptive fields and used these to back-project the brain activity measured during stimulus perception into visual space via a searchlight procedure. Global perception resulted in universal suppression of responses in lower visual cortex accompanied by wide-spread enhancement in higher object-sensitive cortex. However, follow-up experiments indicated that higher object-sensitive cortex is suppressed if global perception lacks shape grouping, and that grouping-related suppression can be diffusely confined to stimulated sites and accompanied by background enhancement once stimulus size is reduced. These results speak to a non-generic involvement of higher object-sensitive cortex in perceptual grouping and point to an enhancement-suppression mechanism mediating the perception of figure and ground.

•

Lower visual cortex activity to grouped vs ungrouped dynamic stimuli is suppressed.

•

When grouping a shape, activity in higher object-sensitive cortex is enhanced.

•

Without shape grouping, activity in higher object-sensitive cortex is suppressed.

•

Grouping-related suppression can be diffusely confined to stimulated cortical sites.

Encoding of Auditory Temporal Gestalt in the Human Brain

The perception of an acoustic rhythm is invariant to the absolute temporal intervals constituting a sound sequence. It is unknown where in the brain temporal Gestalt, the percept emerging from the relative temporal proximity between acoustic events, is encoded. Two different relative temporal patterns, each induced by three experimental conditions with different absolute temporal patterns as sensory basis, were presented to participants. A linear support vector machine classifier was trained to differentiate activation patterns in functional magnetic resonance imaging data to the two different percepts. Across the sensory constituents the classifier decoded which percept was perceived. A searchlight analysis localized activation patterns specific to the temporal Gestalt bilaterally to the temporoparietal junction, including the planum temporale and supramarginal gyrus, and unilaterally to the right inferior frontal gyrus (pars opercularis). We show that auditory areas not only process absolute temporal intervals, but also integrate them into percepts of Gestalt and that encoding of these percepts persists in high-level associative areas. The findings complement existing knowledge regarding the processing of absolute temporal patterns to the processing of relative temporal patterns relevant to the sequential binding of perceptual elements into Gestalt.

Resolving the Spatial Profile of Figure Enhancement in Human V1 through Population Receptive Field Modeling

The detection and segmentation of meaningful figures from their background is one of the primary functions of vision. While work in nonhuman primates has implicated early visual mechanisms in this figure-ground modulation, neuroimaging in humans has instead largely ascribed the processing of figures and objects to higher stages of the visual hierarchy. Here, we used high-field fMRI at 7 Tesla to measure BOLD responses to task-irrelevant orientation-defined figures in human early visual cortex (N = 6, four females). We used a novel population receptive field mapping-based approach to resolve the spatial profiles of two constituent mechanisms of figure-ground modulation: a local boundary response, and a further enhancement spanning the full extent of the figure region that is driven by global differences in features. Reconstructing the distinct spatial profiles of these effects reveals that figure enhancement modulates responses in human early visual cortex in a manner consistent with a mechanism of automatic, contextually driven feedback from higher visual areas.SIGNIFICANCE STATEMENT A core function of the visual system is to parse complex 2D input into meaningful figures. We do so constantly and seamlessly, both by processing information about visible edges and by analyzing large-scale differences between figure and background. While influential neurophysiology work has characterized an intriguing mechanism that enhances V1 responses to perceptual figures, we have a poor understanding of how the early visual system contributes to figure-ground processing in humans. Here, we use advanced computational analysis methods and high-field human fMRI data to resolve the distinct spatial profiles of local edge and global figure enhancement in the early visual system (V1 and LGN); the latter is distinct and consistent with a mechanism of automatic, stimulus-driven feedback from higher-level visual areas.

Evaluating the neurophysiological evidence for predictive processing as a model of perception

For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long-standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.

Tracking the completion of parts into whole objects: Retinotopic activation in response to illusory figures in the lateral occipital complex

Illusory figures demonstrate the visual system's ability to integrate separate parts into coherent, whole objects. The present study was performed to track the neuronal object construction process in human observers, by incrementally manipulating the grouping strength within a given configuration until the emergence of a whole-object representation. Two tasks were employed: First, in the spatial localization task, object completion could facilitate performance and was task-relevant, whereas it was irrelevant in the second, luminance discrimination task. Concurrent functional magnetic resonance imaging (fMRI) used spatial localizers to locate brain regions representing task-critical illusory-figure parts to investigate whether the step-wise object construction process would modulate neural activity in these localized brain regions. The results revealed that both V1 and the lateral occipital complex (LOC, with sub-regions LO1 and LO2) were involved in Kanizsa figure processing. However, completion-specific activations were found predominantly in LOC, where neural activity exhibited a modulation in accord with the configuration's grouping strength, whether or not the configuration was relevant to performing the task at hand. Moreover, right LOC activations were confined to LO2 and responded primarily to surface and shape completions, whereas left LOC exhibited activations in both LO1 and LO2 and was related to encoding shape structures with more detail. Together, these results demonstrate that various grouping properties within a visual scene are integrated automatically in LOC, with sub-regions located in different hemispheres specializing in the component sub-processes that render completed objects.

A roadmap to integrate astrocytes into Systems Neuroscience

Systems neuroscience is still mainly a neuronal field, despite the plethora of evidence supporting the fact that astrocytes modulate local neural circuits, networks, and complex behaviors. In this article, we sought to identify which types of studies are necessary to establish whether astrocytes, beyond their well-documented homeostatic and metabolic functions, perform computations implementing mathematical algorithms that sub-serve coding and higher-brain functions. First, we reviewed Systems-like studies that include astrocytes in order to identify computational operations that these cells may perform, using Ca2+ transients as their encoding language. The analysis suggests that astrocytes may carry out canonical computations in a time scale of subseconds to seconds in sensory processing, neuromodulation, brain state, memory formation, fear, and complex homeostatic reflexes. Next, we propose a list of actions to gain insight into the outstanding question of which variables are encoded by such computations. The application of statistical analyses based on machine learning, such as dimensionality reduction and decoding in the context of complex behaviors, combined with connectomics of astrocyte-neuronal circuits, is, in our view, fundamental undertakings. We also discuss technical and analytical approaches to study neuronal and astrocytic populations simultaneously, and the inclusion of astrocytes in advanced modeling of neural circuits, as well as in theories currently under exploration such as predictive coding and energy-efficient coding. Clarifying the relationship between astrocytic Ca2+ and brain coding may represent a leap forward toward novel approaches in the study of astrocytes in health and disease.

Prediction errors in surface segmentation are reflected in the visual mismatch negativity, independently of task and surface features

The visual system quickly registers perceptual regularities in the environment and responds to violations in these patterns. Errors of perceptual prediction are associated with electrocortical modulation, including the visual mismatch negativity (vMMN) and P2 event-related potential. One relatively unexplored question is whether these prediction error signals can encode higher-level properties such as surface segmentation, or whether they are limited to lower-level perceptual features. Using a roving standard paradigm, a triangle surface appeared either behind (featuring amodal contours) or in front of (featuring real contours) a second surface with hole-like windows. A surface layout appeared for two to five repetitions before switching to the other "deviant" layout; lighting and orientation of stimuli varied across presentations while remaining isoluminant. Observers responded when they detected a rare "pinched" triangle, which occasionally appeared. Cortical activity-reflected in mismatch responses affecting the P2-N2 and P300 amplitudes-was sensitive to a change in stimulus layout, when surfaces shifted position in depth, following several repetitions. Specifically, layout deviants led to a more negative P2-N2 complex at posterior electrodes, and greater P300 positivity at central sites. Independently of these signals of a deviant surface layout, further modulations of the P2 encoded differences between layouts and detection of the rare target stimulus. Comparison of the effect of preceding layout repetitions on this prediction error signal suggests that it is all or none and not graded with respect to the number of previous repetitions. We show that within the visual domain, unnoticed and task-irrelevant changes in visual surface segmentation leads to observable electrophysiological signals of prediction error that are dissociable from stimulus-specific encoding and lower-level perceptual processing.

Top-Down Feedback Controls the Cortical Representation of Illusory Contours in Mouse Primary Visual Cortex

Visual systems have evolved to recognize and extract features from complex scenes using limited sensory information. Contour perception is essential to this process and can occur despite breaks in the continuity of neighboring features. Such robustness of the animal visual system to degraded or occluded shapes may also give rise to an interesting phenomenon of optical illusions. These illusions provide a great opportunity to decipher neural computations underlying contour integration and object detection. Kanizsa illusory contours have been shown to evoke responses in the early visual cortex despite the lack of direct receptive field activation. Recurrent processing between visual areas has been proposed to be involved in this process. However, it is unclear whether higher visual areas directly contribute to the generation of illusory responses in the early visual cortex. Using behavior, in vivo electrophysiology, and optogenetics, we first show that the primary visual cortex (V1) of male mice responds to Kanizsa illusory contours. Responses to Kanizsa illusions emerge later than the responses to the contrast-defined real contours in V1. Second, we demonstrate that illusory responses are orientation-selective. Finally, we show that top-down feedback controls the neural correlates of illusory contour perception in V1. Our results suggest that higher-order visual areas may fill in the missing information in the early visual cortex necessary for illusory contour perception.SIGNIFICANCE STATEMENT Perception of the Kanizsa illusory contours is impaired in neurodevelopmental disorders such as schizophrenia, autism, and Williams syndrome. However, the mechanism of the illusory contour perception is poorly understood. Here we describe the behavioral and neural correlates of Kanizsa illusory contours perception in mice, a genetically tractable model system. We show that top-down feedback controls the neural responses to Kanizsa illusion in V1. To our knowledge, this is the first description of the neural correlates of the Kanizsa illusion in mice and the first causal demonstration of their regulation by top-down feedback.

The hippocampus as a visual area organized by space and time: A spatiotemporal similarity hypothesis

The hippocampus is the canonical memory system in the brain and is not typically considered part of the visual system. Yet, it sits atop the ventral visual stream and has been implicated in certain aspects of vision. Here I review the place of the hippocampal memory system in vision science. After a brief primer on the local circuity, external connectivity, and computational functions of the hippocampus, I explore what can be learned from each field about the other. I first present four areas of vision science (scene perception, imagery, eye movements, attention) that challenge our current understanding of the hippocampus in terms of its role in episodic memory. In the reverse direction, I leverage this understanding to inform vision science in other ways, presenting a working hypothesis about a unique form of visual representation. This spatiotemporal similarity hypothesis states that the hippocampus represents objects according to whether they co-occur in space and/or time, and not whether they look alike, as elsewhere in the visual system. This tuning may reflect hippocampal mechanisms of pattern separation, relational binding, and statistical learning, allowing the hippocampus to generate visual expectations to facilitate search and recognition.