The Role of Feedback in a Hierarchical Model of Object Perception

Laminar functional magnetic resonance imaging in vision research

Magnetic resonance imaging (MRI) scanners at ultra-high magnetic fields have become available to use in humans, thus enabling researchers to investigate the human brain in detail. By increasing the spatial resolution, ultra-high field MR allows both structural and functional characterization of cortical layers. Techniques that can differentiate cortical layers, such as histological studies and electrode-based measurements have made critical contributions to the understanding of brain function, but these techniques are invasive and thus mainly available in animal models. There are likely to be differences in the organization of circuits between humans and even our closest evolutionary neighbors. Thus research on the human brain is essential. Ultra-high field MRI can observe differences between cortical layers, but is non-invasive and can be used in humans. Extensive previous literature has shown that neuronal connections between brain areas that transmit feedback and feedforward information terminate in different layers of the cortex. Layer-specific functional MRI (fMRI) allows the identification of layer-specific hemodynamic responses, distinguishing feedback and feedforward pathways. This capability has been particularly important for understanding visual processing, as it has allowed researchers to test hypotheses concerning feedback and feedforward information in visual cortical areas. In this review, we provide a general overview of successful ultra-high field MRI applications in vision research as examples of future research.

Brain-inspired Multimodal Learning Based on Neural Networks

Modern computational models have leveraged biological advances in human brain research. This study addresses the problem of multimodal learning with the help of brain-inspired models. Specifically, a unified multimodal learning architecture is proposed based on deep neural networks, which are inspired by the biology of the visual cortex of the human brain. This unified framework is validated by two practical multimodal learning tasks: image captioning, involving visual and natural language signals, and visual-haptic fusion, involving haptic and visual signals. Extensive experiments are conducted under the framework, and competitive results are achieved.

Sounds enhance visual completion processes

Everyday vision includes the detection of stimuli, figure-ground segregation, as well as object localization and recognition. Such processes must often surmount impoverished or noisy conditions; borders are perceived despite occlusion or absent contrast gradients. These illusory contours (ICs) are an example of so-called mid-level vision, with an event-related potential (ERP) correlate at ∼100-150 ms post-stimulus onset and originating within lateral-occipital cortices (the IC_effect). Presently, visual completion processes supporting IC perception are considered exclusively visual; any influence from other sensory modalities is currently unknown. It is now well-established that multisensory processes can influence both low-level vision (e.g. detection) as well as higher-level object recognition. By contrast, it is unknown if mid-level vision exhibits multisensory benefits and, if so, through what mechanisms. We hypothesized that sounds would impact the IC_effect. We recorded 128-channel ERPs from 17 healthy, sighted participants who viewed ICs or no-contour (NC) counterparts either in the presence or absence of task-irrelevant sounds. The IC_effect was enhanced by sounds and resulted in the recruitment of a distinct configuration of active brain areas over the 70-170 ms post-stimulus period. IC-related source-level activity within the lateral occipital cortex (LOC), inferior parietal lobe (IPL), as well as primary visual cortex (V1) were enhanced by sounds. Moreover, the activity in these regions was correlated when sounds were present, but not when absent. Results from a control experiment, which employed amodal variants of the stimuli, suggested that sounds impact the perceived brightness of the IC rather than shape formation per se. We provide the first demonstration that multisensory processes augment mid-level vision and everyday visual completion processes, and that one of the mechanisms is brightness enhancement. These results have important implications for the design of treatments and/or visual aids for low-vision patients.

Brain mechanisms for perceiving illusory lines in humans

Illusory contours (ICs) are perceptions of visual borders despite absent contrast gradients. The psychophysical and neurobiological mechanisms of IC processes have been studied across species and diverse brain imaging/mapping techniques. Nonetheless, debate continues regarding whether IC sensitivity results from a (presumably) feedforward process within low-level visual cortices (V1/V2) or instead are processed first within higher-order brain regions, such as lateral occipital cortices (LOC). Studies in animal models, which generally favour a feedforward mechanism within V1/V2, have typically involved stimuli inducing IC lines. By contrast, studies in humans generally favour a mechanism where IC sensitivity is mediated by LOC and have typically involved stimuli inducing IC forms or shapes. Thus, the particular stimulus features used may strongly contribute to the model of IC sensitivity supported. To address this, we recorded visual evoked potentials (VEPs) while presenting human observers with an array of 10 inducers within the central 5°, two of which could be oriented to induce an IC line on a given trial. VEPs were analysed using an electrical neuroimaging framework. Sensitivity to the presence vs. absence of centrally-presented IC lines was first apparent at ∼200 ms post-stimulus onset and was evident as topographic differences across conditions. We also localized these differences to the LOC. The timing and localization of these effects are consistent with a model of IC sensitivity commencing within higher-level visual cortices. We propose that prior observations of effects within lower-tier cortices (V1/V2) are the result of feedback from IC sensitivity that originates instead within higher-tier cortices (LOC).

Pain-Related Expectation and Prediction Error Signals in the Anterior Insula Are Not Related to Aversiveness

The anterior insula has repeatedly been linked to the experience of aversive stimuli, such as pain. Previously, we showed that the anterior insula is involved in the integration of pain intensity and its prior expectation. However, it is unclear whether this integration occurs by a pain-specific expectation or a more general expectation of an aversive event. To dissociate these possibilities, we conducted an experiment using painful stimuli and aversive pictures with three levels of aversiveness on human male volunteers. Stimuli were preceded by a probabilistic, combined modality and intensity cue in a full factorial design. Subjective ratings of pain intensity and skin conductance responses were best explained by a combination of actual pain intensity and expected pain intensity. In addition, using fMRI, we investigated the neuronal implementation of the integration of prior expectation and pain intensity. Similar to subjective ratings and autonomic responses, the dorsal anterior insula represented pain intensity and expectations. The ventral anterior insula additionally represented the absolute difference of the two terms (i.e., the prediction error). The posterior insula only represented pain intensity. Importantly, the pattern observed in the anterior insula was only present if the cued modality was correct (i.e., expect pain); in case of an incorrect modality cue (i.e., expect aversive picture), the ventral anterior insula simply represented pain intensity. The stimulus expectation and prediction error specificity in the ventral anterior insula indicates the integration of expectation with painful stimuli in this area. Importantly, this pattern cannot be explained by aversiveness.SIGNIFICANCE STATEMENT The anterior insula has been shown to integrate pain intensity and their expectation. However, it is unclear whether this integration is pain-specific or related more generally to an aversive event. To address this, we combined painful stimuli and aversive pictures with three levels of aversiveness. The ventral anterior insula represented pain intensity, expectation, and their absolute difference (i.e., the prediction error). Importantly, this pattern was only observed if the cued modality was correct. In case of an incorrect modality cue, this area simply represented as pain intensity. The stimulus expectation and prediction error specificity in the ventral anterior insula indicates the integration of expectation with painful stimuli in this area. Importantly, this pattern cannot be explained by aversiveness.

'Visual' parsing can be taught quickly without visual experience during critical periods

Cases of invasive sight-restoration in congenital blind adults demonstrated that acquiring visual abilities is extremely challenging, presumably because visual-experience during critical-periods is crucial for learning visual-unique concepts (e.g. size constancy). Visual rehabilitation can also be achieved using sensory-substitution-devices (SSDs) which convey visual information non-invasively through sounds. We tested whether one critical concept – visual parsing, which is highly-impaired in sight-restored patients – can be learned using SSD. To this end, congenitally blind adults participated in a unique, relatively short (~70 hours), SSD-‘vision’ training. Following this, participants successfully parsed 2D and 3D visual objects. Control individuals naïve to SSDs demonstrated that while some aspects of parsing with SSD are intuitive, the blind’s success could not be attributed to auditory processing alone. Furthermore, we had a unique opportunity to compare the SSD-users’ abilities to those reported for sight-restored patients who performed similar tasks visually, and who had months of eyesight. Intriguingly, the SSD-users outperformed the patients on most criteria tested. These suggest that with adequate training and technologies, key high-order visual features can be quickly acquired in adulthood, and lack of visual-experience during critical-periods can be somewhat compensated for. Practically, these highlight the potential of SSDs as standalone-aids or combined with invasive restoration approaches.

Late electrophysiological modulations of feature-based attention to object shapes

Feature-based attention has been shown to aid object perception. Our previous ERP effects revealed temporally late feature-based modulation in response to objects relative to motion. The aim of the current study was to confirm the timing of feature-based influences on object perception while cueing within the feature dimension of shape. Participants were told to expect either "pillow" or "flower" objects embedded among random white and black lines. Participants more accurately reported the object's main color for valid compared to invalid shapes. ERPs revealed modulation from 252-502 ms, from occipital to frontal electrodes. Our results are consistent with previous findings examining the time course for processing similar stimuli (illusory contours). Our results provide novel insights into how attending to features of higher complexity aids object perception presumably via feed-forward and feedback mechanisms along the visual hierarchy.

Illusory contours: a window onto the neurophysiology of constructing perception

Seeing seems effortless, despite the need to segregate and integrate visual information that varies in quality, quantity, and location. The extent to which seeing passively recapitulates the external world is challenged by phenomena such as illusory contours, an example of visual completion whereby borders are perceived despite their physical absence in the image. Instead, visual completion and seeing are increasingly conceived as active processes, dependent on information exchange across neural populations. How this is instantiated in the brain remains controversial. Divergent models emanate from single-unit and population-level electrophysiology, neuroimaging, and neurostimulation studies. We reconcile discrepant findings from different methods and disciplines, and underscore the importance of taking into account spatiotemporal brain dynamics in generating models of brain function and perception.