Delayed action does not always require the ventral stream: A study on a patient with visual form agnosia

A computational examination of the two-streams hypothesis: which pathway needs a longer memory?

The two visual streams hypothesis is a robust example of neural functional specialization that has inspired countless studies over the past four decades. According to one prominent version of the theory, the fundamental goal of the dorsal visual pathway is the transformation of retinal information for visually-guided motor behavior. To that end, the dorsal stream processes input using absolute (or veridical) metrics only when the movement is initiated, necessitating very little, or no, memory. Conversely, because the ventral visual pathway does not involve motor behavior (its output does not influence the real world), the ventral stream processes input using relative (or illusory) metrics and can accumulate or integrate sensory evidence over long time constants, which provides a substantial capacity for memory. In this study, we tested these relations between functional specialization, processing metrics, and memory by training identical recurrent neural networks to perform either a viewpoint-invariant object classification task or an orientation/size determination task. The former task relies on relative metrics, benefits from accumulating sensory evidence, and is usually attributed to the ventral stream. The latter task relies on absolute metrics, can be computed accurately in the moment, and is usually attributed to the dorsal stream. To quantify the amount of memory required for each task, we chose two types of neural network models. Using a long-short-term memory (LSTM) recurrent network, we found that viewpoint-invariant object categorization (object task) required a longer memory than orientation/size determination (orientation task). Additionally, to dissect this memory effect, we considered factors that contributed to longer memory in object tasks. First, we used two different sets of objects, one with self-occlusion of features and one without. Second, we defined object classes either strictly by visual feature similarity or (more liberally) by semantic label. The models required greater memory when features were self-occluded and when object classes were defined by visual feature similarity, showing that self-occlusion and visual similarity among object task samples are contributing to having a long memory. The same set of tasks modeled using modified leaky-integrator echo state recurrent networks (LiESN), however, did not replicate the results, except under some conditions. This may be because LiESNs cannot perform fine-grained memory adjustments due to their network-wide memory coefficient and fixed recurrent weights. In sum, the LSTM simulations suggest that longer memory is advantageous for performing viewpoint-invariant object classification (a putative ventral stream function) because it allows for interpolation of features across viewpoints. The results further suggest that orientation/size determination (a putative dorsal stream function) does not benefit from longer memory. These findings are consistent with the two visual streams theory of functional specialization.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s11571-021-09703-z.

Card posting does not rely on visual orientation: A challenge to past neuropsychological dissociations

A common set of tasks frequently employed in the neuropsychological assessment of patients with visuomotor or perceptual deficits are the card-posting and the perceptual orientation matching tasks. In the posting task, patients have to post a card (or their hand) through a slot of varying orientations while the matching task requires them to indicate the slot's orientation as accurately as possible. Observations that damage to different areas of the brain (dorsal vs. ventral stream) is associated with selective impairment in one of the tasks - but not the other - has led to the suggestion that different cortical pathways process visual orientation information for perception versus action. In three experiments, we show that this conclusion may be premature as posting does not seem to rely on the processing of visual orientation information but is instead performed using obstacle avoidance strategies that require an accurate judgement of egocentric distances between the card's and the slot's edges. Specifically, we found that while matching is susceptible to the oblique effect (i.e., common perceptual orientation bias with higher accuracy for cardinal than oblique orientations), this was not the case for posting, neither in immediate nor in memory-guided conditions. In contrast to matching, posting errors primarily depended on biomechanical demands and reflected a preference for performing efficient and comfortable movements. Thus, we suggest that previous dissociations between perceptual and visuomotor performance in letter posting tasks are better explained by impairments in egocentric and allocentric spatial processing than by independent visual processing systems.

Spatial updating of allocentric landmark information in real-time and memory-guided reaching

The 2-streams model of vision suggests that egocentric and allocentric reference frames are utilized by the dorsal and the ventral stream for real-time and memory-guided movements, respectively. Recent studies argue against such a strict functional distinction and suggest that real-time and memory-guided movements recruit the same spatial maps. In this study we focus on allocentric spatial coding and updating of targets by using landmark information in real-time and memory-guided reaching. We presented participants with a naturalistic scene which consisted of six objects on a table that served as potential reach targets. Participants were informed about the target object after scene encoding, and were prompted by a go cue to reach to its position. After target identification a brief air-puff was applied to the participant's right eye inducing an eye blink. During the blink the target object disappeared from the scene, and in half of the trials the remaining objects, that functioned as landmarks, were shifted horizontally in the same direction. We found that landmark shifts systematically influenced participants' reaching endpoints irrespective of whether the movements were controlled online based on available target information (real-time movement) or memory-guided based on remembered target information (memory-guided movement). Overall, the effect of landmark shift was stronger for memory-guided than real-time reaching. Our findings suggest that humans can encode and update reach targets in an allocentric reference frame for both real-time and memory-guided movements and show stronger allocentric coding when the movement is based on memory.

Information generation as a functional basis of consciousness

What is the biological advantage of having consciousness? Functions of consciousness have been elusive due to the subjective nature of consciousness and ample empirical evidence showing the presence of many nonconscious cognitive performances in the human brain. Drawing upon empirical literature, here, we propose that a core function of consciousness be the ability to internally generate representations of events possibly detached from the current sensory input. Such representations are constructed by generative models learned through sensory-motor interactions with the environment. We argue that the ability to generate information underlies a variety of cognitive functions associated with consciousness such as intention, imagination, planning, short-term memory, attention, curiosity, and creativity, all of which contribute to non-reflexive behavior. According to this view, consciousness emerged in evolution when organisms gained the ability to perform internal simulations using internal models, which endowed them with flexible intelligent behavior. To illustrate the notion of information generation, we take variational autoencoders (VAEs) as an analogy and show that information generation corresponds the decoding (or decompression) part of VAEs. In biological brains, we propose that information generation corresponds to top-down predictions in the predictive coding framework. This is compatible with empirical observations that recurrent feedback activations are linked with consciousness whereas feedforward processing alone seems to occur without evoking conscious experience. Taken together, the information generation hypothesis captures many aspects of existing ideas about potential functions of consciousness and provides new perspectives on the functional roles of consciousness.

Biologically-Inspired Computational Neural Mechanism for Human Action/activity Recognition: A Review

Theoretical neuroscience investigation shows valuable information on the mechanism for recognizing the biological movements in the mammalian visual system. This involves many different fields of researches such as psychological, neurophysiology, neuro-psychological, computer vision, and artificial intelligence (AI). The research on these areas provided massive information and plausible computational models. Here, a review on this subject is presented. This paper describes different perspective to look at this task including action perception, computational and knowledge based modeling, psychological, and neuroscience approaches.

Comparing the effect of temporal delay on the availability of egocentric and allocentric information in visual search

Frames of reference play a central role in perceiving an object's location and reaching to pick that object up. It is thought that the ventral stream, believed to subserve vision for perception, utilises allocentric coding, while the dorsal stream, argued to be responsible for vision for action, primarily uses an egocentric reference frame. We have previously shown that egocentric representations can survive a delay; however, it is possible that in comparison to allocentric information, egocentric information decays more rapidly. Here we directly compare the effect of delay on the availability of egocentric and allocentric representations. We used spatial priming in visual search and repeated the location of the target relative to either a landmark in the search array (allocentric condition) or the observer's body (egocentric condition). Three inter-trial intervals created minimum delays between two consecutive trials of 2, 4, or 8seconds. In both conditions, search times to primed locations were faster than search times to un-primed locations. In the egocentric condition the effects were driven by a reduction in search times when egocentric information was repeated, an effect that was observed at all three delays. In the allocentric condition while search times did not change when the allocentric information was repeated, search times to un-primed target locations became slower. We conclude that egocentric representations are not as transient as previously thought but instead this information is still available, and can influence behaviour, after lengthy periods of delay. We also discuss the possible origins of the differences between allocentric and egocentric priming effects.

The Pointing Errors in Optic Ataxia Reveal the Role of "Peripheral Magnification" of the PPC

Interaction with visual objects in the environment requires an accurate correspondence between visual space and its internal representation within the brain. Many clinical conditions involve some impairment in visuo-motor control and the errors created by the lesion of a specific brain region are neither random nor uninformative. Modern approaches to studying the neuropsychology of action require powerful data-driven analyses and error modeling in order to understand the function of the lesioned areas. In the present paper we carried out mixed-effect analyses of the pointing errors of seven optic ataxia patients and seven control subjects. We found that a small parameter set is sufficient to explain the pointing errors produced by unilateral optic ataxia patients. In particular, the extremely stereotypical errors made when pointing toward the contralesional visual field can be fitted by mathematical models similar to those used to model central magnification in cortical or sub-cortical structure(s). Our interpretation is that visual areas that contain this footprint of central magnification guide pointing movements when the posterior parietal cortex (PPC) is damaged and that the functional role of the PPC is to actively compensate for the under-representation of peripheral vision that accompanies central magnification. Optic ataxia misreaching reveals what would be hand movement accuracy and precision if the human motor system did not include elaborated corrective processes for reaching and grasping to non-foveated targets.

The Two Visual Systems Hypothesis: New Challenges and Insights from Visual form Agnosic Patient DF

Patient DF, who developed visual form agnosia following carbon monoxide poisoning, is still able to use vision to adjust the configuration of her grasping hand to the geometry of a goal object. This striking dissociation between perception and action in DF provided a key piece of evidence for the formulation of Goodale and Milner's Two Visual Systems Hypothesis (TVSH). According to the TVSH, the ventral stream plays a critical role in constructing our visual percepts, whereas the dorsal stream mediates the visual control of action, such as visually guided grasping. In this review, we discuss recent studies of DF that provide new insights into the functional organization of the dorsal and ventral streams. We confirm recent evidence that DF has dorsal as well as ventral brain damage - and that her dorsal-stream lesions and surrounding atrophy have increased in size since her first published brain scan. We argue that the damage to DF's dorsal stream explains her deficits in directing actions at targets in the periphery. We then focus on DF's ability to accurately adjust her in-flight hand aperture to changes in the width of goal objects (grip scaling) whose dimensions she cannot explicitly report. An examination of several studies of DF's grip scaling under natural conditions reveals a modest though significant deficit. Importantly, however, she continues to show a robust dissociation between form vision for perception and form vision-for-action. We also review recent studies that explore the role of online visual feedback and terminal haptic feedback in the programming and control of her grasping. These studies make it clear that DF is no more reliant on visual or haptic feedback than are neurologically intact individuals. In short, we argue that her ability to grasp objects depends on visual feedforward processing carried out by visuomotor networks in her dorsal stream that function in the much the same way as they do in neurologically intact individuals.