Cortical dynamics subserving visual apparent motion

Response retention and apparent motion effect in visual cortex models

Apparent motion is a visual illusion in which stationary stimuli, flashing in distinct spatial locations at certain time intervals, are perceived as one stimulus moving between these locations. In the primary visual cortex, apparent-motion stimuli produce smooth spatio-temporal patterns of activity similar to those produced by continuously moving stimuli. An important prerequisite for producing such activity patterns is prolongation of responses to brief stimuli. Indeed, a brief stimulus can evoke in the visual cortex a long response, outlasting the stimulus by hundreds of milliseconds. Here we use firing-rate based models with simple ring structure, and biologically-detailed conductance-based refractory density (CBRD) model with retinotopic space representation to analyze the response retention and the origin of smooth profiles of activity in response to apparent-motion stimuli. We show that the strength of recurrent connectivity is the major factor that endorses neuronal networks with the ability for response retention. The same strengths of recurrent connections mediate the appearance of bump attractor in the ring models. Factors such as synaptic depression, NMDA receptor mediated currents, and conductances regulating spike adaptation influence response retention, but cannot substitute for the weakness of recurrent connections to reproduce response retention in models with weak connectivity. However, the weakness of lateral recurrent connections can be compensated by layering: in multi-layer models even with weaker connections the activity retains due to its feedforward propagation from layer to layer. Using CBRD model with retinotopic space representation we further show that smooth spatio-temporal profiles of activity in response to apparent-motion stimuli are produced in the models expressing response retention, but not in the models that fail to produce response retention. Together, these results demonstrate a link between response retention and the ability of neuronal networks to generate spatio-temporal patterns of activity, which are compatible with perception of apparent motion.

How far neuroscience is from understanding brains

The cellular biology of brains is relatively well-understood, but neuroscientists have not yet generated a theory explaining how brains work. Explanations of how neurons collectively operate to produce what brains can do are tentative and incomplete. Without prior assumptions about the brain mechanisms, I attempt here to identify major obstacles to progress in neuroscientific understanding of brains and central nervous systems. Most of the obstacles to our understanding are conceptual. Neuroscience lacks concepts and models rooted in experimental results explaining how neurons interact at all scales. The cerebral cortex is thought to control awake activities, which contrasts with recent experimental results. There is ambiguity distinguishing task-related brain activities from spontaneous activities and organized intrinsic activities. Brains are regarded as driven by external and internal stimuli in contrast to their considerable autonomy. Experimental results are explained by sensory inputs, behavior, and psychological concepts. Time and space are regarded as mutually independent variables for spiking, post-synaptic events, and other measured variables, in contrast to experimental results. Dynamical systems theory and models describing evolution of variables with time as the independent variable are insufficient to account for central nervous system activities. Spatial dynamics may be a practical solution. The general hypothesis that measurements of changes in fundamental brain variables, action potentials, transmitter releases, post-synaptic transmembrane currents, etc., propagating in central nervous systems reveal how they work, carries no additional assumptions. Combinations of current techniques could reveal many aspects of spatial dynamics of spiking, post-synaptic processing, and plasticity in insects and rodents to start with. But problems defining baseline and reference conditions hinder interpretations of the results. Furthermore, the facts that pooling and averaging of data destroy their underlying dynamics imply that single-trial designs and statistics are necessary.

Backward masking reveals coarse-to-fine dynamics in human V1

Natural images exhibit luminance variations aligned across a broad spectrum of spatial frequencies (SFs). It has been proposed that, at early stages of processing, the coarse signals carried by the low SF (LSF) of the visual input are sent rapidly from primary visual cortex (V1) to ventral, dorsal and frontal regions to form a coarse representation of the input, which is later sent back to V1 to guide the processing of fine-grained high SFs (i.e., HSF). We used functional resonance imaging (fMRI) to investigate the role of human V1 in the coarse-to-fine integration of visual input. We disrupted the processing of the coarse and fine content of full-spectrum human face stimuli via backward masking of selective SF ranges (LSFs: <1.75cpd and HSFs: >1.75cpd) at specific times (50, 83, 100 or 150ms). In line with coarse-to-fine proposals, we found that (1) the selective masking of stimulus LSF disrupted V1 activity in the earliest time window, and progressively decreased in influence, while (2) an opposite trend was observed for the masking of stimulus' HSF. This pattern of activity was found in V1, as well as in ventral (i.e. the Fusiform Face area, FFA), dorsal and orbitofrontal regions. We additionally presented subjects with contrast negated stimuli. While contrast negation significantly reduced response amplitudes in the FFA, as well as coupling between FFA and V1, coarse-to-fine dynamics were not affected by this manipulation. The fact that V1 response dynamics to strictly identical stimulus sets differed depending on the masked scale adds to growing evidence that V1 role goes beyond the early and quasi-passive transmission of visual information to the rest of the brain. It instead indicates that V1 may yield a 'spatially registered common forum' or 'blackboard' that integrates top-down inferences with incoming visual signals through its recurrent interaction with high-level regions located in the inferotemporal, dorsal and frontal regions.

Using the perceptual past to predict the perceptual future influences the perceived present - A novel ERP paradigm

The information available through our senses is noisy, incomplete, and to varying degrees ambiguous. The perceptual system must create stable and reliable percepts out of this restricted information. It solves this perceptual inference problem by integrating memories of previous percepts and making predictions about the perceptual future.

Using ambiguous figures and a new experimental approach, we studied whether generating predictions based on regularities in the past affects processing of the present and how this is done. Event-related potentials (ERPs) were measured to investigate whether a highly regular temporal context of either ambiguous or unambiguous stimulus variants differently affects processing of a current stimulus and/or task execution. Further, we tested whether symbolic announcements about the immediate perceptual future can replace the past experience of regularities as a source for making predictions. Both ERP and reaction time varied as a function of stimulus ambiguity in the temporal context of a present stimulus. No such effects were found with symbolic announcements.

Our results indicate that predictions about the future automatically alter processing of the present, even if the predictions are irrelevant for the present percept and task. However, direct experiences of past regularities are necessary for predicting the future whereas symbolic information about the future is not sufficient.

Suppressive Traveling Waves Shape Representations of Illusory Motion in Primary Visual Cortex of Awake Primate

How does the brain link visual stimuli across space and time? Visual illusions provide an experimental paradigm to study these processes. When two stationary dots are flashed in close spatial and temporal succession, human observers experience a percept of apparent motion. Large spatiotemporal separation challenges the visual system to keep track of object identity along the apparent motion path, the so-called "correspondence problem." Here, we use voltage-sensitive dye imaging in primary visual cortex (V1) of awake monkeys to show that intracortical connections within V1 can solve this issue by shaping cortical dynamics to represent the illusory motion. We find that the appearance of the second stimulus in V1 creates a systematic suppressive wave traveling toward the retinotopic representation of the first. Using a computational model, we show that the suppressive wave is the emergent property of a recurrent gain control fed by the intracortical network. This suppressive wave acts to explain away ambiguous correspondence problems and contributes to precisely encode the expected motion velocity at the surface of V1. Together, these results demonstrate that the nonlinear dynamics within retinotopic maps can shape cortical representations of illusory motion. Understanding these dynamics will shed light on how the brain links sensory stimuli across space and time, by preformatting population responses for a straightforward read-out by downstream areas.SIGNIFICANCE STATEMENT Traveling waves have recently been observed in different animal species, brain areas, and behavioral states. However, it is still unclear what are their functional roles. In the case of cortical visual processing, waves propagate across retinotopic maps and can hereby generate interactions between spatially and temporally separated instances of feedforward driven activity. Such interactions could participate in processing long-range apparent motion stimuli, an illusion for which no clear neuronal mechanisms have yet been proposed. Using this paradigm in awake monkeys, we show that suppressive traveling waves produce a spatiotemporal normalization of apparent motion stimuli. Our study suggests that cortical waves shape the representation of illusory moving stimulus within retinotopic maps for a straightforward read-out by downstream areas.

Cortical travelling waves: mechanisms and computational principles

Multichannel recording technologies have revealed travelling waves of neural activity in multiple sensory, motor and cognitive systems. These waves can be spontaneously generated by recurrent circuits or evoked by external stimuli. They travel along brain networks at multiple scales, transiently modulating spiking and excitability as they pass. Here, we review recent experimental findings that have found evidence for travelling waves at single-area (mesoscopic) and whole-brain (macroscopic) scales. We place these findings in the context of the current theoretical understanding of wave generation and propagation in recurrent networks. During the large low-frequency rhythms of sleep or the relatively desynchronized state of the awake cortex, travelling waves may serve a variety of functions, from long-term memory consolidation to processing of dynamic visual stimuli. We explore new avenues for experimental and computational understanding of the role of spatiotemporal activity patterns in the cortex.

Predictive feedback to V1 dynamically updates with sensory input

Predictive coding theories propose that the brain creates internal models of the environment to predict upcoming sensory input. Hierarchical predictive coding models of vision postulate that higher visual areas generate predictions of sensory inputs and feed them back to early visual cortex. In V1, sensory inputs that do not match the predictions lead to amplified brain activation, but does this amplification process dynamically update to new retinotopic locations with eye-movements? We investigated the effect of eye-movements in predictive feedback using functional brain imaging and eye-tracking whilst presenting an apparent motion illusion. Apparent motion induces an internal model of motion, during which sensory predictions of the illusory motion feed back to V1. We observed attenuated BOLD responses to predicted stimuli at the new post-saccadic location in V1. Therefore, pre-saccadic predictions update their retinotopic location in time for post-saccadic input, validating dynamic predictive coding theories in V1.

Space-Time Dynamics of Membrane Currents Evolve to Shape Excitation, Spiking, and Inhibition in the Cortex at Small and Large Scales

In the cerebral cortex, membrane currents, i.e., action potentials and other membrane currents, express many forms of space-time dynamics. In the spontaneous asynchronous irregular state, their space-time dynamics are local non-propagating fluctuations and sparse spiking appearing at unpredictable positions. After transition to active spiking states, larger structured zones with active spiking neurons appear, propagating through the cortical network, driving it into various forms of widespread excitation, and engaging the network from microscopic scales to whole cortical areas. At each engaged cortical site, the amount of excitation in the network, after a delay, becomes matched by an equal amount of space-time fine-tuned inhibition that might be instrumental in driving the dynamics toward perception and action.

Catching the voltage gradient-asymmetric boost of cortical spread generates motion signals across visual cortex: a brief review with special thanks to Amiram Grinvald

Wide-field voltage imaging is unique in its capability to capture snapshots of activity-across the full gradient of average changes in membrane potentials from subthreshold to suprathreshold levels-of hundreds of thousands of superficial cortical neurons that are simultaneously active. Here, I highlight two examples where voltage-sensitive dye imaging (VSDI) was exploited to track gradual space-time changes of activity within milliseconds across several millimeters of cortex at submillimeter resolution: the line-motion condition, measured in Amiram Grinvald's Laboratory more than 10 years ago and-coming full circle running VSDI in my laboratory-another motion-inducing condition, in which two neighboring stimuli counterchange luminance simultaneously. In both examples, cortical spread is asymmetrically boosted, creating suprathreshold activity drawn out over primary visual cortex. These rapidly propagating waves may integrate brain signals that encode motion independent of direction-selective circuits.

Improving voltage-sensitive dye imaging: with a little help from computational approaches

Voltage-sensitive dye imaging (VSDI) is a key neurophysiological recording tool because it reaches brain scales that remain inaccessible to other techniques. The development of this technique from in vitro to the behaving nonhuman primate has only been made possible thanks to the long-lasting, visionary work of Amiram Grinvald. This work has opened new scientific perspectives to the great benefit to the neuroscience community. However, this unprecedented technique remains largely under-utilized, and many future possibilities await for VSDI to reveal new functional operations. One reason why this tool has not been used extensively is the inherent complexity of the signal. For instance, the signal reflects mainly the subthreshold neuronal population response and is not linked to spiking activity in a straightforward manner. Second, VSDI gives access to intracortical recurrent dynamics that are intrinsically complex and therefore nontrivial to process. Computational approaches are thus necessary to promote our understanding and optimal use of this powerful technique. Here, we review such approaches, from computational models to dissect the mechanisms and origin of the recorded signal, to advanced signal processing methods to unravel new neuronal interactions at mesoscopic scale. Only a stronger development of interdisciplinary approaches can bridge micro- to macroscales.

Evidence for long-range spatiotemporal interactions in infant and adult visual cortex

The development of spatiotemporal interactions giving rise to classical receptive field properties has been well studied in animal models, but little is known about the development of putative nonclassical mechanisms in any species. Here we used visual evoked potentials to study the developmental status of spatiotemporal interactions for stimuli that were biased to engage long-range spatiotemporal integration mechanisms. We compared responses to widely spaced stimuli presented either in temporal succession or at the same time. The former configuration elicits a percept of apparent motion in adults but the latter does not. Component flash responses were summed to make a linear prediction (no spatiotemporal interaction) for comparison with the measured evoked responses to sequential or simultaneous flash conditions. In adults, linear summation of the separate flash responses measured with 40% contrast stimuli predicted sequential flash responses twice as large as those measured, indicating that the response measured under apparent motion conditions is subadditive. Simultaneous-flash responses at the same spatial separation were also subadditive, but substantially less so. The subadditivity in both cases could be modeled as a simple multiplicative gain term across all electrodes and time points. In infants aged 3-8 months, responses to the stimuli used in adults were similar to their linear predictions at 40%, but the responses measured at 80% contrast resembled the subadditive responses of the adults for both sequential and simultaneous flash conditions. We interpret the developmental data as indicating that adult-like long-range spatiotemporal interactions can be demonstrated by 3-8 months, once stimulus contrast is high enough.

Breaking the Excitation-Inhibition Balance Makes the Cortical Network's Space-Time Dynamics Distinguish Simple Visual Scenes

Brain dynamics are often taken to be temporal dynamics of spiking and membrane potentials in a balanced network. Almost all evidence for a balanced network comes from recordings of cell bodies of few single neurons, neglecting more than 99% of the cortical network. We examined the space-time dynamics of excitation and inhibition simultaneously in dendrites and axons over four visual areas of ferrets exposed to visual scenes with stationary and moving objects. The visual stimuli broke the tight balance between excitation and inhibition such that the network exhibited longer episodes of net excitation subsequently balanced by net inhibition, in contrast to a balanced network. Locally in all four areas the amount of net inhibition matched the amount of net excitation with a delay of 125 ms. The space-time dynamics of excitation-inhibition evolved to reduce the complexity of neuron interactions over the whole network to a flow on a low-(3)-dimensional manifold within 80 ms. In contrast to the pure temporal dynamics, the low dimensional flow evolved to distinguish the simple visual scenes.

Intrinsic signal optical imaging of visual brain activity: Tracking of fast cortical dynamics

Hemodynamic-based brain imaging techniques are typically incapable of monitoring brain activity with both high spatial and high temporal resolutions. In this study, we have used intrinsic signal optical imaging (ISOI), a relatively high spatial resolution imaging technique, to examine the temporal resolution of the hemodynamic signal. We imaged V1 responses in anesthetized monkey to a moving light spot. Movies of cortical responses clearly revealed a focus of hemodynamic response traveling across the cortical surface. Importantly, at different locations along the cortical trajectory, response timecourses maintained a similar tri-phasic shape and shifted sequentially across cortex with a predictable delay. We calculated the time between distinguishable timecourses and found that the temporal resolution of the signal at which two events can be reliably distinguished is about 80 milliseconds. These results suggest that hemodynamic-based imaging is suitable for detecting ongoing cortical events at high spatial resolution and with temporal resolution relevant for behavioral studies.

Apparent Motion Suppresses Responses in Early Visual Cortex: A Population Code Model

Two stimuli alternately presented at different locations can evoke a percept of a stimulus continuously moving between the two locations. The neural mechanism underlying this apparent motion (AM) is thought to be increased activation of primary visual cortex (V1) neurons tuned to locations along the AM path, although evidence remains inconclusive. AM masking, which refers to the reduced detectability of stimuli along the AM path, has been taken as evidence for AM-related V1 activation. AM-induced neural responses are thought to interfere with responses to physical stimuli along the path and as such impair the perception of these stimuli. However, AM masking can also be explained by predictive coding models, predicting that responses to stimuli presented on the AM path are suppressed when they match the spatio-temporal prediction of a stimulus moving along the path. In the present study, we find that AM has a distinct effect on the detection of target gratings, limiting the maximum performance at high contrast levels. This masking is strongest when the target orientation is identical to the orientation of the inducers. We developed a V1-like population code model of early visual processing, based on a standard contrast normalization model. We find that AM-related activation in early visual cortex is too small to either cause masking or to be perceived as motion. Our model instead predicts strong suppression of early sensory responses during AM, consistent with the theoretical framework of predictive coding.

Two spatially separate stimuli presented in rapid succession often induce the illusory perception of a moving stimulus (apparent motion or AM). Its underlying mechanism is thought to be increased activation in primary visual cortex representing the motion path. Indirect evidence for this account comes from the reduced detectability of stimuli presented along the motion path (AM masking). Here, we developed a computational model of AM-related effects on visual processing in early visual cortex, which predicted a neural activation that is too small to either account for the observed masking or the perception of motion. Instead, our model predicts strong suppression of neural responses to stimuli presented along the motion path, especially when they match the spatio-temporal prediction of a stimulus moving along the path. Our findings support predictive coding models of visual processing, in which higher-level predictions about motion explain away lower-level responses to expected sensory input.

Decoding information about dynamically occluded objects in visual cortex

During dynamic occlusion, an object passes behind an occluding surface and then later reappears. Even when completely occluded from view, such objects are experienced as continuing to exist or persist behind the occluder even though they are no longer visible. The contents and neural basis of this persistent representation remain poorly understood. Questions remain as to whether there is information maintained about the object itself (i.e. its shape or identity) or non-object-specific information such as its position or velocity as it is tracked behind an occluder, as well as which areas of visual cortex represent such information. Recent studies have found that early visual cortex is activated by "invisible" objects during visual imagery and by unstimulated regions along the path of apparent motion, suggesting that some properties of dynamically occluded objects may also be neurally represented in early visual cortex. We applied functional magnetic resonance imaging in human subjects to examine representations within visual cortex during dynamic occlusion. For gradually occluded, but not for instantly disappearing objects, there was an increase in activity in early visual cortex (V1, V2, and V3). This activity was spatially-specific, corresponding to the occluded location in the visual field. However, the activity did not encode enough information about object identity to discriminate between different kinds of occluded objects (circles vs. stars) using MVPA. In contrast, object identity could be decoded in spatially-specific subregions of higher-order, topographically organized areas such as ventral, lateral, and temporal occipital areas (VO, LO, and TO) as well as the functionally defined LOC and hMT+. These results suggest that early visual cortex may only represent the dynamically occluded object's position or motion path, while later visual areas represent object-specific information.

Temporal Asymmetry in Dark-Bright Processing Initiates Propagating Activity across Primary Visual Cortex

Differences between visual pathways representing darks and lights have been shown to affect spatial resolution and detection timing. Both psychophysical and physiological studies suggest an underlying retinal origin with amplification in primary visual cortex (V1). Here we show that temporal asymmetries in the processing of darks and lights create motion in terms of propagating activity across V1. Exploiting the high spatiotemporal resolution of voltage-sensitive dye imaging, we captured population responses to abrupt local changes of luminance in cat V1. For stimulation we used two neighboring small squares presented on either bright or dark backgrounds. When a single square changed from dark to bright or vice versa, we found coherent population activity emerging at the respective retinal input locations. However, faster rising and decay times were obtained for the bright to dark than the dark to bright changes. When the two squares changed luminance simultaneously in opposite polarities, we detected a propagating wave front of activity that originated at the cortical location representing the darkened square and rapidly expanded toward the region representing the brightened location. Thus, simultaneous input led to sequential activation across cortical retinotopy. Importantly, this effect was independent of the squares' contrast with the background. We suggest imbalance in dark-bright processing as a driving force in the generation of wave-like activity. Such propagation may convey motion signals and influence perception of shape whenever abrupt shifts in visual objects or gaze cause counterchange of luminance at high-contrast borders.

SIGNIFICANCE STATEMENT

An elementary process in vision is the detection of darks and lights through the retina via ON and OFF channels. Psychophysical and physiological studies suggest that differences between these channels affect spatial resolution and detection thresholds. Here we show that temporal asymmetries in the processing of darks and lights create motion signals across visual cortex. Using two neighboring squares, which simultaneously counterchanged luminance, we discovered propagating activity that was strictly drawn out from cortical regions representing the darkened location. Thus, a synchronous stimulus event translated into sequential wave-like brain activation. Such propagation may convey motion signals accessible in higher brain areas, whenever abrupt shifts in visual objects or gaze cause counterchange of luminance at high-contrast borders.

Reconstructing representations of dynamic visual objects in early visual cortex

As raw sensory data are partial, our visual system extensively fills in missing details, creating enriched percepts based on incomplete bottom-up information. Despite evidence for internally generated representations at early stages of cortical processing, it is not known whether these representations include missing information of dynamically transforming objects. Long-range apparent motion (AM) provides a unique test case because objects in AM can undergo changes both in position and in features. Using fMRI and encoding methods, we found that the "intermediate" orientation of an apparently rotating grating, never presented in the retinal input but interpolated during AM, is reconstructed in population-level, feature-selective tuning responses in the region of early visual cortex (V1) that corresponds to the retinotopic location of the AM path. This neural representation is absent when AM inducers are presented simultaneously and when AM is visually imagined. Our results demonstrate dynamic filling-in in V1 for object features that are interpolated during kinetic transformations.

In Vivo Voltage-Sensitive Dye Study of Lateral Spreading of Cortical Activity in Mouse Primary Visual Cortex Induced by a Current Impulse

In the mammalian primary visual cortex (V1), lateral spreading of excitatory potentials is believed to be involved in spatial integrative functions, but the underlying cortical mechanism is not well understood. Visually-evoked population-level responses have been shown to propagate beyond the V1 initial activation site in mouse, similar to higher mammals. Visually-evoked responses are, however, affected by neuronal circuits prior to V1 (retina, LGN), making the separate analysis of V1 difficult. Intracortical stimulation eliminates these initial processing steps. We used in vivo RH1691 voltage-sensitive dye (VSD) imaging and intracortical microstimulation in adult C57BL/6 mice to elucidate the spatiotemporal properties of population-level signal spreading in V1 cortical circuits. The evoked response was qualitatively similar to that measured in single-cell electrophysiological experiments in rodents: a fast transient fluorescence peak followed by a fast and a slow decrease or hyperpolarization, similar to EPSP and fast and slow IPSPs in single cells. The early cortical response expanded at speeds commensurate with long horizontal projections (at 5% of the peak maximum, 0.08–0.15 m/s) however, the bulk of the VSD signal propagated slowly (at half-peak maximum, 0.05–0.08 m/s) suggesting an important role of regenerative multisynaptic transmission through short horizontal connections in V1 spatial integrative functions. We also found a tendency for a widespread and fast cortical response suppression in V1, which was eliminated by GABA_A-antagonists gabazine and bicuculline methiodide. Our results help understand the neuronal circuitry involved in lateral spreading in V1.

Cortico-cortical communication dynamics

In principle, cortico-cortical communication dynamics is simple: neurons in one cortical area communicate by sending action potentials that release glutamate and excite their target neurons in other cortical areas. In practice, knowledge about cortico-cortical communication dynamics is minute. One reason is that no current technique can capture the fast spatio-temporal cortico-cortical evolution of action potential transmission and membrane conductances with sufficient spatial resolution. A combination of optogenetics and monosynaptic tracing with virus can reveal the spatio-temporal cortico-cortical dynamics of specific neurons and their targets, but does not reveal how the dynamics evolves under natural conditions. Spontaneous ongoing action potentials also spread across cortical areas and are difficult to separate from structured evoked and intrinsic brain activity such as thinking. At a certain state of evolution, the dynamics may engage larger populations of neurons to drive the brain to decisions, percepts and behaviors. For example, successfully evolving dynamics to sensory transients can appear at the mesoscopic scale revealing how the transient is perceived. As a consequence of these methodological and conceptual difficulties, studies in this field comprise a wide range of computational models, large-scale measurements (e.g., by MEG, EEG), and a combination of invasive measurements in animal experiments. Further obstacles and challenges of studying cortico-cortical communication dynamics are outlined in this critical review.

Contributions of magno- and parvocellular channels to conscious and non-conscious vision

The dorsal and ventral cortical pathways, driven predominantly by magnocellular (M) and parvocellular (P) inputs, respectively, assume leading roles in models of visual information processing. Although in prior proposals, the dorsal and ventral pathways support non-conscious and conscious vision, respectively, recent modelling and empirical developments indicate that each pathway plays important roles in both non-conscious and conscious vision. In these models, the ventral P-pathway consists of one subpathway processing an object's contour features, e.g. curvature, the other processing its surface attributes, e.g. colour. Masked priming studies have shown that feed-forward activity in the ventral P-pathway on its own supports non-conscious processing of contour and surface features. The dorsal M-pathway activity contributes directly to conscious vision of motion and indirectly to object vision by projecting to prefrontal cortex, which in turn injects top-down neural activity into the ventral P-pathway and there 'ignites' feed-forward-re-entrant loops deemed necessary for conscious vision. Moreover, an object's shape or contour remains invisible without the prior conscious registration of its surface properties, which for that reason are taken to comprise fundamental visual qualia. Besides suggesting avenues for future research, these developments bear on several recent and past philosophical issues.

TMS over V5 disrupts motion prediction

Given the vast amount of sensory information the brain has to deal with, predicting some of this information based on the current context is a resource-efficient strategy. The framework of predictive coding states that higher-level brain areas generate a predictive model to be communicated via feedback connections to early sensory areas. Here, we directly tested the necessity of a higher-level visual area, V5, in this predictive processing in the context of an apparent motion paradigm. We flashed targets on the apparent motion trace in-time or out-of-time with the predicted illusory motion token. As in previous studies, we found that predictable in-time targets were better detected than unpredictable out-of-time targets. However, when we applied functional magnetic resonance imaging-guided, double-pulse transcranial magnetic stimulation (TMS) over left V5 at 13–53 ms before target onset, the detection advantage of in-time targets was eliminated; this was not the case when TMS was applied over the vertex. Our results are causal evidence that V5 is necessary for a prediction effect, which has been shown to modulate V1 activity (Alink et al. 2010). Thus, our findings suggest that information processing between V5 and V1 is crucial for visual motion prediction, providing experimental support for the predictive coding framework.

Laminar firing and membrane dynamics in four visual areas exposed to two objects moving to occlusion

It is not known how visual cortical neurons react to several moving objects and how their firing to the motion of one object is affected by neurons firing to another moving object. Here we combine imaging of voltage sensitive dye (VSD) signals, reflecting the population membrane potential from ferret visual areas 17, 18, 19, and 21, with laminar recordings of multiunit activity, (MUA), when two bars moved toward each other in the visual field, occluded one another, and continued on in opposite directions. Two zones of peak MUA, mapping the bars' motion, moved toward each other along the area 17/18 border, which in the ferret maps the vertical meridian of the field of view. This was reflected also in the VSD signal, at both the 17/18 border as well as at the 19/21 border with a short delay. After some 125 ms at the area 19/21 border, the VSD signal increased and became elongated in the direction of motion in front of both of the moving representations. This was directly followed by the phase of the signal reversing and travelling back from the 19/21 border toward the 17/18 border, seemingly without respect for retinotopic boundaries, where it arrived at 150 ms after stimulus onset. At this point the VSD signal in front of the moving bar representations along the 17/18 border also increased and became elongated in the direction of object motion; the signal now being the linear sum of what has been observed in response to single moving bars. When the neuronal populations representing the bars were some 600 μm apart on the cortex, the dye signal and laminar MUA decreased strongly, with the MUA scaling to that of a single bar during occlusion. Despite a short rebound of the dye signal and MUA, the MUA after the occlusion was significantly depressed. The interactions between the neuronal populations mapping the bars' position, and the neurons in between these populations were, apart from 19/21 to 17/18 interaction, mainly lateral-horizontal; first excitatory and inducing firing at the site of future occlusion, then inhibitory just prior to occlusion. After occlusion the neurons that had fired already to the first bar showed delayed and prolonged inhibition in response to the second bar. Thus, the interactions that were particular to the occlusion condition in these experiments were local and inhibitory at short cortical range, and delayed and inhibitory after the occlusion when the bars moved further apart.

Apparent motion perception in patients with paranoid schizophrenia

Impaired perceptual inference has been suggested to be at the core of positive symptoms in schizophrenia. Apparent motion (AM) is a visual illusion in which perceptual inference gives rise to the experience of a single object moving back and forth when two spatially separated objects are flashed in alternation. Here, we investigated the strength of AM perception in patients with paranoid schizophrenia. Patients were less susceptible to the illusion as indicated by a lower probability of motion perception at the individual's optimal presentation frequency for AM. In addition, the probability of AM perception was inversely related to delusional conviction in the patient group. These results suggest that schizophrenia may be associated with a reduced susceptibility to visual phenomena that commonly rely on perceptual inference.

Propagating waves in thalamus, cortex and the thalamocortical system: Experiments and models

Propagating waves of activity have been recorded in many species, in various brain states, brain areas, and under various stimulation conditions. Here, we review the experimental literature on propagating activity in thalamus and neocortex across various levels of anesthesia and stimulation conditions. We also review computational models of propagating waves in networks of thalamic cells, cortical cells and of the thalamocortical system. Some discrepancies between experiments can be explained by the "network state", which differs vastly between anesthetized and awake conditions. We introduce a network model displaying different states and investigate their effect on the spatial structure of self-sustained and externally driven activity. This approach is a step towards understanding how the intrinsically-generated ongoing activity of the network affects its ability to process and propagate extrinsic input.

Detection of visual events along the apparent motion trace in patients with paranoid schizophrenia

Dysfunctional prediction in sensory processing has been suggested as a possible causal mechanism in the development of delusions in patients with schizophrenia. Previous studies in healthy subjects have shown that while the perception of apparent motion can mask visual events along the illusory motion trace, such motion masking is reduced when events are spatio-temporally compatible with the illusion, and, therefore, predictable. Here we tested the hypothesis that this specific detection advantage for predictable target stimuli on the apparent motion trace is reduced in patients with paranoid schizophrenia. Our data show that, although target detection along the illusory motion trace is generally impaired, both patients and healthy control participants detect predictable targets more often than unpredictable targets. Patients had a stronger motion masking effect when compared to controls. However, patients showed the same advantage in the detection of predictable targets as healthy control subjects. Our findings reveal stronger motion masking but intact prediction of visual events along the apparent motion trace in patients with paranoid schizophrenia and suggest that the sensory prediction mechanism underlying apparent motion is not impaired in paranoid schizophrenia.

Interactions between two propagating waves in rat visual cortex

Sensory-evoked propagating waves are frequently observed in sensory cortex. However, it is largely unknown how an evoked propagating wave affects the activity evoked by subsequent sensory inputs, or how two propagating waves interact when evoked by simultaneous sensory inputs. Using voltage-sensitive dye imaging, we investigated the interactions between two evoked waves in rat visual cortex, and the spatiotemporal patterns of depolarization in the neuronal population due to wave-to-wave interactions. We have found that visually-evoked propagating waves have a refractory period of about 300 ms, within which the response to a subsequent visual stimulus is suppressed. Simultaneous presentation of two visual stimuli at different locations can evoke two waves propagating toward each other, and these two waves fuse. Fusion significantly shortens the latency and half-width of the response, leading to changes in the spatial profile of evoked population activity. The visually-evoked propagating wave may also be suppressed by a preceding spontaneous wave. The refractory period following a propagating wave and the fusion between two waves may contribute to visual sensory processing by modifying the spatiotemporal profile of population neuronal activity evoked by sensory events.

BOLD responses in the superior colliculus and lateral geniculate nucleus of the rat viewing an apparent motion stimulus

In rats, the superior colliculus (SC) is a main destination for retinal ganglion cells and is an important subcortical structure for vision. Electrophysiology studies have observed that many SC neurons are highly sensitive to moving objects, but complementary non-invasive functional imaging studies with larger fields of view have been rarely conducted. In this study, BOLD fMRI is used to measure the SC and nearby lateral geniculate nucleus' (LGN) hemodynamic responses, in normal adult Sprague Dawley (SD) rats, during a dynamic visual stimulus similar to those used in long-range apparent motion studies. The stimulation paradigm consists of four light spots arranged in a linear array and turned on and off sequentially at different rates to create five effective speeds of motion (7, 14, 41, 82, and 164°/s across the visual field). Stationary periods (same light spot always on) are interleaved between the moving periods. The speed response function (SRF), the hemodynamic response amplitude at each speed tested, is measured. Significant responses are observed in the SC and LGN at all speeds. In the SC, the SRF increases monotonically from 7 to 82°/s. The minimum response amplitude occurs at 164°/s. The results suggest that the SC is sensitive to slow moving visual stimuli but the hemodynamic response is reduced at higher speeds. In the LGN, the SRF exhibits a similar trend to that of the SC, but response amplitude during 7°/s stimulation is comparable to that during 164°/s stimulation. These findings are in good agreement with previous electrophysiology studies conducted on albino rats like the SD strain. This work represents the first fMRI study of stimulus speed dependence in the SC and is also the first fMRI study of motion responsiveness in the rat.

Flow detection of propagating waves with temporospatial correlation of activity

Voltage-sensitive dye imaging (VSDI) allows population patterns of cortical activity to be recorded with high temporal resolution, and recent findings ascribe potential significance to these spatial propagation patterns--both for normal cortical processing and in pathologies such as epilepsy. However, analysis of these spatiotemporal patterns has been mostly qualitative to date. In this report, we describe an algorithm to quantify fast local flow patterns of cortical population activation, as measured with VSDI. The algorithm uses correlation of temporal features across space, and therefore differs from conventional optical flow algorithms which use correlation of spatial features over time. This alternative approach allows us to take advantage of the characteristics of fast optical imaging data, which have very high temporal resolution but less spatial resolution. We verify the method both on artificial and biological data, and demonstrate its use.

Adaptation, expertise, and giftedness: towards an understanding of cortical, subcortical, and cerebellar network contributions

Current cortico-centric models of cognition lack a cohesive neuroanatomic framework that sufficiently considers overlapping levels of function, from "pathological" through "normal" to "gifted" or exceptional ability. While most cognitive theories presume an evolutionary context, few actively consider the process of adaptation, including concepts of neurodevelopment. Further, the frequent co-occurrence of "gifted" and "pathological" function is difficult to explain from a cortico-centric point of view. This comprehensive review paper proposes a framework that includes the brain's vertical organization and considers "giftedness" from an evolutionary and neurodevelopmental vantage point. We begin by discussing the current cortico-centric model of cognition and its relationship to intelligence. We then review an integrated, dual-tiered model of cognition that better explains the process of adaptation by simultaneously allowing for both stimulus-based processing and higher-order cognitive control. We consider the role of the basal ganglia within this model, particularly in relation to reward circuitry and instrumental learning. We review the important role of white matter tracts in relation to speed of adaptation and development of behavioral mastery. We examine the cerebellum's critical role in behavioral refinement and in cognitive and behavioral automation, particularly in relation to expertise and giftedness. We conclude this integrated model of brain function by considering the savant syndrome, which we believe is best understood within the context of a dual-tiered model of cognition that allows for automaticity in adaptation as well as higher-order executive control.

Lateral Spread of Orientation Selectivity in V1 is Controlled by Intracortical Cooperativity

Neurons in the primary visual cortex receive subliminal information originating from the periphery of their receptive fields (RF) through a variety of cortical connections. In the cat primary visual cortex, long-range horizontal axons have been reported to preferentially bind to distant columns of similar orientation preferences, whereas feedback connections from higher visual areas provide a more diverse functional input. To understand the role of these lateral interactions, it is crucial to characterize their effective functional connectivity and tuning properties. However, the overall functional impact of cortical lateral connections, whatever their anatomical origin, is unknown since it has never been directly characterized. Using direct measurements of postsynaptic integration in cat areas 17 and 18, we performed multi-scale assessments of the functional impact of visually driven lateral networks. Voltage-sensitive dye imaging showed that local oriented stimuli evoke an orientation-selective activity that remains confined to the cortical feedforward imprint of the stimulus. Beyond a distance of one hypercolumn, the lateral spread of cortical activity gradually lost its orientation preference approximated as an exponential with a space constant of about 1 mm. Intracellular recordings showed that this loss of orientation selectivity arises from the diversity of converging synaptic input patterns originating from outside the classical RF. In contrast, when the stimulus size was increased, we observed orientation-selective spread of activation beyond the feedforward imprint. We conclude that stimulus-induced cooperativity enhances the long-range orientation-selective spread.

Independent encoding of grating motion across stationary feature maps in primary visual cortex visualized with voltage-sensitive dye imaging

In early visual cortex different stimulus parameters are represented in overlaid feature maps. Such functioning was extensively explored by the use of drifting gratings characterized by orientation, spatial-temporal frequency, and direction of motion. However surprisingly, the direct cortical visuotopic drift of the gratings' stripy pattern has never been detected simultaneously to these stationary feature maps. It therefore remains to be demonstrated how physical signals of grating motion across the cortex are represented independently of other parametric maps and thus, how multi-dimensional input is processed independently to enable effective read-out further downstream. Taking advantage of the high spatial and temporal resolution of voltage-sensitive dye imaging, we here show the real-time encoding of position and orientation. By decomposing the cortical responses to drifting gratings we visualize the typical emergence of stationary orientation maps in which specific domains exhibited highest amplitudes. Simultaneously to these patchy maps, we demonstrate coherently propagating waves of activity that precisely matched the actual movement of the gratings in space and time, most dominantly for spatial frequencies lower than the preferred range. Thus, the primary visual cortex multiplexes information about retinotopic motion by additional temporal modulation of stationary orientation signals. These signals may be used to variably extract coarse-grained object motion and form information at higher visual processing stages.

A dynamic neural field model of mesoscopic cortical activity captured with voltage-sensitive dye imaging

A neural field model is presented that captures the essential non-linear characteristics of activity dynamics across several millimeters of visual cortex in response to local flashed and moving stimuli. We account for physiological data obtained by voltage-sensitive dye (VSD) imaging which reports mesoscopic population activity at high spatio-temporal resolution. Stimulation included a single flashed square, a single flashed bar, the line-motion paradigm--for which psychophysical studies showed that flashing a square briefly before a bar produces sensation of illusory motion within the bar--and moving squares controls. We consider a two-layer neural field (NF) model describing an excitatory and an inhibitory layer of neurons as a coupled system of non-linear integro-differential equations. Under the assumption that the aggregated activity of both layers is reflected by VSD imaging, our phenomenological model quantitatively accounts for the observed spatio-temporal activity patterns. Moreover, the model generalizes to novel similar stimuli as it matches activity evoked by moving squares of different speeds. Our results indicate that feedback from higher brain areas is not required to produce motion patterns in the case of the illusory line-motion paradigm. Physiological interpretation of the model suggests that a considerable fraction of the VSD signal may be due to inhibitory activity, supporting the notion that balanced intra-layer cortical interactions between inhibitory and excitatory populations play a major role in shaping dynamic stimulus representations in the early visual cortex.

Six principles of visual cortical dynamics

A fundamental goal in vision science is to determine how many neurons in how many areas are required to compute a coherent interpretation of the visual scene. Here I propose six principles of cortical dynamics of visual processing in the first 150 ms following the appearance of a visual stimulus. Fast synaptic communication between neurons depends on the driving neurons and the biophysical history and driving forces of the target neurons. Under these constraints, the retina communicates changes in the field of view driving large populations of neurons in visual areas into a dynamic sequence of feed-forward communication and integration of the inward current of the change signal into the dendrites of higher order area neurons (30-70 ms). Simultaneously an even larger number of neurons within each area receiving feed-forward input are pre-excited to sub-threshold levels. The higher order area neurons communicate the results of their computations as feedback adding inward current to the excited and pre-excited neurons in lower areas. This feedback reconciles computational differences between higher and lower areas (75-120 ms). This brings the lower area neurons into a new dynamic regime characterized by reduced driving forces and sparse firing reflecting the visual areas interpretation of the current scene (140 ms). The population membrane potentials and net-inward/outward currents and firing are well behaved at the mesoscopic scale, such that the decoding in retinotopic cortical space shows the visual areas' interpretation of the current scene. These dynamics have plausible biophysical explanations. The principles are theoretical, predictive, supported by recent experiments and easily lend themselves to experimental tests or computational modeling.

Stimulus predictability reduces responses in primary visual cortex

In this functional magnetic resonance imaging study we tested whether the predictability of stimuli affects responses in primary visual cortex (V1). The results of this study indicate that visual stimuli evoke smaller responses in V1 when their onset or motion direction can be predicted from the dynamics of surrounding illusory motion. We conclude from this finding that the human brain anticipates forthcoming sensory input that allows predictable visual stimuli to be processed with less neural activation at early stages of cortical processing.

The role of multi-area interactions for the computation of apparent motion

Apparent motion (AM) is a robust visual illusion, in which fast displays of static objects in successively different positions elicit the perception of object motion. Neurons in higher order areas 21 and 19 compute object motion under such conditions and send feedback to early visual areas 18 and 17, which is instrumental in eliciting computation of motion in those very areas. To explore the computational dynamics of AM, we made a neural field model consisting of two one-dimensional rings of simple neurons expressing firing rates, one for areas 17/18 and one for areas 19/21. The model neurons, without any orientation or direction selectivity, computed apparent motion for the range of space-timings of stimuli associated with short- and long-range AM in humans. The computation of long-range AM in 17/18 required two model areas and the presence of feedback and conduction/computation delays between those areas. As in the in vivo experiments of long-range AM, the stationary stimuli were initially mapped as stationary in model area 17/18, but after the feedback also these lower areas computed AM. The dynamics of the two-area network produces short-range and long-range apparent motion for a large range of feedback strengths and a small range of lateral excitation near the bifurcation to an amplitude instability. The computation of AM in higher order areas was due to the neurons in these areas having large receptive fields as a consequence of divergent feed-forward connectivity. This implies that these areas compute long-range AM when early areas 17 and 18 do not, and therefore higher order areas must enslave lower order areas to compute the same, if the whole network is to arrive at a coherent perceptual solution.

Voltage-sensitive dye imaging: Technique review and models

In this review, we present the voltage-sensitive dye imaging (VSDI) method. The possibility offered for in vivo (and in vitro) brain imaging is unprecedented in terms of spatial and temporal resolution. However, the unresolved multi-component origin of the optical signal encourages us to perform a detailed analysis of the method limitation and the existing models. We propose a biophysical model at a mesoscopic scale in order to understand and interpret this signal.