Why we see things the way we do: evidence for a wholly empirical strategy of vision

Population codes of prior knowledge learned through environmental regularities

How the brain makes correct inferences about its environment based on noisy and ambiguous observations is one of the fundamental questions in Neuroscience. Prior knowledge about the probability with which certain events occur in the environment plays an important role in this process. Humans are able to incorporate such prior knowledge in an efficient, Bayes optimal, way in many situations, but it remains an open question how the brain acquires and represents this prior knowledge. The long time spans over which prior knowledge is acquired make it a challenging question to investigate experimentally. In order to guide future experiments with clear empirical predictions, we used a neural network model to learn two commonly used tasks in the experimental literature (i.e. orientation classification and orientation estimation) where the prior probability of observing a certain stimulus is manipulated. We show that a population of neurons learns to correctly represent and incorporate prior knowledge, by only receiving feedback about the accuracy of their inference from trial-to-trial and without any probabilistic feedback. We identify different factors that can influence the neural responses to unexpected or expected stimuli, and find a novel mechanism that changes the activation threshold of neurons, depending on the prior probability of the encoded stimulus. In a task where estimating the exact stimulus value is important, more likely stimuli also led to denser tuning curve distributions and narrower tuning curves, allocating computational resources such that information processing is enhanced for more likely stimuli. These results can explain several different experimental findings, clarify why some contradicting observations concerning the neural responses to expected versus unexpected stimuli have been reported and pose some clear and testable predictions about the neural representation of prior knowledge that can guide future experiments.

Ipsilateral eye contributions to online visuomotor control of right upper-limb movements

A limb's initial position is often biased to the right of the midline during activities of daily living. Given this specific initial limb position, visual cues of the limb become first available to the ipsilateral eye relative to the contralateral eye. The current study investigated online control of the dominant limb as a function of having visual cues available to the ipsilateral or contralateral eye, in relation to the initial start position of the limb. Participants began each trial with their right limb on a home position to the left or right of the midline. After movement onset, a brief visual sample was provided to the ipsilateral or contralateral eye. On one third of the trials, an imperceptible 3 cm target jump was introduced. If visual information from the eye ipsilateral to the limb is preferentially used to control ongoing movements of the dominant limb, corrections for the target jump should be observed when movements began from the right of the body's midline and vision was available to the ipsilateral eye. As expected, limb trajectory corrections for the target jump were only observed when participants started from the right home position and visual information was provided to the ipsilateral eye. We purport that such visuomotor asymmetry specialization emerges via neurophysiological developments, which may arise from naturalistic and probabilistic limb trajectory asymmetries.

A Connectome-wide Functional Signature of Transdiagnostic Risk for Mental Illness

BACKGROUND

High rates of comorbidity, shared risk, and overlapping therapeutic mechanisms have led psychopathology research toward transdiagnostic dimensional investigations of clustered symptoms. One influential framework accounts for these transdiagnostic phenomena through a single general factor, sometimes referred to as the p factor, associated with risk for all common forms of mental illness.

METHODS

We build on previous research identifying unique structural neural correlates of the p factor by conducting a data-driven analysis of connectome-wide intrinsic functional connectivity (n = 605).

RESULTS

We demonstrate that higher p factor scores and associated risk for common mental illness maps onto hyperconnectivity between visual association cortex and both frontoparietal and default mode networks.

CONCLUSIONS

These results provide initial evidence that the transdiagnostic risk for common forms of mental illness is associated with patterns of inefficient connectome-wide intrinsic connectivity between visual association cortex and networks supporting executive control and self-referential processes, networks that are often impaired across categorical disorders.

Functional Organization of Flash-Induced V1 Offline Reactivation

The primary visual cortex exhibits a late, long response with a latency of >300 ms and an immediate early response that occurs ∼100 ms after a visual stimulus. The late response is thought to contribute to visual functions such as sensory perception, iconic memory, working memory, and forming connections between temporally separated stimuli. However, how the visual late response is generated and organized is not completely understood. In the mouse primary visual cortex in vivo, we isolated long-delayed responses by using a brief light-flash stimulus for which the stimulus late response occurred long after the stimulus offset and was not contaminated by the instantaneous response evoked by the stimulus. Using whole-cell patch-clamp recordings, we demonstrated that the late rebound response was shaped by a net-balanced increase in excitatory and inhibitory synaptic conductances, whereas transient imbalances were caused by intermittent inhibitory barrage. In contrast to the common assumption that the neocortical late response reflects a feedback signal from the downstream higher-order cortical areas, our pharmacological and optogenetic analyses demonstrated that the late responses likely have a thalamic origin. Therefore, the late component of a sensory-evoked cortical response should be interpreted with caution.

SIGNIFICANCE STATEMENT

The long-delayed responses of neocortical neurons are thought to arise from cortical feedback activity that is related to sensory perception and cognition. The mechanism of neocortical late responses was investigated using multiple electrophysiological techniques and the findings indicate that it actually arises from the thalamus. In addition, during the late response, excitation and inhibition are balanced, but inhibition is dominant in patterning action potentials.

Theta Oscillations in Visual Cortex Emerge with Experience to Convey Expected Reward Time and Experienced Reward Rate

The primary visual cortex (V1) is widely regarded as faithfully conveying the physical properties of visual stimuli. Thus, experience-induced changes in V1 are often interpreted as improving visual perception (i.e., perceptual learning). Here we describe how, with experience, cue-evoked oscillations emerge in V1 to convey expected reward time as well as to relate experienced reward rate. We show, in chronic multisite local field potential recordings from rat V1, that repeated presentation of visual cues induces the emergence of visually evoked oscillatory activity. Early in training, the visually evoked oscillations relate to the physical parameters of the stimuli. However, with training, the oscillations evolve to relate the time in which those stimuli foretell expected reward. Moreover, the oscillation prevalence reflects the reward rate recently experienced by the animal. Thus, training induces experience-dependent changes in V1 activity that relate to what those stimuli have come to signify behaviorally: when to expect future reward and at what rate.

Neocortical Rebound Depolarization Enhances Visual Perception

Animals are constantly exposed to the time-varying visual world. Because visual perception is modulated by immediately prior visual experience, visual cortical neurons may register recent visual history into a specific form of offline activity and link it to later visual input. To examine how preceding visual inputs interact with upcoming information at the single neuron level, we designed a simple stimulation protocol in which a brief, orientated flashing stimulus was subsequently coupled to visual stimuli with identical or different features. Using in vivo whole-cell patch-clamp recording and functional two-photon calcium imaging from the primary visual cortex (V1) of awake mice, we discovered that a flash of sinusoidal grating per se induces an early, transient activation as well as a long-delayed reactivation in V1 neurons. This late response, which started hundreds of milliseconds after the flash and persisted for approximately 2 s, was also observed in human V1 electroencephalogram. When another drifting grating stimulus arrived during the late response, the V1 neurons exhibited a sublinear, but apparently increased response, especially to the same grating orientation. In behavioral tests of mice and humans, the flashing stimulation enhanced the detection power of the identically orientated visual stimulation only when the second stimulation was presented during the time window of the late response. Therefore, V1 late responses likely provide a neural basis for admixing temporally separated stimuli and extracting identical features in time-varying visual environments.

A study of mice and humans shows that prior activity in the visual cortex induces a long-delayed depolarization that enhances perception of subsequent visual stimuli if these are identical to the previous one, thereby extracting invariant visual features from the constantly changing visual world.

Animals are constantly exposed to a visual world that varies over time. To examine how the visual cortex integrates visual information that is temporally spaced, we monitored neuronal activity of the primary visual cortex (V1) using single- and multicell recording techniques. We discovered that a brief visual stimulus induced an early, transient activation as well as a delayed reactivation of V1 neurons in mice and humans. Notably, this reactivation of visual cortex conveyed information about stimulus orientation: presentation of a second visual stimulus during this reactivation enhanced the V1 response specifically when the orientations of the two stimuli were identical. Behavioral tests in mice and humans revealed that the ability to detect visual stimuli was also enhanced when the second stimulus was presented during the time window of V1 reactivation. Because animals extract visual information from an environment in constant change, the modulation of visual responses through cortical reactivation might be a strategy commonly used in the visual system.

A Bayesian perspective on sensory and cognitive integration in pain perception and placebo analgesia

The placebo effect is a component of any response to a treatment (effective or inert), but we still ignore why it exists. We propose that placebo analgesia is a facet of pain perception, others being the modulating effects of emotions, cognition and past experience, and we suggest that a computational understanding of pain may provide a unifying explanation of these phenomena. Here we show how Bayesian decision theory can account for such features and we describe a model of pain that we tested against experimental data. Our model not only agrees with placebo analgesia, but also predicts that learning can affect pain perception in other unexpected ways, which experimental evidence supports. Finally, the model can also reflect the strategies used by pain perception, showing that modulation by disparate factors is intrinsic to the pain process.

Visual anticipation biases conscious decision making but not bottom-up visual processing

Prediction plays a key role in control of attention but it is not clear which aspects of prediction are most prominent in conscious experience. An evolving view on the brain is that it can be seen as a prediction machine that optimizes its ability to predict states of the world and the self through the top-down propagation of predictions and the bottom-up presentation of prediction errors. There are competing views though on whether prediction or prediction errors dominate the formation of conscious experience. Yet, the dynamic effects of prediction on perception, decision making and consciousness have been difficult to assess and to model. We propose a novel mathematical framework and a psychophysical paradigm that allows us to assess both the hierarchical structuring of perceptual consciousness, its content and the impact of predictions and/or errors on conscious experience, attention and decision-making. Using a displacement detection task combined with reverse correlation, we reveal signatures of the usage of prediction at three different levels of perceptual processing: bottom-up fast saccades, top-down driven slow saccades and consciousnes decisions. Our results suggest that the brain employs multiple parallel mechanism at different levels of perceptual processing in order to shape effective sensory consciousness within a predicted perceptual scene. We further observe that bottom-up sensory and top-down predictive processes can be dissociated through cognitive load. We propose a probabilistic data association model from dynamical systems theory to model the predictive multi-scale bias in perceptual processing that we observe and its role in the formation of conscious experience. We propose that these results support the hypothesis that consciousness provides a time-delayed description of a task that is used to prospectively optimize real time control structures, rather than being engaged in the real-time control of behavior itself.

Geometrical illusions are not always where you think they are: a review of some classical and less classical illusions, and ways to describe them

Geometrical illusions are known through a small core of classical illusions that were discovered in the second half of the nineteenth century. Most experimental studies and most theoretical discussions revolve around this core of illusions, as though all other illusions were obvious variants of these. Yet, many illusions, mostly described by German authors at the same time or at the beginning of the twentieth century have been forgotten and are awaiting their rehabilitation. Recently, several new illusions were discovered, mainly by Italian authors, and they do not seem to take place into any current classification. Among the principles that are invoked to explain the illusions, there are principles relating to the metric aspects (contrast, assimilation, shrinkage, expansion, attraction of parallels) principles relating to orientations (regression to right angles, orthogonal expansion) or, more recently, to gestalt effects. Here, metric effects are discussed within a measurement framework, in which the geometric illusions are the outcome of a measurement process. There would be a main "convexity" bias in the measures: the measured value m(x) of an extant x would grow more than proportionally with x. This convexity principle, completed by a principle of compromise for conflicting measures can replace, for a large number of patterns, both the assimilation and the contrast effects. We know from evolutionary theory that the most pertinent classification criteria may not be the most salient ones (e.g., a dolphin is not a fish). In order to obtain an objective classification of illusions, I initiated with Kevin O'Regan systematic work on "orientation profiles" (describing how the strength of an illusion varies with its orientation in the plane). We showed first that the Zöllner illusion already exists at the level of single stacks, and that it does not amount to a rotation of the stacks. Later work suggested that it is best described by an "orthogonal expansion"-an expansion of the stacks applied orthogonally to the oblique segments of the stacks, generating an apparent rotation effect. We showed that the Poggendorff illusion was mainly a misangulation effect. We explained the hierarchy of the illusion magnitudes found among variants of the Poggendorff illusion by the existence of control devices that counteract the loss of parallelism or the loss of collinearity produced by the biased measurements. I then studied the trapezium illusion. The oblique sides, but not the bases, were essential to the trapezium illusion, suggesting the existence of a common component between the trapezium and the Zöllner illusion. Unexpectedly, the trapeziums sometimes appeared as twisted surfaces in 3d. It also appeared impossible, using a nulling procedure, to make all corresponding sides of two trapeziums simultaneously equal. The square-diamond illusion is usually presented with one apex of the diamond pointing toward the square. I found that when the figures were displayed more symmetrically, the illusion was significantly reduced. Furthermore, it is surpassed, for all subjects, by an illusion that goes in the opposite direction, in which the diagonal of a small diamond is underestimated with respect to the side of a larger square. In general, the experimental work generated many unexpected results. Each illusory stimulus was compared to a number of control variants, and often, I measured larger distortions in a variant than in the standard stimulus. In the Discussion, I will stress what I think are the main ordering principle in the metric and the orientation domains for illusory patterns. The convexity bias principle and the orthogonal expansion principles help to establish unsuspected links between apparently unrelated stimuli, and reduce their apparently extreme heterogeneity. However, a number of illusions (e.g., those of the twisted cord family, or the Poggendorff illusions) remain unpredicted by the above principles. Finally, I will develop the idea that the brain is constructing several representations, and the one that is commonly used for the purpose of shape perception generates distortions inasmuch as it must satisfy a number of conflicting constraints, such as the constraint of producing a stable shape despite the changing perspectives produced by eye movements.

A systems neuroscience perspective of schizophrenia and bipolar disorder

Neuroimaging studies have generated a large body of knowledge regarding the neural correlates of schizophrenia (SZ) and bipolar disorder (BD). However, the initial goal of identifying disease-specific topographical mappings to localized brain regions or to distinct neural networks has not materialized and may be untenable. This contribution will argue that a systems neuroscience approach may prove more fruitful. The supporting evidence presented covers (a) brain structural, functional, and connectivity alterations and their implication for the clinical and cognitive manifestations of SZ and BD, (b) the prevailing system neuroscience models of the 2 disorders, and (c) key hypotheses likely to produce new insights into the mechanisms of underlying psychotic disorders.

Selection of visual information for lightness judgements by eye movements

When judging the lightness of objects, the visual system has to take into account many factors such as shading, scene geometry, occlusions or transparency. The problem then is to estimate global lightness based on a number of local samples that differ in luminance. Here, we show that eye fixations play a prominent role in this selection process. We explored a special case of transparency for which the visual system separates surface reflectance from interfering conditions to generate a layered image representation. Eye movements were recorded while the observers matched the lightness of the layered stimulus. We found that observers did focus their fixations on the target layer, and this sampling strategy affected their lightness perception. The effect of image segmentation on perceived lightness was highly correlated with the fixation strategy and was strongly affected when we manipulated it using a gaze-contingent display. Finally, we disrupted the segmentation process showing that it causally drives the selection strategy. Selection through eye fixations can so serve as a simple heuristic to estimate the target reflectance.

The coherent organization of mental life depends on mechanisms for context-sensitive gain-control that are impaired in schizophrenia

There is rapidly growing evidence that schizophrenia involves changes in context-sensitive gain-control and probabilistic inference. In addition to the well-known cognitive disorganization to which these changes lead, basic aspects of vision are also impaired, as discussed by other papers on this Frontiers Research Topic. The aim of this paper is to contribute to our understanding of such findings by examining five central hypotheses. First, context-sensitive gain-control is fundamental to brain function and mental life. Second, it occurs in many different regions of the cerebral cortex of many different mammalian species. Third, it has several computational functions, each with wide generality. Fourth, it is implemented by several neural mechanisms at cellular and circuit levels. Fifth, impairments of context-sensitive gain-control produce many of the well-known symptoms of schizophrenia and change basic processes of visual perception. These hypotheses suggest why disorders of vision in schizophrenia may provide insights into the nature and mechanisms of impaired reality testing and thought disorder in psychosis. They may also cast light on normal mental function and its neural bases. Limitations of these hypotheses, and ways in which they need further testing and development, are outlined.

Motor-sensory confluence in tactile perception

Perception involves motor control of sensory organs. However, the dynamics underlying emergence of perception from motor-sensory interactions are not yet known. Two extreme possibilities are as follows: (1) motor and sensory signals interact within an open-loop scheme in which motor signals determine sensory sampling but are not affected by sensory processing and (2) motor and sensory signals are affected by each other within a closed-loop scheme. We studied the scheme of motor-sensory interactions in humans using a novel object localization task that enabled monitoring the relevant overt motor and sensory variables. We found that motor variables were dynamically controlled within each perceptual trial, such that they gradually converged to steady values. Training on this task resulted in improvement in perceptual acuity, which was achieved solely by changes in motor variables, without any change in the acuity of sensory readout. The within-trial dynamics is captured by a hierarchical closed-loop model in which lower loops actively maintain constant sensory coding, and higher loops maintain constant sensory update flow. These findings demonstrate interchangeability of motor and sensory variables in perception, motor convergence during perception, and a consistent hierarchical closed-loop perceptual model.

Neural activity at the human olfactory epithelium reflects olfactory perception

Organization of receptive surfaces reflects primary axes of perception. In vision, retinal coordinates reflect spatial coordinates. In audition, cochlear coordinates reflect tonal coordinates. However, the rules underlying the organization of the olfactory receptive surface are unknown. To test the hypothesis that organization of the olfactory epithelium reflects olfactory perception, we inserted an electrode into the human olfactory epithelium to directly measure odorant-induced evoked responses. We found that pairwise differences in odorant pleasantness predicted pairwise differences in response magnitude; that is, a location that responded maximally to a pleasant odorant was likely to respond strongly to other pleasant odorants, and a location that responded maximally to an unpleasant odorant was likely to respond strongly to other unpleasant odorants. Moreover, the extent of an individual's perceptual span predicted their span in evoked response. This suggests that, similarly to receptor surfaces for vision and audition, organization of the olfactory receptor surface reflects key axes of perception.

Shading, a view from the inside

Shape from shading arose from artistic practice, and later experimental psychology, but its formal structure has only been established recently by computer vision. Some of its algorithms have led to useful applications. Psychology has reversely borrowed these formalisms in attempts to come to grips with shading as a depth cue. Results have been less than spectacular. The reason might well be that these formalisms are all based on Euclidean geometry and physics (radiometry), which, are the right tools in third person accounts, but have little relevance to first person accounts, and thus are biologically (and consequently psychologically) of minor interest. We propose a formal theory of the shading cue in the first person account, 'a view from the inside'. Such a perspective is also required for autonomous robots in AI. This formalism cannot be based on Euclidean geometry, nor on radiometry, but on the structure of pictorial space, and the structure of brightness space. The formalism, though different in kind, has a simple relation to the computer vision accounts. It has great robustness, is free from calibration issues, and allows purely local shape inferences. It is especially suited to biological (and thus AI) implementation. We consider a number of predictions and confront them with available empirical evidence.

Comprehensive model of how reality distortion and symptoms occur in schizophrenia: could impairment in learning-dependent predictive perception account for the manifestations of schizophrenia?

Conventional wisdom has not laid out a clear and uniform profile of schizophrenia as a unitary entity. One of the key first steps in elucidating the neurobiology of this entity would be to characterize the essential and common elements in the group of entities called schizophrenia. Kraepelin in his introduction notes 'the conviction seems to be more and more gaining ground that dementia praecox on the whole represents, a well characterized form of disease, and that we are justified in regarding the majority of the clinical pictures which are brought together here as the expression of a single morbid process, though outwardly they often diverge very far from one another'. But what is that single morbid process? We suggest that just as the uniform defect in all types of cancer is impaired regulation of cell proliferation, the primary defect in the group of entities called schizophrenia is persistent defective hierarchical temporal processing. This manifests in the form of chronic memory-prediction errors or deficits in learning-dependent predictive perception. These deficits account for the symptoms that present as reality distortion (delusions, thought disorder and hallucinations). This constellation of symptoms corresponds with the profile of most patients currently diagnosed as suffering from schizophrenia. In this paper we describe how these deficits can lead to the various symptoms of schizophrenia.

Hierarchical temporal processing deficit model of reality distortion and psychoses

We posit in this article that hierarchical temporal processing deficit is the underlying basis of reality distortion and psychoses. Schizophrenia is a prototypical reality distortion disorder in which the patient manifests with auditory hallucinations, delusions, disorganized speech and thinking, cognitive impairment, avolition and social and occupational dysfunction. Reality distortion can be present in many other disorders including bipolar disorder, major depression and even dementia. Conceptually, schizophrenia is a heterogeneous entity likely to be because of numerous causes similar to dementia. Although no single symptom or set of symptoms is pathognomonic, a cardinal feature in all patients with schizophrenia is chronic distortion of reality. The model that we have proposed accounts for the varied manifestations of reality distortion including hallucinations and delusions. In this paper we consider the implications of this model for the underlying biology of psychoses and also for the neurobiology of schizophrenia and suggest potential targets to consider for the etiology and pathophysiology of reality distortion, especially in the context of schizophrenia.

Seeing grating-textured surface begins at the border

Two experiments were conducted to reveal that the human visual system represents grating texture surface using a border-to-interior strategy. This strategy dictates that the visual system first registers the surface boundary contour and then sequentially spreads texture from the border to the interior of the image. Our experiments measured the perceived grating texture surface at various stimulus durations after the onset of a grating texture image. We found that the grating texture is initially seen near the boundary contours, with eventual spreading inward to the center of the image. To quantify the observation, the extent of the texture spreading from the boundary contour is measured as a function of the stimulus duration (30-500 ms). This allows us to analyze the texture spreading in retinal and cortical distances, based on human fMRI studies of the cortical magnification factor in cortical areas V1-V4, and to derive the spreading speed. We found that the spreading speed is constant when scaled according to the cortical distance. Similar findings are obtained no matter whether the grating texture image is presented monocularly or dichoptically, suggesting the generality of the border-to-interior strategy for representing surfaces.

Temporal Integration in Visual Word Recognition

When two displays are presented in close temporal succession at the same location, how does the brain assign them to one versus two conscious percepts? We investigate this issue using a novel reading paradigm in which the odd and even letters of a string are presented alternatively at a variable rate. The results reveal a window of temporal integration during reading, with a nonlinear boundary around ∼80 msec of presentation duration. Below this limit, the oscillating stimulus is easily fused into a single percept, with all characteristics of normal reading. Above this limit, reading times are severely slowed and suffer from a word-length effect. ERPs indicate that, even at the fastest frequency, the oscillating stimulus elicits synchronous oscillations in posterior visual cortices, while late ERP components sensitive to lexical status vanish beyond the fusion threshold. Thus, the fusion/segregation dilemma is not resolved by retinal or subcortical filtering, but at cortical level by at most 300 msec. The results argue against theories of visual word recognition and letter binding that rely on temporal synchrony or other fine temporal codes.

Coexistence of binocular integration and suppression determined by surface border information

The visual system relies on both the integration and interocular inhibitory processes to achieve single vision from different images in the two eyes. It is generally assumed that the integration process first searches for matching local features between the two eyes. If the matching fails, an interocular inhibitory process is triggered to suppress the image representation of one eye, leading to visual perception that is essentially contributed by the other eye. Here, using a stimulus comprising of binocularly corresponding features (vertical gratings) but incompatible surface border information, we found evidence to the contrary. In one half-image, a circular patch of vertical grating was phase-shifted relative to the surrounding vertical grating to create a circular, monocular boundary contour (MBC), while the other half-image had a similar vertical grating. The two half-images had a binocular disparity at the circular grating patch area, leading to the percept of a disc in depth. Concurrent with the stereo percept, threshold for detecting a Gabor probe on the half-image without the MBC was higher than that on the corresponding area with the grating disc, indicating binocular suppression. These findings reveal that when we perceive depth, which requires the integration process to obtain binocular disparity from the two eyes, one eye's image could simultaneously be suppressed from visual awareness by the interocular inhibitory process. Our study also presents a provocative example of where the brain selectively binds some, but not all, features of the images from the two eyes for visual perception.

Subordinate Categorization Enhances the Neural Selectivity in Human Object-selective Cortex for Fine Shape Differences

There is substantial evidence that object representations in adults are dynamically updated by learning. However, it is not clear to what extent these effects are induced by active processing of visual objects in a particular task context on top of the effects of mere exposure to the same objects. Here we show that the task does matter. We performed an event-related fMRI adaptation study in which we derived neural selectivity from a release of adaptation. We had two training conditions: “categorized objects” were categorized at a subordinate level based on fine shape differences (Which type of fish is this?), whereas “control objects” were seen equally often in a task context requiring no subordinate categorization (Is this a vase or not?). After training, the object-selective cortex was more selective for differences among categorized objects than for differences among control objects. This result indicates that the task context during training modulates the extent to which object selectivity is enhanced as a result of training.

Schizophrenia is a disorder of higher order hierarchical processing

Schizophrenia is a mental disorder in which the patient manifests with auditory hallucinations, paranoid or bizarre delusions, and disorganized speech and thinking. It is associated with significant social dysfunction. There are many hypotheses regarding schizophrenia. Most of these focus on schizophrenia as a manifestation of abnormalities from genetic [Mulle JG. Genomic structural variation and schizophrenia. Curr Psychiatry Rep 2008;10(2):171-7], viral [Fruntes V, Limosin F. Schizophrenia and viral infection during neurodevelopment: a pathogenesis model? Med Sci Monit 2008;14(6):RA71-7], neurochemical [e.g. dopamine (Lewis DA, Akil M. Cortical dopamine in schizophrenia: strategies for postmortem studies. J Psychiatr Res 1997;31(2):175-95) or interactions between neurotransmitters (Duncan GE, Sheitman BB, Lieberman JA. An integrated view of pathophysiological models of schizophrenia. Brain Res Brain Res Rev 1999;29(2):250-64)] or brain structural [Kotrla KJ, Weinberger DR. Brain imaging in schizophrenia. Annu Rev Med 1995;46:113-22] origins. Most of these hypotheses do not account for how or why these presumed causes lead to the manifestations of schizophrenia. We argue that brain structure and function is compatible with a hierarchical processing structure that forms the basis for perception and thought in healthy humans. We propose that perturbations of the types listed above lead to disruption of higher levels of perception and hierarchical temporal processing by the brain and that this constitutes the core deficit in schizophrenia. We present evidence that this model explains many of the features of schizophrenia and we make a series of predictions about schizophrenia.

A history of the concept of the stimulus and the role it played in the neurosciences

The term stimulus, as it was used in science from its earliest appearance in the sixteenth century up to the beginning of the nineteenth century, shows a gradual progress in denotation from the physical object designed to produce nervous and muscular excitation to the generically conceived event or object that initiates sensory or motor activity. To this shift corresponds a shift in the understanding of sensory experience. Johannes Muller's law of specific energy of sensory nerves played a major role in the shift, and Hermann von Helmholtz gave the shift its most thorough philosophical explanation.

Reward timing in the primary visual cortex

We discovered that when adult rats experience an association between visual stimuli and subsequent rewards, the responses of a substantial fraction of neurons in the primary visual cortex evolve from those that relate solely to the physical attributes of the stimuli to those that accurately predict the timing of reward. In addition to revealing a remarkable type of response plasticity in adult V1, these data demonstrate that reward-timing activity-a "higher" brain function-can occur very early in sensory-processing paths. These findings challenge the traditional interpretation of activity in the primary visual cortex.

Colour constancy under simultaneous changes in surface position and illuminant

Two kinds of constancy underlie the everyday perception of surface colour: constancy under changes in illuminant and constancy under changes in surface position. Classically, these two constancies seem to place conflicting demands on the visual system: to both take into account the region surrounding a surface and also discount it. It is shown here, however, that the ability of observers to make surface-colour matches across simultaneous changes in test-surface position and illuminant in computer-generated 'Mondrian' patterns is almost as good as across changes in illuminant alone. Performance was no poorer when the surfaces surrounding the test surface were permuted, or when information from a potential comparison surface, the one with the highest luminance, was suppressed. Computer simulations of cone-photoreceptor activity showed that a reliable cue for making surface-colour matches in all experimental conditions was provided by the ratios of cone excitations between the test surfaces and a spatial average over the whole pattern.

Modality maps within primate somatosensory cortex

The sensations of pressure, flutter, and vibration are psychophysically distinct tactile modalities produced by frequency-specific vibrotactile stimulation of different mechanoreceptors in the skin. The information coded by the different low-threshold mechanoreceptors are carried by anatomically and electrophysiologically distinct pathways that remain separate at least up to and including the input stage of primary somatosensory cortex (SI) in primates, area 3b. Little is known about the functional organization of tactile representation beyond that stage. By using intrinsic optical imaging methods to record from area 1, the second processing stage of SI, we present evidence that pressure, flutter, and vibratory stimuli activate spatially distinct cortical domains in area 1, further strengthening the foundation for modality-specific processing streams in SI. These modality domains exhibit an organization that is unlike the discontinuous modality maps in visual area V2 but more like the continuous visual orientation maps in V1. The results demonstrate that psychophysically distinct sensory modalities can have fundamentally different modes of cortical representation.

Physics embedded in visual perception of three-dimensional shape from motion

Visual perception, and by implication underlying neural events, can become unstable when optical information specifying objects is ambiguous. Here we report that one striking form of instability-perceived three-dimensional structure-from-motion (SFM)-can be stabilized when an otherwise ambiguous object appears within a context implying frictional interactions with another rotating object; violations of physical conditions specifying friction disrupt stabilization. Evidently, information about frictional interaction is embedded within neural mechanisms specifying SFM.

The statistical structure of human speech sounds predicts musical universals

The similarity of musical scales and consonance judgments across human populations has no generally accepted explanation. Here we present evidence that these aspects of auditory perception arise from the statistical structure of naturally occurring periodic sound stimuli. An analysis of speech sounds, the principal source of periodic sound stimuli in the human acoustical environment, shows that the probability distribution of amplitude-frequency combinations in human utterances predicts both the structure of the chromatic scale and consonance ordering. These observations suggest that what we hear is determined by the statistical relationship between acoustical stimuli and their naturally occurring sources, rather than by the physical parameters of the stimulus per se.

The statistical structure of human speech sounds predicts musical universals

The similarity of musical scales and consonance judgments across human populations has no generally accepted explanation. Here we present evidence that these aspects of auditory perception arise from the statistical structure of naturally occurring periodic sound stimuli. An analysis of speech sounds, the principal source of periodic sound stimuli in the human acoustical environment, shows that the probability distribution of amplitude-frequency combinations in human utterances predicts both the structure of the chromatic scale and consonance ordering. These observations suggest that what we hear is determined by the statistical relationship between acoustical stimuli and their naturally occurring sources, rather than by the physical parameters of the stimulus per se.

A common oscillator for perceptual rivalries?

Perceptual rivalry is an oscillation of conscious experience that takes place despite univarying. if ambiguous, sensory input. Much current interest is focused on the controversy over the neural site of binocular rivalry, a variety of perceptual rivalry for which a number of different cortical regions have been implicated. Debate continues over the relative role of higher levels of processing compared with primary visual cortex and the suggestion that different forms of rivalry involve different cortical areas. Here we show that the temporal pattern of disappearance and reappearance in motion-induced blindness (MIB) (Bonneh et al, 2001 Nature 411 798-801) is highly correlated with the pattern of oscillation reported during binocular rivalry in the same individual. This correlation holds over a wide range of inter-individual variation. Temporal similarity in the two phenomena was strikingly confirmed by the effects of the hallucinogen LSD, which produced the same, extraordinary, pattern of increased rhythmicity in both kinds of perceptual oscillation. Furthermore. MIB demonstrates the two properties previously considered characteristic of binocular rivalry. Namely the distribution of dominance periods can be approximated by a gamma distribution and, in line with Levelt's second proposition of binocular rivalry, predominance of one perceptual phase can be increased through a reduction in the predominance time of the opposing phase. We conclude that (i) MIB is a form of perceptual rivalry, and (ii) there may be a common oscillator responsible for timing aspects of all forms of perceptual rivalry.

The empirical basis of color perception

Rationalizing the perceptual effects of spectral stimuli has been a major challenge in vision science for at least the last 200 years. Here we review evidence that this otherwise puzzling body of phenomenology is generated by an empirical strategy of perception in which the color an observer sees is entirely determined by the probability distribution of the possible sources of the stimulus. The rationale for this strategy in color vision, as in other visual perceptual domains, is the inherent ambiguity of the real-world origins of any spectral stimulus.

A probabilistic explanation of brightness scaling

The perceptions of lightness or brightness elicited by a visual target are linked to its luminance by a nonlinear function that varies according to the physical characteristics of the target and the background on which it is presented. Although no generally accepted explanation of this scaling relationship exists, it has long been considered a byproduct of low- or mid-level visual processing. Here we examine the possibility that brightness scaling is actually the signature of a biological strategy for dealing with inevitably ambiguous visual stimuli, in which percepts of lightness/brightness are determined by the probabilistic relationship between luminances in the image plane and their possible real-world sources.

Perception of objects that are translating and rotating

The motion of objects that are both translating and rotating can be decomposed into an infinite number of translational and rotational combinations. How, then, do such stimuli routinely elicit specific percepts and behavioral responses that are usually appropriate? A possible answer is that motion percepts are fully determined by the probability distributions of all the possible correspondences and differences in the stimulus sequence. To test the merits of this conceptual framework, we investigated the perceived motion elicited by a line that is both translating and rotating behind an aperture. When stimuli are presented such that a particular sequence of appearance and disappearance occurs at the aperture boundary, subjects report that the line is rotating only; furthermore, the perceived centers of rotation appear to describe a cycloidal trajectory, even when one aperture shape is replaced by another. These and other perceptual effects elicited by translating and rotating stimuli are all accurately predicted by the probability distribution of the possible sources of the physical movements, supporting the conclusion that motion perception is indeed generated by a wholly probabilistic strategy.

An empirical explanation of the Chubb illusion

The perceived difference in brightness between elements of a patterned target is diminished when the target is embedded in a similar surround of higher luminance contrast (the Chubb illusion). Here we show that this puzzling effect can be explained by the degree to which imperfect transmittance is likely to have affected the light that reaches the eye. These observations indicate that this 'illusion' is yet another signature of the fundamentally empirical strategy of visual perception, in this case generated by the typical influence of transmittance on inherently ambiguous stimuli.

A wholly empirical explanation of perceived motion

Because the retinal activity generated by a moving object cannot specify which of an infinite number of possible physical displacements underlies the stimulus, its real-world cause is necessarily uncertain. How, then, do observers respond successfully to sequences of images whose provenance is ambiguous? Here we explore the hypothesis that the visual system solves this problem by a probabilistic strategy in which perceived motion is generated entirely according to the relative frequency of occurrence of the physical sources of the stimulus. The merits of this concept were tested by comparing the directions and speeds of moving lines reported by subjects to the values determined by the probability distribution of all the possible physical displacements underlying the stimulus. The velocities reported by observers in a variety of stimulus contexts can be accounted for in this way.