Linearly additive shape and color signals in monkey inferotemporal cortex

Comparing the Dominance of Color and Form Information across the Human Ventral Visual Pathway and Convolutional Neural Networks

Color and form information can be decoded in every region of the human ventral visual hierarchy, and at every layer of many convolutional neural networks (CNNs) trained to recognize objects, but how does the coding strength of these features vary over processing? Here, we characterize for these features both their absolute coding strength-how strongly each feature is represented independent of the other feature-and their relative coding strength-how strongly each feature is encoded relative to the other, which could constrain how well a feature can be read out by downstream regions across variation in the other feature. To quantify relative coding strength, we define a measure called the form dominance index that compares the relative influence of color and form on the representational geometry at each processing stage. We analyze brain and CNN responses to stimuli varying based on color and either a simple form feature, orientation, or a more complex form feature, curvature. We find that while the brain and CNNs largely differ in how the absolute coding strength of color and form vary over processing, comparing them in terms of their relative emphasis of these features reveals a striking similarity: For both the brain and for CNNs trained for object recognition (but not for untrained CNNs), orientation information is increasingly de-emphasized, and curvature information is increasingly emphasized, relative to color information over processing, with corresponding processing stages showing largely similar values of the form dominance index.

Representation of Color, Form, and their Conjunction across the Human Ventral Visual Pathway

Despite decades of research, our understanding of the relationship between color and form processing in the primate ventral visual pathway remains incomplete. Using fMRI multivoxel pattern analysis, we examined coding of color and form, using a simple form feature (orientation) and a mid-level form feature (curvature), in human ventral visual processing regions. We found that both color and form could be decoded from activity in early visual areas V1 to V4, as well as in the posterior color-selective region and shape-selective regions in ventral and lateral occipitotemporal cortex defined based on their univariate selectivity to color or shape, respectively (the central color region only showed color but not form decoding). Meanwhile, decoding biases towards one feature or the other existed in the color- and shape-selective regions, consistent with their univariate feature selectivity reported in past studies. Additional extensive analyses show that while all these regions contain independent (linearly additive) coding for both features, several early visual regions also encode the conjunction of color and the simple, but not the complex, form feature in a nonlinear, interactive manner. Taken together, the results show that color and form are encoded in a biased distributed and largely independent manner across ventral visual regions in the human brain.

Encoding of Partially Occluded and Occluding Objects in Primate Inferior Temporal Cortex

Object segmentation-the process of parsing visual scenes-is essential for object recognition and scene understanding. We investigated how responses of neurons in macaque inferior temporal (IT) cortex contribute to object segmentation under partial occlusion. Specifically, we asked whether IT responses to occluding and occluded objects are bound together as in the visual image or linearly separable reflecting their segmentation. We recorded the activity of 121 IT neurons while two male animals performed a shape discrimination task under partial occlusion. We found that for a majority (60%) of neurons, responses were enhanced by partial occlusion, but they were only weakly shape selective for the discriminanda at all levels of occlusion. Enhancement of IT responses in these neurons depended largely on the area of occlusion but only minimally on the color and shape of the occluding dots. In contrast to the above group of neurons, a sizable minority responded best to the unoccluded stimulus and showed strong selectivity for the shape of the discriminanda. In these neurons, response magnitude and shape selectivity declined with increasing levels of occlusion. Simulations revealed that the response characteristics of both classes of neurons were consistent with a model in which the responses to the occluded shape and the occluders are weighted separately and linearly combined. Overall, our results support the hypothesis that information about occluded and occluding stimuli are linearly separable and easily decodable from IT responses and that IT neurons encode a segmented representation of the visual scene.SIGNIFICANCE STATEMENT Recognizing partially occluded objects can be challenging, yet the primate visual system achieves it rapidly and effortlessly. For successful recognition in the face of occlusion, segmentation of the occluded and occluding objects is a critical first step. Using a combination of experimental data and simulations, here we demonstrate that responses of neurons in macaque IT cortex, the highest stage of the form processing pathway, reflect occluded and occluding stimuli as segmented components and are not bound together as they appear in the visual image. These results support the idea that segmentation and perception of occluded and occluding stimuli are directly mirrored in the responses of neurons in the highest form processing stages.

Joint representation of color and form in convolutional neural networks: A stimulus-rich network perspective

To interact with real-world objects, any effective visual system must jointly code the unique features defining each object. Despite decades of neuroscience research, we still lack a firm grasp on how the primate brain binds visual features. Here we apply a novel network-based stimulus-rich representational similarity approach to study color and form binding in five convolutional neural networks (CNNs) with varying architecture, depth, and presence/absence of recurrent processing. All CNNs showed near-orthogonal color and form processing in early layers, but increasingly interactive feature coding in higher layers, with this effect being much stronger for networks trained for object classification than untrained networks. These results characterize for the first time how multiple basic visual features are coded together in CNNs. The approach developed here can be easily implemented to characterize whether a similar coding scheme may serve as a viable solution to the binding problem in the primate brain.

Temporal organization of color and shape processing during visual search

The mechanisms guiding visual attention are of great interest within cognitive and perceptual psychology. Many researchers have proposed models of these mechanisms, which serve to both formalize their theories and to guide further empirical investigations. The assumption that a number of basic features are processed in parallel early in the attentional process is common among most models of visual attention and visual search. To date, much of the evidence for parallel processing has been limited to set-size manipulations. Unfortunately, set-size manipulations have been shown to be insufficient evidence for parallel processing. We applied Systems Factorial Technology, a general nonparametric framework, to test this assumption, specifically whether color and shape are processed in parallel or in serial, in three experiments representative of feature search, conjunctive search, and odd-one-out search, respectively. Our results provide strong evidence that color and shape information guides search through parallel processes. Furthermore, we found evidence for facilitation between color and shape when the target was known in advance but performance consistent with unlimited capacity, independent parallel processing in odd-one-out search. These results confirm core assumptions about color and shape feature processing instantiated in most models of visual search and provide more detailed clues about the manner in which color and shape information is combined to guide search.

Object shape and surface properties are jointly encoded in mid-level ventral visual cortex

Recognizing a myriad visual objects rapidly is a hallmark of the primate visual system. Traditional theories of object recognition have focused on how crucial form features, for example, the orientation of edges, may be extracted in early visual cortex and utilized to recognize objects. An alternative view argues that much of early and mid-level visual processing focuses on encoding surface characteristics, for example, texture. Neurophysiological evidence from primate area V4 supports a third alternative - the joint, but independent, encoding of form and texture - that would be advantageous for segmenting objects from the background in natural scenes and for object recognition that is independent of surface texture. Future studies that leverage deep convolutional network models, especially focusing on network failures to match biology and behavior, can advance our insights into how such a joint representation of form and surface properties might emerge in visual cortex.

Parts-based representations of perceived face movements in the superior temporal sulcus

Facial motion is a primary source of social information about other humans. Prior fMRI studies have identified regions of the superior temporal sulcus (STS) that respond specifically to perceived face movements (termed fSTS), but little is known about the nature of motion representations in these regions. Here we use fMRI and multivoxel pattern analysis to characterize the representational content of the fSTS. Participants viewed a set of specific eye and mouth movements, as well as combined eye and mouth movements. Our results demonstrate that fSTS response patterns contain information about face movements, including subtle distinctions between types of eye and mouth movements. These representations generalize across the actor performing the movement, and across small differences in visual position. Critically, patterns of response to combined movements could be well predicted by linear combinations of responses to individual eye and mouth movements, pointing to a parts-based representation of complex face movements. These results indicate that the fSTS plays an intermediate role in the process of inferring social content from visually perceived face movements, containing a representation that is sufficiently abstract to generalize across low-level visual details, but still tied to the kinematics of face part movements.

Beyond Rehabilitation of Acuity, Ocular Alignment, and Binocularity in Infantile Strabismus

Infantile strabismus impairs the perception of all attributes of the visual scene. High spatial frequency components are no longer visible, leading to amblyopia. Binocularity is altered, leading to the loss of stereopsis. Spatial perception is impaired as well as detection of vertical orientation, the fastest movements, directions of movement, the highest contrasts and colors. Infantile strabismus also affects other vision-dependent processes such as control of postural stability. But presently, rehabilitative therapies for infantile strabismus by ophthalmologists, orthoptists and optometrists are restricted to preventing or curing amblyopia of the deviated eye, aligning the eyes and, whenever possible, preserving or restoring binocular vision during the critical period of development, i.e., before ~10 years of age. All the other impairments are thus ignored; whether they may recover after strabismus treatment even remains unknown. We argue here that medical and paramedical professionals may extend their present treatments of the perceptual losses associated with infantile strabismus. This hypothesis is based on findings from fundamental research on visual system organization of higher mammals in particular at the cortical level. In strabismic subjects (as in normal-seeing ones), information about all of the visual attributes converge, interact and are thus inter-dependent at multiple levels of encoding ranging from the single neuron to neuronal assemblies in visual cortex. Thus if the perception of one attribute is restored this may help to rehabilitate the perception of other attributes. Concomitantly, vision-dependent processes may also improve. This could occur spontaneously, but still should be assessed and validated. If not, medical and paramedical staff, in collaboration with neuroscientists, will have to break new ground in the field of therapies to help reorganize brain circuitry and promote more comprehensive functional recovery. Findings from fundamental research studies in both young and adult patients already support our hypothesis and are reviewed here. For example, presenting different contrasts to each eye of a strabismic patient during training sessions facilitates recovery of acuity in the amblyopic eye as well as of 3D perception. Recent data also demonstrate that visual recoveries in strabismic subjects improve postural stability. These findings form the basis for a roadmap for future research and clinical development to extend presently applied rehabilitative therapies for infantile strabismus.

Multiplicative mixing of object identity and image attributes in single inferior temporal neurons

Vision is a challenging problem because the same object can produce a variety of images on the retina, mixing signals related to its identity with signals related to its viewing attributes, such as size, position, rotation, etc. Precisely how the brain separates these signals to form an efficient representation is unknown. Here, we show that single neurons in high-level visual cortex encode object identity and attribute multiplicatively and that doing so allows for better decoding of each signal.

Object recognition is challenging because the same object can produce vastly different images, mixing signals related to its identity with signals due to its image attributes, such as size, position, rotation, etc. Previous studies have shown that both signals are present in high-level visual areas, but precisely how they are combined has remained unclear. One possibility is that neurons might encode identity and attribute signals multiplicatively so that each can be efficiently decoded without interference from the other. Here, we show that, in high-level visual cortex, responses of single neurons can be explained better as a product rather than a sum of tuning for object identity and tuning for image attributes. This subtle effect in single neurons produced substantially better population decoding of object identity and image attributes in the neural population as a whole. This property was absent both in low-level vision models and in deep neural networks. It was also unique to invariances: when tested with two-part objects, neural responses were explained better as a sum than as a product of part tuning. Taken together, our results indicate that signals requiring separate decoding, such as object identity and image attributes, are combined multiplicatively in IT neurons, whereas signals that require integration (such as parts in an object) are combined additively.

The representation of colored objects in macaque color patches

An important question about color vision is how does the brain represent the color of an object? The recent discovery of “color patches” in macaque inferotemporal (IT) cortex, the part of the brain responsible for object recognition, makes this problem experimentally tractable. Here we recorded neurons in three color patches, middle color patch CLC (central lateral color patch), and two anterior color patches ALC (anterior lateral color patch) and AMC (anterior medial color patch), while presenting images of objects systematically varied in hue. We found that all three patches contain high concentrations of hue-selective cells, and that the three patches use distinct computational strategies to represent colored objects: while all three patches multiplex hue and shape information, shape-invariant hue information is much stronger in anterior color patches ALC/AMC than CLC. Furthermore, hue and object shape specifically for primate faces/bodies are over-represented in AMC, but not in the other two patches.

Neurons in the inferotemporal cortex (IT) encode object identity; however, how object color is represented here is not well understood. Here the authors report that neurons from three color patches in macaque IT encode significant information regarding the hue and shape of objects in a hierarchical manner.

Seeing a straight line on a curved surface: decoupling of patterns from surfaces by single IT neurons

We have no difficulty seeing a straight line on a curved piece of paper, but in fact, doing so requires decoupling the shape of the surface from the pattern itself. Here we report a novel form of invariance in the visual cortex: single neurons in monkey inferior temporal cortex respond similarly to congruent transformations of patterns and surfaces, in effect decoupling patterns from the surface on which they are overlaid.

We have no difficulty seeing a straight line drawn on a paper even when the paper is bent, but this inference is in fact nontrivial. Doing so requires either matching local features or representing the pattern after factoring out the surface shape. Here we show that single neurons in the monkey inferior temporal (IT) cortex show invariant responses to patterns across rigid and nonrigid changes of surfaces. We recorded neuronal responses to stimuli in which the pattern and the surrounding surface were varied independently. In a subset of neurons, we found pattern-surface interactions that produced similar responses to stimuli across congruent pattern and surface transformations. These interactions produced systematic shifts in curvature tuning of patterns when overlaid on convex and flat surfaces. Our results show that surfaces are factored out of patterns by single neurons, thereby enabling complex perceptual inferences.

NEW & NOTEWORTHY We have no difficulty seeing a straight line on a curved piece of paper, but in fact, doing so requires decoupling the shape of the surface from the pattern itself. Here we report a novel form of invariance in the visual cortex: single neurons in monkey inferior temporal cortex respond similarly to congruent transformations of patterns and surfaces, in effect decoupling patterns from the surface on which they are overlaid.

Object attributes combine additively in visual search

We perceive objects as containing a variety of attributes: local features, relations between features, internal details, and global properties. But we know little about how they combine. Here, we report a remarkably simple additive rule that governs how these diverse object attributes combine in vision. The perceived dissimilarity between two objects was accurately explained as a sum of (a) spatially tuned local contour-matching processes modulated by part decomposition; (b) differences in internal details, such as texture; (c) differences in emergent attributes, such as symmetry; and (d) differences in global properties, such as orientation or overall configuration of parts. Our results elucidate an enduring question in object vision by showing that the whole object is not a sum of its parts but a sum of its many attributes.

Color-motion feature-binding errors are mediated by a higher-order chromatic representation

Peripheral and central moving objects of the same color may be perceived to move in the same direction even though peripheral objects have a different true direction of motion [Nature429, 262 (2004)10.1038/429262a]. The perceived, illusory direction of peripheral motion is a color-motion feature-binding error. Recent work shows that such binding errors occur even without an exact color match between central and peripheral objects, and, moreover, the frequency of the binding errors in the periphery declines as the chromatic difference increases between the central and peripheral objects [J. Opt. Soc. Am. A31, A60 (2014)JOAOD60740-323210.1364/JOSAA.31.000A60]. This change in the frequency of binding errors with the chromatic difference raises the general question of the chromatic representation from which the difference is determined. Here, basic properties of the chromatic representation are tested to discover whether it depends on independent chromatic differences on the l and the s cardinal axes or, alternatively, on a more specific higher-order chromatic representation. Experimental tests compared the rate of feature-binding errors when the central and peripheral colors had the identical s chromaticity (so zero difference in s) and a fixed magnitude of l difference, while varying the identical s level in center and periphery (thus always keeping the s difference at zero). A chromatic representation based on independent l and s differences would result in the same frequency of color-motion binding errors at everyslevel. The results are contrary to this prediction, thus showing that the chromatic representation at the level of color-motion feature binding depends on a higher-order chromatic mechanism.

The physiology and psychophysics of the color-form relationship: a review

The relationship between color and form has been a long standing issue in visual science. A picture of functional segregation and topographic clustering emerges from anatomical and electrophysiological studies in animals, as well as by brain imaging studies in human. However, one of the many roles of chromatic information is to support form perception, and in some cases it can do so in a way superior to achromatic (luminance) information. This occurs both at an early, contour-detection stage, as well as in late, higher stages involving spatial integration and the perception of global shapes. Pure chromatic contrast can also support several visual illusions related to form-perception. On the other hand, form seems a necessary prerequisite for the computation and assignment of color across space, and there are several respects in which the color of an object can be influenced by its form. Evidently, color and form are mutually dependent. Electrophysiological studies have revealed neurons in the visual brain able to signal contours determined by pure chromatic contrast, the spatial tuning of which is similar to that of neurons carrying luminance information. It seems that, especially at an early stage, form is processed by several, independent systems that interact with each other, each one having different tuning characteristics in color space. At later processing stages, mechanisms able to combine information coming from different sources emerge. A clear interaction between color and form is manifested by the fact that color-form contingencies can be observed in various perceptual phenomena such as adaptation aftereffects and illusions. Such an interaction suggests a possible early binding between these two attributes, something that has been verified by both electrophysiological and fMRI studies.

Single-unit activity during natural vision: diversity, consistency, and spatial sensitivity among AF face patch neurons

Several visual areas within the STS of the macaque brain respond strongly to faces and other biological stimuli. Determining the principles that govern neural responses in this region has proven challenging, due in part to the inherently complex stimulus domain of dynamic biological stimuli that are not captured by an easily parameterized stimulus set. Here we investigated neural responses in one fMRI-defined face patch in the anterior fundus (AF) of the STS while macaques freely view complex videos rich with natural social content. Longitudinal single-unit recordings allowed for the accumulation of each neuron's responses to repeated video presentations across sessions. We found that individual neurons, while diverse in their response patterns, were consistently and deterministically driven by the video content. We used principal component analysis to compute a family of eigenneurons, which summarized 24% of the shared population activity in the first two components. We found that the most prominent component of AF activity reflected an interaction between visible body region and scene layout. Close-up shots of faces elicited the strongest neural responses, whereas far away shots of faces or close-up shots of hindquarters elicited weak or inhibitory responses. Sensitivity to the apparent proximity of faces was also observed in gamma band local field potential. This category-selective sensitivity to spatial scale, together with the known exchange of anatomical projections of this area with regions involved in visuospatial analysis, suggests that the AF face patch may be specialized in aspects of face perception that pertain to the layout of a social scene.

Dynamics of 3D view invariance in monkey inferotemporal cortex

Rotations in depth are challenging for object vision because features can appear, disappear, be stretched or compressed. Yet we easily recognize objects across views. Are the underlying representations view invariant or dependent? This question has been intensely debated in human vision, but the neuronal representations remain poorly understood. Here, we show that for naturalistic objects, neurons in the monkey inferotemporal (IT) cortex undergo a dynamic transition in time, whereby they are initially sensitive to viewpoint and later encode view-invariant object identity. This transition depended on two aspects of object structure: it was strongest when objects foreshortened strongly across views and were similar to each other. View invariance in IT neurons was present even when objects were reduced to silhouettes, suggesting that it can arise through similarity between external contours of objects across views. Our results elucidate the viewpoint debate by showing that view invariance arises dynamically in IT neurons out of a representation that is initially view dependent.

One month in the life of a neuron: longitudinal single-unit electrophysiology in the monkey visual system

Conventional recording methods generally preclude following the activity of the same neurons in awake animals across days. This limits our ability to systematically investigate the principles of neuronal specialization, or to study phenomena that evolve over multiple days such as experience-dependent plasticity. To redress this shortcoming, we developed a drivable, chronically implanted microwire recording preparation that allowed us to follow visual responses in inferotemporal (IT) cortex in awake behaving monkeys across multiple days, and in many cases across months. The microwire bundle and other implanted components were MRI compatible and thus permitted in the same animals both functional imaging and long-term recording from multiple neurons in deep structures within a region the approximate size of one voxel (<1 mm). The distinct patterns of stimulus selectivity observed in IT neurons, together with stable features in spike waveforms and interspike interval distributions, allowed us to track individual neurons across weeks and sometimes months. The long-term consistency of visual responses shown here permits large-scale mappings of neuronal properties using massive image libraries presented over the course of days. We demonstrate this possibility by screening the visual responses of single neurons to a set of 10,000 stimuli.

Acceptably aware during general anaesthesia: 'dysanaesthesia'--the uncoupling of perception from sensory inputs

This review makes the case for 'dysanaesthesia', a term encompassing states of mind that can arise in the course of anaesthesia during surgery, characterised by an uncoupling of sensation and perceptual experience. This is reflected in a macroscopic, functional model of anaesthetically-relevant consciousness. Patients in this state can be aware of events but in a neutral way, not in pain, sometimes personally dissociated from the experiences. This makes events associated with surgery peripheral to their whole experience, such that recall is less likely and if it exists, makes any spontaneous report of awareness unlikely. This state of perception-sensation uncoupling is therefore broadly acceptable (a minimum requirement for acceptable anaesthesia) but since it is likely a dose-related phenomenon, may also represent a precursor for awareness with adverse recall. This hypothesis uniquely explains the often inconsistent responses seen during the experimental paradigm of the 'isolated forearm technique', wherein apparently anaesthetised patients exhibit a positive motor response to verbal command, but no spontaneous movement to surgery. The hypothesis can also explain the relatively high incidence of positive response to relatively direct questions for recall (e.g., using the Brice questionnaire; ∼1:500; the vast majority of these being neutral reports) versus the very low incidence of spontaneous reports of awareness (∼1:15,000; a higher proportion of these being adverse recollections). The hypothesis is consistent with relevant notions from philosophical discussions of consciousness, and neuroscientific evidence. Dysanaesthesia has important implications for research and also for the development of appropriate monitoring.

Features in visual search combine linearly

Single features such as line orientation and length are known to guide visual search, but relatively little is known about how multiple features combine in search. To address this question, we investigated how search for targets differing in multiple features (intensity, length, orientation) from the distracters is related to searches for targets differing in each of the individual features. We tested race models (based on reaction times) and co-activation models (based on reciprocal of reaction times) for their ability to predict multiple feature searches. Multiple feature searches were best accounted for by a co-activation model in which feature information combined linearly (r = 0.95). This result agrees with the classic finding that these features are separable i.e., subjective dissimilarity ratings sum linearly. We then replicated the classical finding that the length and width of a rectangle are integral features-in other words, they combine nonlinearly in visual search. However, to our surprise, upon including aspect ratio as an additional feature, length and width combined linearly and this model outperformed all other models. Thus, length and width of a rectangle became separable when considered together with aspect ratio. This finding predicts that searches involving shapes with identical aspect ratio should be more difficult than searches where shapes differ in aspect ratio. We confirmed this prediction on a variety of shapes. We conclude that features in visual search co-activate linearly and demonstrate for the first time that aspect ratio is a novel feature that guides visual search.

Shape encoding consistency across colors in primate V4

Neurons in primate cortical area V4 are sensitive to the form and color of visual stimuli. To determine whether form selectivity remains consistent across colors, we studied the responses of single V4 neurons in awake monkeys to a set of two-dimensional shapes presented in two different colors. For each neuron, we chose two colors that were visually distinct and that evoked reliable and different responses. Across neurons, the correlation coefficient between responses in the two colors ranged from -0.03 to 0.93 (median 0.54). Neurons with highly consistent shape responses, i.e., high correlation coefficients, showed greater dispersion in their responses to the different shapes, i.e., greater shape selectivity, and also tended to have less eccentric receptive field locations; among shape-selective neurons, shape consistency ranged from 0.16 to 0.93 (median 0.63). Consistency of shape responses was independent of the physical difference between the stimulus colors used and the strength of neuronal color tuning. Finally, we found that our measurement of shape response consistency was strongly influenced by the number of stimulus repeats: consistency estimates based on fewer than 10 repeats were substantially underestimated. In conclusion, our results suggest that neurons that are likely to contribute to shape perception and discrimination exhibit shape responses that are largely consistent across colors, facilitating the use of simpler algorithms for decoding shape information from V4 neuronal populations.

Reciprocal connectivity of identified color-processing modules in the monkey inferior temporal cortex

The inferior temporal (IT) cortex is the last unimodal visual area in the ventral visual pathway and is essential for color discrimination. Recent imaging and electrophysiological studies have revealed the presence of several distinct patches of color-selective cells in the anterior IT cortex (AIT) and posterior IT cortex (PIT). To understand the neural machinery for color processing in the IT cortex, in the present study, we combined anatomical tracing methods with electrophysiological unit recordings to investigate the anatomical connections of identified clusters of color-selective cells in monkey IT cortex. We found that a color cluster in AIT received projections from a color cluster in PIT as well as from discrete clusters of cells in other occipitotemporal areas, in the superior temporal sulcus, and in prefrontal and parietal cortices. The distribution of the labeled cells in PIT closely corresponded with that of the physiologically identified color-selective cells in this region. Furthermore, retrograde tracer injections in the posterior color cluster resulted in labeled cells in the anterior cluster. Thus, temporal lobe color-processing modules form a reciprocally interconnected loop within a distributed network.

Responses to compound objects in monkey inferotemporal cortex: the whole is equal to the sum of the discrete parts

It is commonly thought that neurons in monkey inferotemporal cortex are conjunction selective--that a neuron will respond to an image if and only if it contains a required combination of parts. However, this view is based on the results of experiments manipulating closely adjacent or confluent parts. Neurons may have been sensitive not to the conjunction of parts as such but to the presence of a unique feature created where they abut. Here, we compare responses to two sets of images, one composed of spatially separate and the other of abutting parts. We show that the influences of spatially separate parts combine, to a very close approximation, according to a linear rule. Nonlinearities are more prominent--although still weak--in responses to images composed of abutting parts.