Numerosity processing in early visual cortex

Visual sense of number vs. sense of magnitude in humans and machines

Numerosity perception is thought to be foundational to mathematical learning, but its computational bases are strongly debated. Some investigators argue that humans are endowed with a specialized system supporting numerical representations; others argue that visual numerosity is estimated using continuous magnitudes, such as density or area, which usually co-vary with number. Here we reconcile these contrasting perspectives by testing deep neural networks on the same numerosity comparison task that was administered to human participants, using a stimulus space that allows the precise measurement of the contribution of non-numerical features. Our model accurately simulates the psychophysics of numerosity perception and the associated developmental changes: discrimination is driven by numerosity, but non-numerical features also have a significant impact, especially early during development. Representational similarity analysis further highlights that both numerosity and continuous magnitudes are spontaneously encoded in deep networks even when no task has to be carried out, suggesting that numerosity is a major, salient property of our visual environment.

Unconscious Number Discrimination in the Human Visual System

How do humans compute approximate number? According to one influential theory, approximate number representations arise in the intraparietal sulcus and are amodal, meaning that they arise independent of any sensory modality. Alternatively, approximate number may be computed initially within sensory systems. Here we tested for sensitivity to approximate number in the visual system using steady state visual evoked potentials. We recorded electroencephalography from humans while they viewed dotclouds presented at 30 Hz, which alternated in numerosity (ranging from 10 to 20 dots) at 15 Hz. At this rate, each dotcloud backward masked the previous dotcloud, disrupting top-down feedback to visual cortex and preventing conscious awareness of the dotclouds' numerosities. Spectral amplitude at 15 Hz measured over the occipital lobe (Oz) correlated positively with the numerical ratio of the stimuli, even when nonnumerical stimulus attributes were controlled, indicating that subjects' visual systems were differentiating dotclouds on the basis of their numerical ratios. Crucially, subjects were unable to discriminate the numerosities of the dotclouds consciously, indicating the backward masking of the stimuli disrupted reentrant feedback to visual cortex. Approximate number appears to be computed within the visual system, independently of higher-order areas, such as the intraparietal sulcus.

Distinct strategies for estimating the temporal average of numerical and perceptual information

Humans can estimate global trends in dynamic information presented either as perceptual features or as symbolic codes such as numbers. Previous studies on temporal statistics estimation have shown that observers judge the temporal average of visual attributes according to information from the last few frames of the presentation sequence (in what is referred to as the recency effect). Here, we investigated how humans estimate the temporal average of number vs. orientation using identical stimuli for the two tasks. In Experiment 1, a randomly-selected single-digit number was serially presented at orientations randomly varying over time. In Experiment 2, a texture comprising a random number of Gabor elements was shown at orientations randomly varying over time. In both experiments, observers judged the temporal averages of the numerical values and orientations in separate blocks. Results showed that observers judging the temporal average of orientation relied upon information from later frames as predicted by a typical model of perceptual decision making. By contrast, for the judgement of numerical values, we found that the impacts of each temporal frame were constant or varied little across temporal frames regardless of whether the numerical information was given as digits or by the number of texture elements. The results are interpreted as evidence that distinct computational strategies may be involved in estimating the temporal averages of perceptual features and numerical information.

The neural signature of numerosity by separating numerical and continuous magnitude extraction in visual cortex with frequency-tagged EEG

The ability to handle approximate quantities, or number sense, has been recurrently linked to mathematical skills, although the nature of the mechanism allowing to extract numerical information (i.e., numerosity) from environmental stimuli is still debated. A set of objects is indeed not only characterized by its numerosity but also by other features, such as the summed area occupied by the elements, which often covary with numerosity. These intrinsic relations between numerosity and nonnumerical magnitudes led some authors to argue that numerosity is not independently processed but extracted through a weighting of continuous magnitudes. This view cannot be properly tested through classic behavioral and neuroimaging approaches due to these intrinsic correlations. The current study used a frequency-tagging EEG approach to separately measure responses to numerosity as well as to continuous magnitudes. We recorded occipital responses to numerosity, total area, and convex hull changes but not to density and dot size. We additionally applied a model predicting primary visual cortex responses to the set of stimuli. The model output was closely aligned with our electrophysiological data, since it predicted discrimination only for numerosity, total area, and convex hull. Our findings thus demonstrate that numerosity can be independently processed at an early stage in the visual cortex, even when completely isolated from other magnitude changes. The similar implicit discrimination for numerosity as for some continuous magnitudes, which correspond to basic visual percepts, shows that both can be extracted independently, hence substantiating the nature of numerosity as a primary feature of the visual scene.

Neural Dynamics of Serial Dependence in Numerosity Perception

Serial dependence—an attractive perceptual bias whereby a current stimulus is perceived to be similar to previously seen ones—is thought to represent the process that facilitates the stability and continuity of visual perception. Recent results demonstrate a neural signature of serial dependence in numerosity perception, emerging very early in the time course during perceptual processing. However, whether such a perceptual signature is retained after the initial processing remains unknown. Here, we address this question by investigating the neural dynamics of serial dependence using a recently developed technique that allowed a reactivation of hidden memory states. Participants performed a numerosity discrimination task during EEG recording, with task-relevant dot array stimuli preceded by a task-irrelevant stimulus inducing serial dependence. Importantly, the neural network storing the representation of the numerosity stimulus was perturbed (or pinged) so that the hidden states of that representation can be explicitly quantified. The results first show that a neural signature of serial dependence emerges early in the brain signals, starting soon after stimulus onset. Critical to the central question, the pings at a later latency could successfully reactivate the biased representation of the initial stimulus carrying the signature of serial dependence. These results provide one of the first pieces of empirical evidence that the biased neural representation of a stimulus initially induced by serial dependence is preserved throughout a relatively long period.

Challenging the neurobiological link between number sense and symbolic numerical abilities

A significant body of research links individual differences in symbolic numerical abilities, such as arithmetic, to number sense, the neurobiological system used to approximate and manipulate quantities without language or symbols. However, recent findings from cognitive neuroscience challenge this influential theory. Our current review presents an overview of evidence for the number sense account of symbolic numerical abilities and then reviews recent studies that challenge this account, organized around the following four assertions. (1) There is no number sense as traditionally conceived. (2) Neural substrates of number sense are more widely distributed than common consensus asserts, complicating the neurobiological evidence linking number sense to numerical abilities. (3) The most common measures of number sense are confounded by other cognitive demands, which drive key correlations. (4) Number sense and symbolic number systems (Arabic digits, number words, and so on) rely on distinct neural mechanisms and follow independent developmental trajectories. The review follows each assertion with comments on future directions that may bring resolution to these issues.

Attentional amplification of neural codes for number independent of other quantities along the dorsal visual stream

Humans and other animals base important decisions on estimates of number, and intraparietal cortex is thought to provide a crucial substrate of this ability. However, it remains debated whether an independent neuronal processing mechanism underlies this ‘number sense’, or whether number is instead judged indirectly on the basis of other quantitative features. We performed high-resolution 7 Tesla fMRI while adult human volunteers attended either to the numerosity or an orthogonal dimension (average item size) of visual dot arrays. Along the dorsal visual stream, numerosity explained a significant amount of variance in activation patterns, above and beyond non-numerical dimensions. Its representation was selectively amplified and progressively enhanced across the hierarchy when task relevant. Our results reveal a sensory extraction mechanism yielding information on numerosity separable from other dimensions already at early visual stages and suggest that later regions along the dorsal stream are most important for explicit manipulation of numerical quantity.

Numbers and the ability to count and calculate are an essential part of human culture. They are part of everyday life, featuring in calendars, computers or the weekly shop, but also in some of humanity’s biggest achievements: without them the pyramids or space travel would not exist. A precursor of sophisticated mathematical skill could reside in a simpler mental ability: the capacity to assess numerical quantities at a glance. This ‘number sense’ appears in humans in early childhood and it is also present in other animals, but it is still poorly understood.

Brain imaging techniques have identified the parts of the brain that are active when perceiving numbers or making calculations. As techniques have advanced, it has become possible to resolve fine differences in brain activity that occur when people switch their attention between different visual tasks. But how exactly does the human brain process visual information to make sense of numbers? One theory suggests that humans use visual cues, such as the size of a group of objects or how densely packed objects are, to estimate numbers. On the other hand, it is also possible that humans can sense number directly, without reference to other properties of the group being observed.

Castaldi et al. presented twenty adult volunteers with groups of dots and asked them to focus either on the number of dots or on the size of the dots during a brain scan. This approach allowed the separation of brain signals specific to number from signals corresponding to other visual cues, such as size or density of the group. The experiment revealed that brain activity changed depending on the number of dots displayed. The signal related to number became stronger when people focused on the number of dots, while signals related to other properties of the group remained unchanged. Moreover, brain signals for number were observed at the very early stages of visual processing, in the parts of the brain that receive input from the eyes first.

These results suggest that the human visual system perceives number directly, and not by processing information about the size or density of a group of objects. This finding provides insights into how human brains encode numbers, which could be important to understand disorders where number sense can be impaired leading to difficulties learning math and operating with numbers.

Asymmetrical interference between number and item size perception provides evidence for a domain specific impairment in dyscalculia

Dyscalculia, a specific learning disability that impacts arithmetical skills, has previously been associated to a deficit in the precision of the system that estimates the approximate number of objects in visual scenes (the so called 'number sense' system). However, because in tasks involving numerosity comparisons dyscalculics' judgements appears disproportionally affected by continuous quantitative dimensions (such as the size of the items), an alternative view linked dyscalculia to a domain-general difficulty in inhibiting task-irrelevant responses. To arbitrate between these views, we evaluated the degree of reciprocal interference between numerical and non-numerical quantitative dimensions in adult dyscalculics and matched controls. We used a novel stimulus set orthogonally varying in mean item size and numerosity, putting particular attention into matching both features' perceptual discriminability. Participants compared those stimuli based on each of the two dimensions. While control subjects showed no significant size interference when judging numerosity, dyscalculics' numerosity judgments were strongly biased by the unattended size dimension. Importantly however, both groups showed the same degree of interference from the unattended dimension when judging mean size. Moreover, only the ability to discard the irrelevant size information when comparing numerosity (but not the reverse) significantly predicted calculation ability across subjects. Overall, our results show that numerosity discrimination is less prone to interference than discrimination of another quantitative feature (mean item size) when the perceptual discriminability of these features is matched, as here in control subjects. By quantifying, for the first time, dyscalculic subjects' degree of interference on another orthogonal dimension of the same stimuli, we are able to exclude a domain-general inhibition deficit as explanation for their poor / biased numerical judgement. We suggest that enhanced reliance on non-numerical cues during numerosity discrimination can represent a strategy to cope with a less precise number sense.

Early Numerosity Encoding in Visual Cortex Is Not Sufficient for the Representation of Numerical Magnitude

Recent studies have demonstrated that the numerosity of visually presented dot arrays is represented in low-level visual cortex extremely early in latency. However, whether or not such an early neural signature reflects the perceptual representation of numerosity remains unknown. Alternatively, such a signature may indicate the raw sensory representation of the dot-array stimulus before becoming the perceived representation of numerosity. Here, we addressed this question by using the connectedness illusion, whereby arrays with pairwise connected dots are perceived to be less numerous compared with arrays containing isolated dots. Using EEG and fMRI in two independent experiments, we measured neural responses to dot-array stimuli comprising 16 or 32 dots, either isolated or pairwise connected. The effect of connectedness, which reflects the segmentation of the visual stimulus into perceptual units, was observed in the neural activity after 150 msec post stimulus onset in the EEG experiment and in area V3 in the fMRI experiment using a multivariate pattern analysis. In contrast, earlier neural activity before 100 msec and in area V2 was strictly modulated by numerosity regardless of connectedness, suggesting that this early activity reflects the sensory representation of a dot array before perceptual segmentation. Our findings thus demonstrate that the neural representation for numerosity in early visual cortex is not sufficient for visual number perception and suggest that the perceptual encoding of numerosity occurs at or after the segmentation process that takes place later in area V3.

Serial dependence in numerosity perception

Our conscious experience of the external world is remarkably stable and seamless, despite the intrinsically discontinuous and noisy nature of sensory information. Serial dependencies in visual perception-reflecting attractive biases making a current stimulus to appear more similar to previous ones-have been recently hypothesized to be involved in perceptual continuity. However, while these effects have been observed across a variety of visual features and at the neural level, several aspects of serial dependence and how it generalizes across visual dimensions is still unknown. Here we explore the behavioral signature of serial dependence in numerosity perception by assessing how the perceived numerosity of dot-array stimuli is biased by a task-irrelevant "inducer" stimulus presented before task-relevant stimuli. First, although prior work suggests that numerosity perception starts in the subcortex, the current study rules out a possible involvement of subcortical processing in serial dependence, confirming that the effect likely starts in the visual cortex. Second, we show that the effect is coarsely spatially localized to the position of the inducer stimulus. Third, we demonstrate that the effect is present even with a stimulus presentation procedure minimizing the involvement of post-perceptual processes, but only when participants actively pay attention to the inducer stimulus. Overall, these results provide a comprehensive characterization of serial dependencies in numerosity perception, demonstrating that attractive biases occur by means of spatially localized attentional modulations of early sensory activity.

Examining the Triple Code Model in numerical cognition: An fMRI study

The Triple Code Model (TCM) of numerical cognition argues for the existence of three representational codes for number: Arabic digits, verbal number words, and analog nonsymbolic magnitude representations, each subserved by functionally dissociated neural substrates. Despite the popularity of the TCM, no study to date has explored all three numerical codes within one fMRI paradigm. We administered three tasks, associated with each of the aforementioned numerical codes, in order to explore the neural correlates of numerosity processing in a sample of adults (N = 46). Independent task-control contrast analyses revealed task-dependent activity in partial support of the model, but also highlight the inherent complexity of a distributed and overlapping fronto-parietal network involved in all numerical codes. The results indicate that the TCM correctly predicts the existence of some functionally dissociated neural substrates, but requires an update that accounts for interactions with attentional processes. Parametric contrasts corresponding to differences in task difficulty revealed specific neural correlates of the distance effect, where closely spaced numbers become more difficult to discriminate than numbers spaced further apart. A conjunction analysis illustrated overlapping neural correlates across all tasks, in line with recent proposals for a fronto-parietal network of number processing. We additionally provide tentative results suggesting the involvement of format-independent numerosity-sensitive retinotopic maps in the early visual stream, extending previous findings of nonsymbolic stimulus selectivity. We discuss the functional roles of the components associated with the model, as well as the purported fronto-parietal network, and offer arguments in favor of revising the TCM.

Motion-induced compression of perceived numerosity

It has been recently proposed that space, time, and number might share a common representation in the brain. Evidence supporting this idea comes from adaptation studies demonstrating that prolonged exposure to a given stimulus feature distorts the perception of different characteristics. For example, visual motion adaptation affects both perceived position and duration of subsequent stimuli presented in the adapted location. Here, we tested whether motion adaptation also affects perceived numerosity, by testing the effect of adaptation to translating or rotating stimuli moving either at high (20 Hz) or low (5 Hz) speed. Adaptation to fast translational motion yielded a robust reduction in the apparent numerosity of the adapted stimulus (~25%) while adaptation to slow translational or circular motion (either 20 Hz or 5 Hz) yielded a weaker but still significant compression. Control experiments suggested that none of these results could be accounted for in terms of stimulus masking. Taken together, our results are consistent with the extant literature supporting the idea of a generalized magnitude system underlying the representation of numerosity, space and time via common metrics. However, as changes in perceived numerosity co-varied with both adapting motion profile and speed, our evidence also suggests complex and asymmetric interactions between different magnitude representations.

Numerical encoding in early visual cortex

The ability to estimate numerosity in a visual array arose early in evolution, develops early in human development, and is correlated with mathematical ability. Previous work with visually presented arrays indicates that the intraparietal sulcus (IPS) represents number. However, it is not clear if the number signal originates in IPS or is propagated from earlier visual areas. Previous work from our group has demonstrated a rapidly instantiated representation of number in low-level regions of visual cortex using the high temporal resolution of event-related electro-encephalography (EEG). Here, we use a rapid event-related functional magnetic resonance imaging (fMRI) paradigm and find convergent evidence for a number signal in low-level visual cortex (areas V1, V2, and V3). Employing a stringent set of stimulus controls, we demonstrate that this signal cannot be explained by the total extent of the array, the density of the items in the array, the aggregate visual area of the items, the size of individual items, the proportion of the array covered by items, nor the overall scale of the array and items. Our findings thus provide strong support for the hypothesis that number is rapidly and directly encoded early in the visual processing stream.

Attractive Serial Dependence in the Absence of an Explicit Task

Attractive serial dependence refers to an adaptive change in the representation of sensory information, whereby a current stimulus appears to be similar to a previous one. The nature of this phenomenon is controversial, however, as serial dependence could arise from biased perceptual representations or from biased traces of working memory representation at a decisional stage. Here, we demonstrated a neural signature of serial dependence in numerosity perception emerging early in the visual processing stream even in the absence of an explicit task. Furthermore, a psychophysical experiment revealed that numerosity perception is biased by a previously presented stimulus in an attractive way, not by repulsive adaptation. These results suggest that serial dependence is a perceptual phenomenon starting from early levels of visual processing and occurring independently from a decision process, which is consistent with the view that these biases smooth out noise from neural signals to establish perceptual continuity.