The visual system supports online translation invariance for object identification

Deep problems with neural network models of human vision

Deep neural networks (DNNs) have had extraordinary successes in classifying photographic images of objects and are often described as the best models of biological vision. This conclusion is largely based on three sets of findings: (1) DNNs are more accurate than any other model in classifying images taken from various datasets, (2) DNNs do the best job in predicting the pattern of human errors in classifying objects taken from various behavioral datasets, and (3) DNNs do the best job in predicting brain signals in response to images taken from various brain datasets (e.g., single cell responses or fMRI data). However, these behavioral and brain datasets do not test hypotheses regarding what features are contributing to good predictions and we show that the predictions may be mediated by DNNs that share little overlap with biological vision. More problematically, we show that DNNs account for almost no results from psychological research. This contradicts the common claim that DNNs are good, let alone the best, models of human object recognition. We argue that theorists interested in developing biologically plausible models of human vision need to direct their attention to explaining psychological findings. More generally, theorists need to build models that explain the results of experiments that manipulate independent variables designed to test hypotheses rather than compete on making the best predictions. We conclude by briefly summarizing various promising modeling approaches that focus on psychological data.

Positional cueing, string location variability, and letter-in-string identification

In three experiments we measured accuracy in identifying a single letter among a string of five briefly presented consonants followed by a post-mask. The position of the to-be-identified letter was either indicated by an ordinal cue (e.g., position 2) or an underscore cue (e.g., #####). In Experiment 1 the ordinal cue was presented prior to onset of the letter string, and the underscore cue presented at string offset. In Experiments 2 and 3, both the ordinal and the underscore cues were pre-cues. In all experiments, letter strings could either appear centered on fixation or shifted randomly to the left or to the right. Participants were tested in separate blocks of trials for each of the four conditions generated by the combination of cue-type and string-location variability. In Experiment 1, letter identification accuracy was higher with ordinal cues and with fixed string locations, and ordinal cueing was more affected by string location variability. In Experiments 2 and 3, letter identification accuracy was higher with underscore pre-cues. We conclude that under conditions of brief stimulus durations (100 ms) and backward masking, letter-in-string identification accuracy is determined by read-out from location-specific letter detectors, independently of the type of cueing. Differences in the effectiveness of different types of cue are determined by differences in the ease of isolating a given gaze-centered location, and by differences in the ease with which attention can be directed to that location.

The human visual system and CNNs can both support robust online translation tolerance following extreme displacements

Visual translation tolerance refers to our capacity to recognize objects over a wide range of different retinal locations. Although translation is perhaps the simplest spatial transform that the visual system needs to cope with, the extent to which the human visual system can identify objects at previously unseen locations is unclear, with some studies reporting near complete invariance over 10 degrees and other reporting zero invariance at 4 degrees of visual angle. Similarly, there is confusion regarding the extent of translation tolerance in computational models of vision, as well as the degree of match between human and model performance. Here, we report a series of eye-tracking studies (total N = 70) demonstrating that novel objects trained at one retinal location can be recognized at high accuracy rates following translations up to 18 degrees. We also show that standard deep convolutional neural networks (DCNNs) support our findings when pretrained to classify another set of stimuli across a range of locations, or when a global average pooling (GAP) layer is added to produce larger receptive fields. Our findings provide a strong constraint for theories of human vision and help explain inconsistent findings previously reported with convolutional neural networks (CNNs).

Orthographic processing: A 'mid-level' vision of reading: The 44th Sir Frederic Bartlett Lecture

I will describe how orthographic processing acts as a central interface between visual and linguistic processing during reading, and as such can be considered to be the 'mid-level vision' of reading research. In order to make this case, I first summarize the evidence in favour of letter-based word recognition before examining work investigating how orthographic similarities among words influence single word reading. I describe how evidence gradually accumulated against traditional measures of orthographic similarity and the associated theories of orthographic processing, forcing a reconsideration of how letter-position information is represented by skilled readers. Then, I present the theoretical framework that was developed to explain these findings, with a focus on the distinction between location-specific and location-invariant orthographic representations. Finally, I describe work extending this theoretical framework in two main directions: first, to the realm of reading development, with the aim to specify the key changes in the processing of letters and letter strings that accompany successful learning to read, and second, to the realm of sentence reading, in order to specify how orthographic information can be processed across several words in parallel, and how skilled readers keep track of which letters belong to which words.

Parallel Distributed Processing Theory in the Age of Deep Networks

Parallel distributed processing (PDP) models in psychology are the precursors of deep networks used in computer science. However, only PDP models are associated with two core psychological claims, namely that all knowledge is coded in a distributed format and cognition is mediated by non-symbolic computations. These claims have long been debated in cognitive science, and recent work with deep networks speaks to this debate. Specifically, single-unit recordings show that deep networks learn units that respond selectively to meaningful categories, and researchers are finding that deep networks need to be supplemented with symbolic systems to perform some tasks. Given the close links between PDP and deep networks, it is surprising that research with deep networks is challenging PDP theory.