Internal representations for face detection: an application of noise-based image classification to BOLD responses

Rapid neural categorization of facelike objects predicts the perceptual awareness of a face (face pareidolia)

The human brain rapidly and automatically categorizes faces vs. other visual objects. However, whether face-selective neural activity predicts the subjective experience of a face - perceptual awareness - is debated. To clarify this issue, here we use face pareidolia, i.e., the illusory perception of a face, as a proxy to relate the neural categorization of a variety of facelike objects to conscious face perception. In Experiment 1, scalp electroencephalogram (EEG) is recorded while pictures of human faces or facelike objects - in different stimulation sequences - are interleaved every second (i.e., at 1 Hz) in a rapid 6-Hz train of natural images of nonface objects. Participants do not perform any explicit face categorization task during stimulation, and report whether they perceived illusory faces post-stimulation. A robust categorization response to facelike objects is identified at 1 Hz and harmonics in the EEG frequency spectrum with a facelike occipito-temporal topography. Across all individuals, the facelike categorization response is of about 20% of the response to human faces, but more strongly right-lateralized. Critically, its amplitude is much larger in participants who report having perceived illusory faces. In Experiment 2, facelike or matched nonface objects from the same categories appear at 1 Hz in sequences of nonface objects presented at variable stimulation rates (60 Hz to 12 Hz) and participants explicitly report after each sequence whether they perceived illusory faces. The facelike categorization response already emerges at the shortest stimulus duration (i.e., 17 ms at 60 Hz) and predicts the behavioral report of conscious perception. Strikingly, neural facelike-selectivity emerges exclusively when participants report illusory faces. Collectively, these experiments characterize a neural signature of face pareidolia in the context of rapid categorization, supporting the view that face-selective brain activity reliably predicts the subjective experience of a face from a single glance at a variety of stimuli.

Decreased frontotemporal connectivity in patients with parkinson's disease experiencing face pareidolia

The precise neural underpinnings of face pareidolia in patients with Parkinson’s disease (PD) remain unclear. We aimed to clarify face recognition network abnormalities associated with face pareidolia in such patients. Eighty-three patients with PD and 40 healthy controls were recruited in this study. Patients with PD were classified into pareidolia and nonpareidolia groups. Volumetric analyses revealed no significant differences between the pareidolia (n = 39) and nonpareidolia (n = 44) patient groups. We further observed decreased functional connectivity among regions of interest in the bilateral frontotemporal lobes in patients with pareidolia. Seed-based analysis using bilateral temporal fusiform cortices as seeds revealed significantly decreased connectivity with the bilateral inferior medial prefrontal cortices in the pareidolia group. Post hoc regression analysis further demonstrated that the severity of face pareidolia was negatively correlated with functional connectivity between the bilateral temporal fusiform and medial prefrontal cortices. Our findings suggest that top-down modulation of the face recognition network is impaired in patients with PD experiencing face pareidolia.

Rapid and dynamic processing of face pareidolia in the human brain

The human brain is specialized for face processing, yet we sometimes perceive illusory faces in objects. It is unknown whether these natural errors of face detection originate from a rapid process based on visual features or from a slower, cognitive re-interpretation. Here we use a multifaceted approach to understand both the spatial distribution and temporal dynamics of illusory face representation in the brain by combining functional magnetic resonance imaging and magnetoencephalography neuroimaging data with model-based analysis. We find that the representation of illusory faces is confined to occipital-temporal face-selective visual cortex. The temporal dynamics reveal a striking evolution in how illusory faces are represented relative to human faces and matched objects. Illusory faces are initially represented more similarly to real faces than matched objects are, but within ~250 ms, the representation transforms, and they become equivalent to ordinary objects. This is consistent with the initial recruitment of a broadly-tuned face detection mechanism which privileges sensitivity over selectivity.

The Face of Image Reconstruction: Progress, Pitfalls, Prospects

Recent research has demonstrated that neural and behavioral data acquired in response to viewing face images can be used to reconstruct the images themselves. However, the theoretical implications, promises, and challenges of this direction of research remain unclear. We evaluate the potential of this research for elucidating the visual representations underlying face recognition. Specifically, we outline complementary and converging accounts of the visual content, the representational structure, and the neural dynamics of face processing. We illustrate how this research addresses fundamental questions in the study of normal and impaired face recognition, and how image reconstruction provides a powerful framework for uncovering face representations, for unifying multiple types of empirical data, and for facilitating both theoretical and methodological progress.

Data-driven brain network models differentiate variability across language tasks

The relationship between brain structure and function has been probed using a variety of approaches, but how the underlying structural connectivity of the human brain drives behavior is far from understood. To investigate the effect of anatomical brain organization on human task performance, we use a data-driven computational modeling approach and explore the functional effects of naturally occurring structural differences in brain networks. We construct personalized brain network models by combining anatomical connectivity estimated from diffusion spectrum imaging of individual subjects with a nonlinear model of brain dynamics. By performing computational experiments in which we measure the excitability of the global brain network and spread of synchronization following a targeted computational stimulation, we quantify how individual variation in the underlying connectivity impacts both local and global brain dynamics. We further relate the computational results to individual variability in the subjects' performance of three language-demanding tasks both before and after transcranial magnetic stimulation to the left-inferior frontal gyrus. Our results show that task performance correlates with either local or global measures of functional activity, depending on the complexity of the task. By emphasizing differences in the underlying structural connectivity, our model serves as a powerful tool to assess individual differences in task performances, to dissociate the effect of targeted stimulation in tasks that differ in cognitive demand, and to pave the way for the development of personalized therapeutics.

Mental imagery of face enhances face-sensitive event-related potentials to ambiguous visual stimuli

Visual mental imagery forms mental representations of visual objects when correspondent stimuli are absent and shares some characters with visual perception. Both the vertex-positive-potential (VPP) and N170 components of event-related potentials (ERPs) to visual stimuli have a remarkable preference to faces. This study investigated whether visual mental imagery modulates the face-sensitive VPP and/or N170 components. The results showed that with significantly larger amplitudes under the face-imagery condition than the house-imagery condition, the VPP and P2 responses, but not the N170 component, were elicited by phase-randomized ambiguous stimuli. Thus, the brain substrates underlying VPP are not completely identical to those underlying N170, and the VPP/P2 manifestation of the category selectivity in imagery probably reflects an integration of top-down mental imagery signals (from the prefrontal cortex) and bottom-up perception signals (from the early visual cortex) in the occipito-temporal cortex where VPP and P2 originate.

Detection of functional brain network reconfiguration during task-driven cognitive states

Network science offers computational tools to elucidate the complex patterns of interactions evident in neuroimaging data. Recently, these tools have been used to detect dynamic changes in network connectivity that may occur at short time scales. The dynamics of fMRI connectivity, and how they differ across time scales, are far from understood. A simple way to interrogate dynamics at different time scales is to alter the size of the time window used to extract sequential (or rolling) measures of functional connectivity. Here, in n=82 participants performing three distinct cognitive visual tasks in recognition memory and strategic attention, we subdivided regional BOLD time series into variable sized time windows and determined the impact of time window size on observed dynamics. Specifically, we applied a multilayer community detection algorithm to identify temporal communities and we calculated network flexibility to quantify changes in these communities over time. Within our frequency band of interest, large and small windows were associated with a narrow range of network flexibility values across the brain, while medium time windows were associated with a broad range of network flexibility values. Using medium time windows of size 75-100s, we uncovered brain regions with low flexibility (considered core regions, and observed in visual and attention areas) and brain regions with high flexibility (considered periphery regions, and observed in subcortical and temporal lobe regions) via comparison to appropriate dynamic network null models. Generally, this work demonstrates the impact of time window length on observed network dynamics during task performance, offering pragmatic considerations in the choice of time window in dynamic network analysis. More broadly, this work reveals organizational principles of brain functional connectivity that are not accessible with static network approaches.

Feature-based face representations and image reconstruction from behavioral and neural data

The reconstruction of images from neural data can provide a unique window into the content of human perceptual representations. Although recent efforts have established the viability of this enterprise using functional magnetic resonance imaging (MRI) patterns, these efforts have relied on a variety of prespecified image features. Here, we take on the twofold task of deriving features directly from empirical data and of using these features for facial image reconstruction. First, we use a method akin to reverse correlation to derive visual features from functional MRI patterns elicited by a large set of homogeneous face exemplars. Then, we combine these features to reconstruct novel face images from the corresponding neural patterns. This approach allows us to estimate collections of features associated with different cortical areas as well as to successfully match image reconstructions to corresponding face exemplars. Furthermore, we establish the robustness and the utility of this approach by reconstructing images from patterns of behavioral data. From a theoretical perspective, the current results provide key insights into the nature of high-level visual representations, and from a practical perspective, these findings make possible a broad range of image-reconstruction applications via a straightforward methodological approach.

Simultaneously uncovering the patterns of brain regions involved in different story reading subprocesses

Story understanding involves many perceptual and cognitive subprocesses, from perceiving individual words, to parsing sentences, to understanding the relationships among the story characters. We present an integrated computational model of reading that incorporates these and additional subprocesses, simultaneously discovering their fMRI signatures. Our model predicts the fMRI activity associated with reading arbitrary text passages, well enough to distinguish which of two story segments is being read with 74% accuracy. This approach is the first to simultaneously track diverse reading subprocesses during complex story processing and predict the detailed neural representation of diverse story features, ranging from visual word properties to the mention of different story characters and different actions they perform. We construct brain representation maps that replicate many results from a wide range of classical studies that focus each on one aspect of language processing and offer new insights on which type of information is processed by different areas involved in language processing. Additionally, this approach is promising for studying individual differences: it can be used to create single subject maps that may potentially be used to measure reading comprehension and diagnose reading disorders.

Seeing Jesus in toast: neural and behavioral correlates of face pareidolia

Face pareidolia is the illusory perception of non-existent faces. The present study, for the first time, contrasted behavioral and neural responses of face pareidolia with those of letter pareidolia to explore face-specific behavioral and neural responses during illusory face processing. Participants were shown pure-noise images but were led to believe that 50% of them contained either faces or letters; they reported seeing faces or letters illusorily 34% and 38% of the time, respectively. The right fusiform face area (rFFA) showed a specific response when participants "saw" faces as opposed to letters in the pure-noise images. Behavioral responses during face pareidolia produced a classification image (CI) that resembled a face, whereas those during letter pareidolia produced a CI that was letter-like. Further, the extent to which such behavioral CIs resembled faces was directly related to the level of face-specific activations in the rFFA. This finding suggests that the rFFA plays a specific role not only in processing of real faces but also in illusory face perception, perhaps serving to facilitate the interaction between bottom-up information from the primary visual cortex and top-down signals from the prefrontal cortex (PFC). Whole brain analyses revealed a network specialized in face pareidolia, including both the frontal and occipitotemporal regions. Our findings suggest that human face processing has a strong top-down component whereby sensory input with even the slightest suggestion of a face can result in the interpretation of a face.