Object-related activity revealed by functional magnetic resonance imaging in human occipital cortex

Mapping Macaque to Human Cortex with Natural Scene Responses

Neuroscience has long relied on macaque studies to infer human brain function, yet identifying functionally corresponding brain regions across species and measurement modalities remains a fundamental challenge. This is especially true for higher-order cortex, where functional interpretations are constrained by narrow hypotheses and anatomical landmarks are often non-homologous. We present a data-driven approach for mapping functional correspondence across species using rich, naturalistic stimuli. By directly comparing macaque electrophysiology with human fMRI responses to 700 natural scenes, we identify fine-grained alignment based on response pattern similarity, without relying on predefined tuning concepts or hand-picked stimuli. As a test case, we examine the ventral face patch system, a well-studied but contested domain in cross-species alignment. Our approach resolves a longstanding ambiguity, yielding a correspondence consistent with full-brain anatomical warping but inconsistent with prior studies limited by narrow functional hypotheses. These findings show that natural image-evoked response patterns provide a robust foundation for cross-species functional alignment, supporting scalable comparisons as large-scale primate recordings become more widespread.

Object representations drive emotion schemas across a large and diverse set of daily-life scenes

The rapid emotional evaluation of objects and events is essential in daily life. While visual scenes reliably evoke emotions, it remains unclear whether emotion schemas evoked by daily-life scenes depend on object processing systems or are extracted independently. To explore this, we collected emotion ratings for 4913 daily-life scenes from 300 participants, and predicted these ratings from representations in deep neural networks and functional magnetic resonance imaging (fMRI) activity patterns in visual cortex. AlexNet, an object-based model, outperformed EmoNet, an emotion-based model, in predicting emotion ratings for daily-life scenes, while EmoNet excelled for explicitly evocative scenes. Emotion information was processed hierarchically within the object recognition system, consistent with the visual cortex's organization. Activity patterns in the lateral occipital complex (LOC), an object-selective region, reliably predicted emotion ratings and outperformed other visual regions. These findings suggest that the emotional evaluation of daily-life scenes is mediated by visual object processing, with additional mechanisms engaged when object content is uninformative.

Dissociable effects of psilocybin and escitalopram for depression on processing of musical surprises

Psilocybin therapy (PT) is emerging as an effective intervention for Major Depressive Disorder (MDD), offering comparable efficacy to conventional treatments like selective serotonin reuptake inhibitors (SSRIs). Music, an emotionally evocative stimulus, provides a valuable tool to explore changes in hedonic and predictive processing mechanisms via expectancy violations, or 'surprises'. This study sought to compare behavioural and functional magnetic resonance imaging (fMRI) responses to musical surprises in MDD patients treated with either PT or the SSRI, escitalopram. In this secondary analysis of a trial, 41 MDD patients (with usable fMRI data) were randomly assigned to either PT (n = 22) or escitalopram (n = 19) treatment groups. Participants listened to music during fMRI and tracked their emotional experience, both before and after a 6-week intervention. Surprise-related valence and arousal indices were calculated. Musical surprises were entered as regressors for whole-brain and region of interest fMRI analyses. PT caused a greater decrease in anhedonia scores compared with escitalopram. While escitalopram led to reductions in surprise-related affective responses, PT showed no significant change. Escitalopram was associated with increased activation in memory and emotional processing areas during musical surprises (versus control events) when compared with PT. Following PT, there was greater activation in the ventromedial prefrontal cortex and sensory regions, and reduced activation in the angular gyrus. PT may allow for the subjective response to musical surprises to be maintained through a lasting reduction in the salience of prediction errors, or, alternatively, by increasing hedonic priors. Contrastingly, escitalopram may diminish hedonic priors, highlighting fundamental differences in treatment mechanisms.

Understanding episodic memory dynamics: Retrieval and updating mechanisms revealed by fMRI and tDCS

This study investigates brain mechanisms in memory preservation and alteration using a three-phase design: memory encoding (Day 1), interference under fMRI (Day 2), and testing (Day 3). Thirty-one participants completed the core experiment, supplemented by a tDCS study (n = 118) targeting the visual cortex. Original memories were more susceptible to incorrect updating during old-background/new-object interference compared to relearning and no-retrieval conditions. Interference trials elicited heightened activation in the Inferior Parietal Lobe (IPL), Dorsolateral Prefrontal Cortex (DLPFC), and Dorsal Anterior Cingulate Gyrus (dACC) versus no-retrieval controls, and increased frontoparietal and Occipital Fusiform Gyrus (OFG) activation versus relearning. Analyzing interference trials by Day 3 outcomes revealed preserved memories correlated with stronger cingulo-opercular and frontoparietal activation (indicating effective conflict resolution), whereas updated memories showed elevated OFG activity (suggesting new sensory integration). Crucially, IPL/DLPFC activation during interference positively correlated with original memory accuracy, while OFG activation showed negative correlation. tDCS stimulation of the occipital cortex during memory reactivation significantly enhanced memory updating, confirming visual cortex involvement in contextual distortion. Findings demonstrate distinct neural profiles underlie memory outcomes: preserved memories require efficient conflict processing networks, while perceptual interference during reactivation promotes updates through sensory integration systems. This supports targeted neuromodulation approaches for memory modification, highlighting intervention potential through visual cortex engagement during critical memory phases.

Visual mental imagery in typical imagers and in aphantasia: A millimeter-scale 7-T fMRI study

Most of us effortlessly describe visual objects, whether seen or remembered. Yet, around 4% of people report congenital aphantasia: they struggle to visualize objects despite being able to describe their visual appearance. What neural mechanisms create this disparity between subjective experience and objective performance? Aphantasia can provide novel insights into conscious processing and awareness. We used ultra-high field 7T fMRI to establish the neural circuits involved in visual mental imagery and perception, and to elucidate the neural mechanisms associated with the processing of internally generated visual information in the absence of imagery experience in congenital aphantasia. Ten typical imagers and 10 aphantasic individuals performed imagery and perceptual tasks in five domains: object shape, object color, written words, faces, and spatial relationships. In typical imagers, imagery tasks activated left-hemisphere frontoparietal areas, the relevant domain-preferring areas in the ventral temporal cortex partly overlapping with the perceptual domain-preferring areas, and a domain-general area in the left fusiform gyrus (the Fusiform Imagery Node). The results were valid for each individual participant. In aphantasic individuals, imagery activated similar visual areas, but there was reduced functional connectivity between the Fusiform Imagery Node and frontoparietal areas. Our results unveil the domain-general and domain-specific circuits of visual mental imagery, their functional disorganization in aphantasia, and support the general hypothesis that conscious visual experience - whether perceived or imagined - depends on the integrated activity of high-level visual cortex and frontoparietal networks.

Thalamocortical interactions reflecting the intensity of flicker light-induced visual hallucinatory phenomena

Aberrant thalamocortical connectivity occurs together with visual hallucinations in various pathologies and drug-induced states, highlighting the need to better understand how thalamocortical interactions may contribute to hallucinatory phenomena. Flicker light stimulation (FLS) at 10-Hz reliably and selectively induces transient visual hallucinations in healthy participants. Arrhythmic flicker elicits fewer hallucinatory effects while delivering equal amounts of visual stimulation, together facilitating a well-controlled experimental setup to investigate the neural correlates of visual hallucinations driven by flicker rhythmicity. Using rhythmic and arrhythmic FLS during fMRI scanning, we found that rhythmic FLS elicited stronger activation in higher order visual cortices compared with arrhythmic control. Consistently, we found that rhythmic flicker selectively increased connectivity between ventroanterior thalamic nuclei and higher order visual cortices, which was also positively associated with the subjective intensity of visual hallucinatory effects. As these thalamic and cortical areas do not receive primary visual inputs, it suggests that the thalamocortical connectivity changes relate to a higher order function of the thalamus, such as in the coordination of cortical activity. In sum, we present novel evidence for the role of specific thalamocortical interactions with ventroanterior nuclei within visual hallucinatory experiences. Importantly, this can inform future clinical research into the mechanistic underpinnings of pathologic hallucinations.

The effects of flight training on flying cadets' brain structure

In recent years, the impact of professional training on brain structure has sparked extensive research interest. Research into pilots as a high-demand, high-load, and high-cost occupation holds significant academic and economic value. The aim of this study is to investigate the effects of flight training on the brain structure and cognitive functions of flying cadets. The structural magnetic resonance imaging (sMRI) data from 39 flying cadets and 37 general college students underwent analysis using voxel-based morphometry (VBM) and surface-based morphometry (SBM) methods to quantitatively detect and compute multiple indicators, including gray matter volume (GMV), curvature, mean curvature of the white matter surface (MC-WMS), the percentage of surface white matter gray matter (WM-GM percentage), surface Jacobi (S-Jacobi), and Gaussian curvature of white matter surface (GC-WMS). At the voxel level, the GMV in the left temporal pole: middle temporal gyrus region of flying cadets significantly decreased (Gaussian random field, GRF, P < 0.05). At the surface level, there was a significant increase in curvature, MC-WMS, and S-Jacobi in the lateral occipital region of flight cadets (Monte Carlo block level correction, MCBLC, P<0.05), a significant increase in WM-GM percentage in the cuneus region of flight cadets (MCBLC, P<0.05), and a significant increase in GC-WMS in the middle temporal region of flight cadets (MCBLC, P<0.05). In addition, these changes were correlated with behavioral tests. Research suggested that flight training might induce changes in certain brain regions of flying cadets, enabling them to adapt to evolving training content and environments, thereby enhancing their problem-solving and flight abilities. By analyzing multiple indicators at the voxel and surface levels in an integrated manner, it advances our understanding of brain structure, function, and plasticity, while also facilitating a more profound exploration of the neural mechanisms within the pilot's brain.

Quantification of regional CMRO2 in human brain using dynamic 17O-MRI at 3T

OBJECTIVE

To investigate the feasibility of cerebral metabolic rate of oxygen consumption (CMRO2) measurements with MRI at 3 Tesla in different brain regions.

METHODS

CMRO2 represents a key indicator of the physiological state of brain tissue. Dynamic 17O-MRI with inhalation of isotopically enriched 17O gas has been used to quantify global CMRO2 in brain white (WM) and gray matter (GM). However, global CMRO2 can only reflect the overall oxygen metabolism of the brain and cannot provide enough information on local tissue oxygen metabolism. To investigate the feasibility of determination of regional CMRO2 at a clinical 3 T MRI system, CMRO2 values in frontal, parietal and occipital WM and GM were determined in 5 healthy volunteers and compared to evaluate the regional differences of oxygen metabolism in WM and GM. Additionally, regional CMRO2 values were determined in deep brain structures including thalamus, dorsal striatum, caudate nucleus and insula cortex and in the cerebella, and compared with literature values from 15O-PET studies.

RESULTS

In cortical GM the determined CMRO2 values were in good agreement with the literature, whereas values in WM were about 32-48% higher than literature values. Regional analysis revealed a significantly higher CMRO2 in the occipital GM compared to the frontal and parietal GM. By contrast, no significant difference of CMRO2 was observed across the WM. In addition, CMRO2 in deep brain structures was lower compared to literature values and in the cerebella a good hemispheric symmetry of the tissue oxygen metabolism was found.

CONCLUSION

Dynamic 17O-MRI enables direct, non-invasive determination of regional CMRO2 in brain structures in healthy volunteers at 3T.

The negative relationship between brain-age gap and psychological resilience defines the age-related neurocognitive status in older people

Biological brain age is a brain-predicted age using machine learning to indicate brain health and its associated conditions. The presence of an older predicted brain age relative to the actual chronological age is indicative of accelerated aging processes. Consequently, the disparity between the brain's chronological age and its predicted age (brain-age gap) and the factors influencing this disparity provide critical insights into cerebral health dynamics during aging. In this study, we employed a Lasso regression model and analyzed multimodal imaging data from 124 participants aged 53 to 76 to formulate and predict brain age. Additionally, we conducted partial correlation analyses to explore the complex relationship between the brain-age gap and network metrics, cognitive assessments, and emotional evaluations, while controlling for chronological age, gender, and education. Our findings highlight psychological resilience as a significant mitigating factor against premature brain aging. It is established that psychological resilience significantly influences the modulation of the brain-age gap. Moreover, psychological resilience and the brain-age gap exhibit a high accuracy (above 0.72) in segregating Montreal Cognitive Assessment score-based cohorts. This observation underscores significant insight into the potential of utilizing the brain-age gap as a diagnostic tool for the early detection of accelerated aging. It advocates for the timely application of interventions, including the development of programs aimed at bolstering psychological resilience.

A double dissociation between semantic and spatial cognition in visual to default network pathways

Processing pathways between sensory and default mode network (DMN) regions support recognition, navigation, and memory but their organisation is not well understood. We show that functional subdivisions of visual cortex and DMN sit at opposing ends of parallel streams of information processing that support visually mediated semantic and spatial cognition, providing convergent evidence from univariate and multivariate task responses, intrinsic functional and structural connectivity. Participants learned virtual environments consisting of buildings populated with objects, drawn from either a single semantic category or multiple categories. Later, they made semantic and spatial context decisions about these objects and buildings during functional magnetic resonance imaging. A lateral ventral occipital to fronto-temporal DMN pathway was primarily engaged by semantic judgements, while a medial visual to medial temporal DMN pathway supported spatial context judgements. These pathways had distinctive locations in functional connectivity space: the semantic pathway was both further from unimodal systems and more balanced between visual and auditory-motor regions compared with the spatial pathway. When semantic and spatial context information could be integrated (in buildings containing objects from a single category), regions at the intersection of these pathways responded, suggesting that parallel processing streams interact at multiple levels of the cortical hierarchy to produce coherent memory-guided cognition.

The brain prioritizes the basic level of object category abstraction

The same object can be described at multiple levels of abstraction ("parka", "coat", "clothing"), yet human observers consistently name objects at a mid-level of specificity known as the basic level. Little is known about the temporal dynamics involved in retrieving neural representations that prioritize the basic level, nor how these dynamics change with evolving task demands. In this study, observers viewed 1080 objects arranged in a three-tier category taxonomy while 64-channel EEG was recorded. Observers performed a categorical one-back task in different recording sessions on the basic or subordinate levels. We used time-resolved multiple regression to assess the utility of superordinate-, basic-, and subordinate-level categories across the scalp. We found robust use of basic-level category information starting at about 50 ms after stimulus onset and moving from posterior electrodes (149 ms) through lateral (261 ms) to anterior sites (332 ms). Task differences were not evident in the first 200 ms of processing but were observed between 200-300 ms after stimulus presentation. Together, this work demonstrates that the object category representations prioritize the basic level and do so relatively early, congruent with results that show that basic-level categorization is an automatic and obligatory process.

Seeing and visualizing across the hemispheres

Despite our subjective experience of a largely symmetric visual world, the human brain exhibits varying patterns and degrees of hemispheric asymmetry in distinct processes of visual cognition. This chapter reviews behavioral and neuroimaging evidence from neurotypical individuals and neurological patients, concerning functional asymmetries between the right hemisphere (RH) and the left hemisphere (LH) in visual object processing and mental imagery. Hierarchical perception shows RH preference for global processing and LH preference for local processing. At later stages of visual object processing, RH-based circuits exhibit a relative advantage in terms of perceptual integration, with a subsequent shift toward LH-based circuits for processing at higher conceptual and semantic levels. In voluntary visual mental imagery, circuits in the LH ventral temporal cortex play a pivotal role in transitioning from object meaning to simulated visualization. These hemispheric asymmetries in visual object processing might, in part, be influenced by the overall need to minimize wiring, coupled with the presence of distinct specialized networks within each hemisphere, such as the RH attention networks and the LH language networks. From a broader viewpoint, the evidence examined in this chapter indicates that visual object processing involves the interactions of large-scale cortical circuits within and between the hemispheres.

Attention modulates human visual responses to objects by tuning sharpening

Visual stimuli compete with each other for cortical processing and attention biases this competition in favor of the attended stimulus. How does the relationship between the stimuli affect the strength of this attentional bias? Here, we used functional MRI to explore the effect of target-distractor similarity in neural representation on attentional modulation in the human visual cortex using univariate and multivariate pattern analyses. Using stimuli from four object categories (human bodies, cats, cars, and houses), we investigated attentional effects in the primary visual area V1, the object-selective regions LO and pFs, the body-selective region EBA, and the scene-selective region PPA. We demonstrated that the strength of the attentional bias toward the target is not fixed but decreases with increasing target-distractor similarity. Simulations provided evidence that this result pattern is explained by tuning sharpening rather than an increase in gain. Our findings provide a mechanistic explanation for the behavioral effects of target-distractor similarity on attentional biases and suggest tuning sharpening as the underlying mechanism in object-based attention.

Cross-sectional and longitudinal changes in category-selectivity in visual cortex following pediatric cortical resection

The topographic organization of category-selective responses in human ventral occipitotemporal cortex (VOTC) and its relationship to regions subserving language functions is remarkably uniform across individuals. This arrangement is thought to result from the clustering of neurons responding to similar inputs, constrained by intrinsic architecture and tuned by experience. We examined the malleability of this organization in individuals with unilateral resection of VOTC during childhood for the management of drug-resistant epilepsy. In cross-sectional and longitudinal functional imaging studies, we compared the topography and neural representations of 17 category-selective regions in individuals with a VOTC resection, a 'control patient' with resection outside VOTC, and typically developing matched controls. We demonstrated both adherence to and deviation from the standard topography and uncovered fine-grained competitive dynamics between word- and face-selectivity over time in the single, preserved VOTC. The findings elucidate the nature and extent of cortical plasticity and highlight the potential for remodeling of extrastriate architecture and function.

cTBS over ventral cortex enhances depth perception

Stereoscopic capacities vary widely across the normal population. It has become increasingly apparent, however, that mechanisms underlying stereoscopic depth perception retain a considerable degree of plasticity through adulthood. Here, we contrast the capacity for neurostimulation in the form of continuous theta-burst stimulation (cTBS) over strategically-chosen sites in the visual cortex to bring about improvements in stereoscopic depth perception. cTBS was delivered to occipital cortex (V1/V2), lateral occipital complex (LOC), along with a control site (Cz). We measured performance on depth and luminance discrimination tasks before and after stimulation. We found a significant improvement in depth (but not luminance) discrimination performance following cTBS over LOC. By contrast, cTBS over occipital cortex and Cz did not affect performance on either task. These findings suggest that ventral (lateral-occipital) cortex is a key node for governing plasticity of stereoscopic vision in visually normal human observers. We speculate that cTBS exerts inhibitory influences that may suppress internal noise within the nervous system, leading to an improved read-out of depth features.

Moving and Static Faces, Bodies, Objects, and Scenes Are Differentially Represented across the Three Visual Pathways

Models of human cortex propose the existence of neuroanatomical pathways specialized for different behavioral functions. These pathways include a ventral pathway for object recognition, a dorsal pathway for performing visually guided physical actions, and a recently proposed third pathway for social perception. In the current study, we tested the hypothesis that different categories of moving stimuli are differentially processed across the dorsal and third pathways according to their behavioral implications. Human participants (n = 30) were scanned with fMRI while viewing moving and static stimuli from four categories (faces, bodies, scenes, and objects). A whole-brain group analysis showed that moving bodies and moving objects increased neural responses in the bilateral posterior parietal cortex, parts of the dorsal pathway. By contrast, moving faces and moving bodies increased neural responses, the superior temporal sulcus, part of the third pathway. This pattern of results was also supported by a separate ROI analysis showing that moving stimuli produced more robust neural responses for all visual object categories, particularly in lateral and dorsal brain areas. Our results suggest that dynamic naturalistic stimuli from different categories are routed in specific visual pathways that process dissociable behavioral functions.

A data-driven analysis of the perceptual and neural responses to natural objects reveals organising principles of human visual cognition

A key challenge in understanding the functional organisation of visual cortex stems from the fact that only a small proportion of the objects experienced during natural viewing can be presented in a typical experiment. This constraint often leads to experimental designs that compare responses to objects from experimenter-defined stimulus conditions, potentially limiting the interpretation of the data. To overcome this issue, we used images from the THINGS initiative, which provides a systematic sampling of natural objects. A data-driven analysis was then applied to reveal the functional organisation of the visual brain, incorporating both perceptual and neural responses to these objects. Perceptual properties of the objects were taken from an analysis of similarity judgements, and neural properties were taken from whole brain fMRI responses to the same objects. Partial least squares regression (PLSR) was then used to predict neural responses across the brain from the perceptual properties while simultaneously applying dimensionality reduction. The PLSR model accurately predicted neural responses across visual cortex using only a small number of components. These components revealed smooth, graded neural topographies, which were similar in both hemispheres, and captured a variety of object properties including animacy, real-world size, and object category. However, they did not accord in any simple way with previous theoretical perspectives on object perception. Instead, our findings suggest that visual cortex encodes information in a statistically efficient manner, reflecting natural variability among objects.Significance statement The ability to recognise objects is fundamental to how we interact with our environment, yet the organising principles underlying neural representations of visual objects remain contentious. In this study, we sought to address this question by analysing perceptual and neural responses to a large, unbiased sample of objects. Using a data-driven approach, we leveraged perceptual properties of objects to predict neural responses using a small number of components. This model predicted neural responses with a high degree of accuracy across visual cortex. The components did not directly align with previous explanations of object perception. Instead, our findings suggest the organisation of the visual brain is based on the statistical properties of objects in the natural world.

Using imagination and the contents of memory to create new scene and object representations: A functional MRI study

Humans can use the contents of memory to construct scenarios and events that they have not encountered before, a process colloquially known as imagination. Much of our current understanding of the neural mechanisms mediating imagination is limited by paradigms that rely on participants' subjective reports of imagined content. Here, we used a novel behavioral paradigm that was designed to systematically evaluate the contents of an individual's imagination. Participants first learned the layout of four distinct rooms containing five wall segments with differing geometrical characteristics, each associated with a unique object. During functional MRI, participants were then shown two different wall segments or objects on each trial and asked to first, retrieve the associated objects or walls, respectively (retrieval phase) and then second, imagine the two objects side-by-side or combine the two wall segments (imagination phase). Importantly, the contents of each participant's imagination were interrogated by having them make a same/different judgment about the properties of the imagined objects or scenes. Using univariate and multivariate analyses, we observed widespread activity across occipito-temporal cortex for the retrieval of objects and for the imaginative creation of scenes. Interestingly, a classifier, whether trained on the imagination or retrieval data, was able to successfully differentiate the neural patterns associated with the imagination of scenes from that of objects. Our results reveal neural differences in the cued retrieval of object and scene memoranda, demonstrate that different representations underlie the creation and/or imagination of scene and object content, and highlight a novel behavioral paradigm that can be used to systematically evaluate the contents of an individual's imagination.

Explorations of using a convolutional neural network to understand brain activations during movie watching

Background

Naturalistic stimuli, such as videos, can elicit complex brain activations. However, the intricate nature of these stimuli makes it challenging to attribute specific brain functions to the resulting activations, particularly for higher-level processes such as social interactions.

Objective

We hypothesized that activations in different layers of a convolutional neural network (VGG-16) would correspond to varying levels of brain activation, reflecting the brain's visual processing hierarchy. Additionally, we aimed to explore which brain regions would be linked to the deeper layers of the network.

Methods

This study analyzed functional MRI data from participants watching a cartoon video. Using a pre-trained VGG-16 convolutional neural network, we mapped hierarchical features of the video to different levels of brain activation. Activation maps from various kernels and layers were extracted from video frames, and the time series of average activation patterns for each kernel were used in a voxel-wise model to examine brain responses.

Results

Lower layers of the network were primarily associated with activations in lower visual regions, although some kernels also unexpectedly showed associations with the posterior cingulate cortex. Deeper layers were linked to more anterior and lateral regions of the visual cortex, as well as the supramarginal gyrus.

Conclusions

This analysis demonstrated both the potential and limitations of using convolutional neural networks to connect video content with brain functions, providing valuable insights into how different brain regions respond to varying levels of visual processing.

Classification of Internal and External Distractions in an Educational VR Environment Using Multimodal Features

Virtual reality (VR) can potentially enhance student engagement and memory retention in the classroom. However, distraction among participants in a VR-based classroom is a significant concern. Several factors, including mind wandering, external noise, stress, etc., can cause students to become internally and/or externally distracted while learning. To detect distractions, single or multi-modal features can be used. A single modality is found to be insufficient to detect both internal and external distractions, mainly because of individual variability. In this work, we investigated multi-modal features: eye tracking and EEG data, to classify the internal and external distractions in an educational VR environment. We set up our educational VR environment and equipped it for multi-modal data collection. We implemented different machine learning (ML) methods, including k-nearest-neighbors (kNN), Random Forest (RF), one-dimensional convolutional neural network - long short-term memory (1 D-CNN-LSTM), and two-dimensional convolutional neural networks (2D-CNN) to classify participants' internal and external distraction states using the multi-modal features. We performed cross-subject, cross-session, and gender-based grouping tests to evaluate our models. We found that the RF classifier achieves the highest accuracy over 83% in the cross-subject test, around 68% to 78% in the cross-session test, and around 90% in the gender-based grouping test compared to other models. SHAP analysis of the extracted features illustrated greater contributions from the occipital and prefrontal regions of the brain, as well as gaze angle, gaze origin, and head rotation features from the eye tracking data.

Human Visual Pathways for Action Recognition versus Deep Convolutional Neural Networks: Representation Correspondence in Late but Not Early Layers

Deep convolutional neural networks (DCNNs) have attained human-level performance for object categorization and exhibited representation alignment between network layers and brain regions. Does such representation alignment naturally extend to other visual tasks beyond recognizing objects in static images? In this study, we expanded the exploration to the recognition of human actions from videos and assessed the representation capabilities and alignment of two-stream DCNNs in comparison with brain regions situated along ventral and dorsal pathways. Using decoding analysis and representational similarity analysis, we show that DCNN models do not show hierarchical representation alignment to human brain across visual regions when processing action videos. Instead, later layers of DCNN models demonstrate greater representation similarities to the human visual cortex. These findings were revealed for two display formats: photorealistic avatars with full-body information and simplified stimuli in the point-light display. The discrepancies in representation alignment suggest fundamental differences in how DCNNs and the human brain represent dynamic visual information related to actions.

The human posterior parietal cortices orthogonalize the representation of different streams of information concurrently coded in visual working memory

The key to adaptive visual processing lies in the ability to maintain goal-directed visual representation in the face of distraction. In visual working memory (VWM), distraction may come from the coding of distractors or other concurrently retained targets. This fMRI study reveals a common representational geometry that our brain uses to combat both types of distractions in VWM. Specifically, using fMRI pattern decoding, the human posterior parietal cortex is shown to orthogonalize the representations of different streams of information concurrently coded in VWM, whether they are targets and distractors, or different targets concurrently held in VWM. The latter is also seen in the human occipitotemporal cortex. Such a representational geometry provides an elegant and simple solution to enable independent information readout, effectively combating distraction from the different streams of information, while accommodating their concurrent representations. This representational scheme differs from mechanisms that actively suppress or block the encoding of distractors to reduce interference. It is likely a general neural representational principle that supports our ability to represent information beyond VWM in other situations where multiple streams of visual information are tracked and processed simultaneously.

Attention Modulates Human Visual Responses to Objects by Tuning Sharpening

Visual stimuli compete with each other for cortical processing and attention biases this competition in favor of the attended stimulus. How does the relationship between the stimuli affect the strength of this attentional bias? Here, we used functional MRI to explore the effect of target-distractor similarity in neural representation on attentional modulation in the human visual cortex using univariate and multivariate pattern analyses. Using stimuli from four object categories (human bodies, cats, cars and houses), we investigated attentional effects in the primary visual area V1, the object-selective regions LO and pFs, the body-selective region EBA, and the scene-selective region PPA. We demonstrated that the strength of the attentional bias towards the target is not fixed but decreases with increasing target-distractor similarity. Simulations provided evidence that this result pattern is explained by tuning sharpening rather than an increase in gain. Our findings provide a mechanistic explanation for behavioral effects of target-distractor similarity on attentional biases and suggest tuning sharpening as the underlying mechanism in object-based attention.

HD-tDCS to the lateral occipital complex improves haptic object recognition

High-definition transcranial direct current stimulation (HD-tDCS) is a non-invasive brain stimulation technique that has been shown to be safe and effective in modulating neuronal activity. The present study investigates the effect of anodal HD-tDCS on haptic object perception and memory through stimulation of the lateral occipital complex (LOC), a structure that has been shown to be involved in both visual and haptic object recognition. In this single-blind, sham-controlled, between-subjects study, blindfolded healthy, sighted participants used their right (dominant) hand to perform haptic discrimination and recognition tasks with 3D-printed, novel objects called "Greebles" while receiving 20 min of 2 milliamp (mA) anodal stimulation (or sham) to the left or right LOC. Compared to sham, those who received left LOC stimulation (contralateral to the hand used) showed an improvement in haptic object recognition but not discrimination-a finding that was evident from the start of the behavioral tasks. A second experiment showed that this effect was not observed with right LOC stimulation (ipsilateral to the hand used). These results suggest that HD-tDCS to the left LOC can improve recognition of objects perceived via touch. Overall, this work sheds light on the LOC as a multimodal structure that plays a key role in object recognition in both the visual and haptic modalities.

Object-based attention requires monocular visual pathways

Mechanisms of object-based attention (OBA) are commonly associated with the cerebral cortex. However, less is known about the involvement of subcortical visual pathways in these processes. Knowledge of the neural mechanisms subserving OBA can provide insight into the evolutionary trajectory of attentional selection. In the current study, the classic double-rectangle cueing task was implemented using a stereoscope in order to differentiate between the involvement of lower (monocular) and higher (binocular) visual pathways in OBA processes. We found that monocular visual pathways are involved in two main aspects of OBA: exogenous orienting towards a cued object (Experiment 1; N =33) and attentional deployment within a cued object (Experiment 2; N =23); this is evident by the presence of OBA only when both the cue and target were presented to the same eye. Thus, these results indicate that monocular (mostly subcortical) visual regions are not simply passing information to higher cortical areas but have a functional computational role in OBA. These findings emphasize the importance of lower regions in attentional processes and, more specifically, in OBA.

Unsupervised learning as a computational principle works in visual learning of natural scenes, but not of artificial stimuli

The question of whether we learn exposed visual features remains a subject of controversy. A prevalent computational model suggests that visual features frequently exposed to observers in natural environments are likely to be learned. However, this unsupervised learning model appears to be contradicted by the significant body of experimental results with human participants that indicates visual perceptual learning (VPL) of visible task-irrelevant features does not occur with frequent exposure. Here, we demonstrate a resolution to this controversy with a new finding: Exposure to a dominant global orientation as task-irrelevant leads to VPL of the orientation, particularly when the orientation is derived from natural scene images, whereas VPL did not occur with artificial images even with matched distributions of local orientations and spatial frequencies to natural scene images. Further investigation revealed that this disparity arises from the presence of higher-order statistics derived from natural scene images-global structures such as correlations between different local orientation and spatial frequency channels. Moreover, behavioral and neuroimaging results indicate that the dominant orientation from these higher-order statistics undergoes less attentional suppression than that from artificial images, which may facilitate VPL. Our results contribute to resolving the controversy by affirming the validity of unsupervised learning models for natural scenes but not for artificial stimuli. They challenge the assumption that VPL occurring in everyday life can be predicted by laws governing VPL for conventionally used artificial stimuli.

Occipital-temporal cortical tuning to semantic and affective features of natural images predicts associated behavioral responses

In everyday life, people need to respond appropriately to many types of emotional stimuli. Here, we investigate whether human occipital-temporal cortex (OTC) shows co-representation of the semantic category and affective content of visual stimuli. We also explore whether OTC transformation of semantic and affective features extracts information of value for guiding behavior. Participants viewed 1620 emotional natural images while functional magnetic resonance imaging data were acquired. Using voxel-wise modeling we show widespread tuning to semantic and affective image features across OTC. The top three principal components underlying OTC voxel-wise responses to image features encoded stimulus animacy, stimulus arousal and interactions of animacy with stimulus valence and arousal. At low to moderate dimensionality, OTC tuning patterns predicted behavioral responses linked to each image better than regressors directly based on image features. This is consistent with OTC representing stimulus semantic category and affective content in a manner suited to guiding behavior.

Contribution of the lateral occipital and parahippocampal cortices to pattern separation of objects and contexts

Contextual features are integral to episodic memories; yet, we know little about context effects on pattern separation, a hippocampal function promoting orthogonalization of overlapping memory representations. Recent studies suggested that various extrahippocampal brain regions support pattern separation; however, the specific role of the parahippocampal cortex-a region involved in context representation-in pattern separation has not yet been studied. Here, we investigated the contribution of the parahippocampal cortex (specifically, the parahippocampal place area) to context reinstatement effects on mnemonic discrimination, using functional magnetic resonance imaging. During scanning, participants saw object images on unique context scenes, followed by a recognition task involving the repetitions of encoded objects or visually similar lures on either their original context or a lure context. Context reinstatement at retrieval improved item recognition but hindered mnemonic discrimination. Crucially, our region of interest analyses of the parahippocampal place area and an object-selective visual area, the lateral occipital cortex indicated that while during successful mnemonic decisions parahippocampal place area activity decreased for old contexts compared to lure contexts irrespective of object novelty, lateral occipital cortex activity differentiated between old and lure objects exclusively. These results imply that pattern separation of contextual and item-specific memory features may be differentially aided by scene and object-selective cortical areas.

Human genetics of face recognition: discovery of MCTP2 mutations in humans with face blindness (congenital prosopagnosia)

Face recognition is important for both visual and social cognition. While prosopagnosia or face blindness has been known for seven decades and face-specific neurons for half a century, the molecular genetic mechanism is not clear. Here we report results after 17 years of research with classic genetics and modern genomics. From a large family with 18 congenital prosopagnosia (CP) members with obvious difficulties in face recognition in daily life, we uncovered a fully cosegregating private mutation in the MCTP2 gene which encodes a calcium binding transmembrane protein expressed in the brain. After screening through cohorts of 6589, we found more CPs and their families, allowing detection of more CP associated mutations in MCTP2. Face recognition differences were detected between 14 carriers with the frameshift mutation S80fs in MCTP2 and 19 noncarrying volunteers. Six families including one with 10 members showed the S80fs-CP correlation. Functional magnetic resonance imaging found association of impaired recognition of individual faces by MCTP2 mutant CPs with reduced repetition suppression to repeated facial identities in the right fusiform face area. Our results have revealed genetic predisposition of MCTP2 mutations in CP, 76 years after the initial report of prosopagnosia and 47 years after the report of the first CP. This is the first time a gene required for a higher form of visual social cognition was found in humans.

Novelty preference assessed by eye tracking: A sensitive measure of impaired recognition memory in epilepsy

OBJECTIVE

Epilepsy patients often report memory deficits despite normal objective testing, suggesting that available measures are insensitive or that non-mnemonic factors are involved. The Visual Paired Comparison Task (VPCT) assesses novelty preference, the tendency to fixate on novel images rather than previously viewed items, requiring recognition memory for the "old" images. As novelty preference is a sensitive measure of hippocampal-dependent memory function, we predicted impaired VPCT performance in epilepsy patients compared to healthy controls.

METHODS

We assessed 26 healthy adult controls and 31 epilepsy patients (16 focal-onset, 13 generalized-onset, 2 unknown-onset) with the VPCT using delays of 2 or 30 s between encoding and recognition. Fifteen healthy controls and 17 epilepsy patients (10 focal-onset, 5 generalized-onset, 2 unknown-onset) completed the task at 2-, 5-, and 30-minute delays. Subjects also performed standard memory measures, including the Medical College of Georgia (MCG) Paragraph Test, California Verbal Learning Test-Second Edition (CVLT-II), and Brief Visual Memory Test-Revised (BVMT-R).

RESULTS

The epilepsy group was high functioning, with greater estimated IQ (p = 0.041), greater years of education (p = 0.034), and higher BVMT-R scores (p = 0.024) compared to controls. Both the control group and epilepsy cohort, as well as focal- and generalized-onset subgroups, had intact novelty preference at the 2- and 30-second delays (p-values ≤ 0.001) and declined at 30 min (p-values > 0.05). Only the epilepsy patients had early declines at 2- and 5-minute delays (controls with intact novelty preference at p = 0.003 and p ≤ 0.001, respectively; epilepsy groups' p-values > 0.05).

CONCLUSIONS

Memory for the "old" items decayed more rapidly in overall, focal-onset, and generalized-onset epilepsy groups. The VPCT detected deficits while standard memory measures were largely intact, suggesting that the VPCT may be a more sensitive measure of temporal lobe memory function than standard neuropsychological batteries.

Childhood adversity modulates structural brain changes in borderline personality but not in major depression disorder

Adverse childhood experiences (ACEs) negatively affect the function and structure of emotion brain circuits, increasing the risk of various psychiatric disorders. It is unclear if ACEs show disorder specificity with respect to their effects on brain structure. We aimed to investigate whether the structural brain effects of ACEs differ between patients with major depression (MDD) and borderline personality disorder (BPD). These disorders share many symptoms but likely have different etiologies. To achieve our goal, we obtained structural 3T-MRI images from 20 healthy controls (HC), 19 MDD patients, and 18 BPD patients, and measured cortical thickness and subcortical gray matter volumes. We utilized the Adverse Childhood Experiences (ACE) questionnaire to quantify self-reported exposure to childhood trauma. Our findings suggest that individuals with MDD exhibit a smaller cortical thickness when compared to those with BPD. However, ACEs showed a significantly affected relationship with cortical thickness in BPD but not in MDD. ACEs were found to be associated with thinning in cortical regions involved in emotional behavior in BPD, whereas HC showed an opposite association. Our results suggest a potential mechanism of ACE effects on psychopathology involving changes in brain structure. These findings highlight the importance of early detection and intervention strategies.

Long-Term Bridge Training Induces Functional Plasticity Changes in the Brain of Early-Adult Individuals

The aim of this study was to investigate the impact of extended bridge expertise on rapid perceptual processing and brain functional plasticity in early adulthood, utilizing functional magnetic resonance imaging (fMRI). In this investigation, we compared 6 high-level college bridge players with 25 college students lacking bridge experience, assessing their intelligence and working memory. Additionally, we scrutinized behavioral performance and whole-brain activation patterns during an image perceptual judgment task. Findings indicated significant group and interaction effects at the behavioral level. Bridge players exhibited prolonged reaction times and enhanced accuracy on card tasks. At the neural level, the activation level of bridge players in the occipital lobe exceeded that of ordinary college students, with more pronounced group effects in the motor area and inferior parietal lobule during card tasks. This implies that bridge expertise in early adulthood induces functional plasticity changes in regions associated with visual processing and automated mathematical computation.

Neuronal tuning and population representations of shape and category in human visual cortex

Object recognition and categorization are essential cognitive processes which engage considerable neural resources in the human ventral visual stream. However, the tuning properties of human ventral stream neurons for object shape and category are virtually unknown. We performed large-scale recordings of spiking activity in human Lateral Occipital Complex in response to stimuli in which the shape dimension was dissociated from the category dimension. Consistent with studies in nonhuman primates, the neuronal representations were primarily shape-based, although we also observed category-like encoding for images of animals. Surprisingly, linear decoders could reliably classify stimulus category even in data sets that were entirely shape-based. In addition, many recording sites showed an interaction between shape and category tuning. These results represent a detailed study on shape and category coding at the neuronal level in the human ventral visual stream, furnishing essential evidence that reconciles human imaging and macaque single-cell studies.

Modulation of the neuronal response in human primary visual cortex by re-entrant projections during retinal input processing as manifest in the visual evoked potential

Initial deflections in the visual evoked potential (VEP) reflect the neuronal process of extracting features from the retinal input; a process not modulated by re-entrant projections. Later deflections in the VEP reflect the neuronal process of combining features into an object, a process referred to as 'object closure' and modulated by re-entrant projections. Our earlier work indicated that the VEP reflects independent neuronal responses processing temporal - and spatial luminance contrast and that these responses arise from an interaction between forward and re-entrant input. In this earlier work, changing the temporal luminance contrast property of a stimulus altered its spatial luminance contrast property. We recorded the VEP in 12 volunteers viewing image pairs of a windmill, regular dartboard or an RMS dartboard rotated by either Π/4, Π/2, 3Π/4 or Π radians with respect to each other. The windmill and regular dartboard had identical white to black ratio, while the two dartboards identical contrast edges per unit area. Rotation varied temporal luminance contrast of a stimulus without affecting its spatial luminance contrast. N75, P100, N135 and P240 amplitude and latency were compared and a source localisation and temporal frequency analysis performed. P100 amplitude signals a neuronal response processing temporal luminance contrast that is modulated by re-entrant projections with fast axonal conduction velocities. N135 and P240 signal the neuronal response processing spatial luminance contrast and is modulated by re-entrant projections with slow axonal conduction velocities. The dorsal stream is interconnected by fast axonal conduction velocities, the ventral stream by slow axonal conduction velocities.

Investigating the neural effects of typicality and predictability for face and object stimuli

The brain calibrates itself based on the past stimulus diet, which makes frequently observed stimuli appear as typical (as opposed to uncommon stimuli, which appear as distinctive). Based on predictive processing theory, the brain should be more "prepared" for typical exemplars, because these contain information that has been encountered frequently, allowing it to economically represent items of that category. Thus, one could ask whether predictability and typicality of visual stimuli interact, or rather act in an additive manner. We adapted the design by Egner and colleagues (2010), who used cues to induce expectations about stimulus category (face vs. chair) occurrence during an orthogonal inversion detection task. We measured BOLD responses with fMRI in 35 participants. First, distinctive stimuli always elicited stronger responses than typical ones in all ROIs, and our whole-brain directional contrasts for the effects of typicality and distinctiveness converge with previous findings. Second and importantly, we could not replicate the interaction between category and predictability reported by Egner et al. (2010), which casts doubt on whether cueing designs are ideal to elicit reliable predictability effects. Third, likely as a consequence of the lack of predictability effects, we found no interaction between predictability and typicality in any of the four tested regions (bilateral fusiform face areas, lateral occipital complexes) when considering both categories, nor in the whole brain. We discuss the issue of replicability in neuroscience and sketch an agenda for how future studies might address the same question.

A Unifying Model for Discordant and Concordant Results in Human Neuroimaging Studies of Facial Viewpoint Selectivity

Recognizing faces regardless of their viewpoint is critical for social interactions. Traditional theories hold that view-selective early visual representations gradually become tolerant to viewpoint changes along the ventral visual hierarchy. Newer theories, based on single-neuron monkey electrophysiological recordings, suggest a three-stage architecture including an intermediate face-selective patch abruptly achieving invariance to mirror-symmetric face views. Human studies combining neuroimaging and multivariate pattern analysis (MVPA) have provided convergent evidence of view selectivity in early visual areas. However, contradictory conclusions have been reached concerning the existence in humans of a mirror-symmetric representation like that observed in macaques. We believe these contradictions arise from low-level stimulus confounds and data analysis choices. To probe for low-level confounds, we analyzed images from two face databases. Analyses of image luminance and contrast revealed biases across face views described by even polynomials-i.e., mirror-symmetric. To explain major trends across neuroimaging studies, we constructed a network model incorporating three constraints: cortical magnification, convergent feedforward projections, and interhemispheric connections. Given the identified low-level biases, we show that a gradual increase of interhemispheric connections across network-layers is sufficient to replicate view-tuning in early processing stages and mirror-symmetry in later stages. Data analysis decisions-pattern dissimilarity measure and data recentering-accounted for the inconsistent observation of mirror-symmetry across prior studies. Pattern analyses of human fMRI data (of either sex) revealed biases compatible with our model. The model provides a unifying explanation of MVPA studies of viewpoint selectivity and suggests observations of mirror-symmetry originate from ineffectively normalized signal imbalances across different face views.

Experience transforms crossmodal object representations in the anterior temporal lobes

Combining information from multiple senses is essential to object recognition, core to the ability to learn concepts, make new inferences, and generalize across distinct entities. Yet how the mind combines sensory input into coherent crossmodal representations - the crossmodal binding problem - remains poorly understood. Here, we applied multi-echo fMRI across a 4-day paradigm, in which participants learned three-dimensional crossmodal representations created from well-characterized unimodal visual shape and sound features. Our novel paradigm decoupled the learned crossmodal object representations from their baseline unimodal shapes and sounds, thus allowing us to track the emergence of crossmodal object representations as they were learned by healthy adults. Critically, we found that two anterior temporal lobe structures - temporal pole and perirhinal cortex - differentiated learned from non-learned crossmodal objects, even when controlling for the unimodal features that composed those objects. These results provide evidence for integrated crossmodal object representations in the anterior temporal lobes that were different from the representations for the unimodal features. Furthermore, we found that perirhinal cortex representations were by default biased toward visual shape, but this initial visual bias was attenuated by crossmodal learning. Thus, crossmodal learning transformed perirhinal representations such that they were no longer predominantly grounded in the visual modality, which may be a mechanism by which object concepts gain their abstraction.

Body size as a metric for the affordable world

The physical body of an organism serves as a vital interface for interactions with its environment. Here, we investigated the impact of human body size on the perception of action possibilities (affordances) offered by the environment. We found that the body size delineated a distinct boundary on affordances, dividing objects of continuous real-world sizes into two discrete categories with each affording distinct action sets. Additionally, the boundary shifted with imagined body sizes, suggesting a causal link between body size and affordance perception. Intriguingly, ChatGPT, a large language model lacking physical embodiment, exhibited a modest yet comparable affordance boundary at the scale of human body size, suggesting the boundary is not exclusively derived from organism-environment interactions. A subsequent fMRI experiment offered preliminary evidence of affordance processing exclusively for objects within the body size range, but not for those beyond. This suggests that only objects capable of being manipulated are the objects capable of offering affordance in the eyes of an organism. In summary, our study suggests a novel definition of object-ness in an affordance-based context, advocating the concept of embodied cognition in understanding the emergence of intelligence constrained by an organism's physical attributes.

Preliminary Evidence for Global Properties in Human Listeners During Natural Auditory Scene Perception

Theories of auditory and visual scene analysis suggest the perception of scenes relies on the identification and segregation of objects within it, resembling a detail-oriented processing style. However, a more global process may occur while analyzing scenes, which has been evidenced in the visual domain. It is our understanding that a similar line of research has not been explored in the auditory domain; therefore, we evaluated the contributions of high-level global and low-level acoustic information to auditory scene perception. An additional aim was to increase the field's ecological validity by using and making available a new collection of high-quality auditory scenes. Participants rated scenes on 8 global properties (e.g., open vs. enclosed) and an acoustic analysis evaluated which low-level features predicted the ratings. We submitted the acoustic measures and average ratings of the global properties to separate exploratory factor analyses (EFAs). The EFA of the acoustic measures revealed a seven-factor structure explaining 57% of the variance in the data, while the EFA of the global property measures revealed a two-factor structure explaining 64% of the variance in the data. Regression analyses revealed each global property was predicted by at least one acoustic variable (R2 = 0.33-0.87). These findings were extended using deep neural network models where we examined correlations between human ratings of global properties and deep embeddings of two computational models: an object-based model and a scene-based model. The results support that participants' ratings are more strongly explained by a global analysis of the scene setting, though the relationship between scene perception and auditory perception is multifaceted, with differing correlation patterns evident between the two models. Taken together, our results provide evidence for the ability to perceive auditory scenes from a global perspective. Some of the acoustic measures predicted ratings of global scene perception, suggesting representations of auditory objects may be transformed through many stages of processing in the ventral auditory stream, similar to what has been proposed in the ventral visual stream. These findings and the open availability of our scene collection will make future studies on perception, attention, and memory for natural auditory scenes possible.

Efficient coding of natural images in the mouse visual cortex

How the activity of neurons gives rise to natural vision remains a matter of intense investigation. The mid-level visual areas along the ventral stream are selective to a common class of natural images-textures-but a circuit-level understanding of this selectivity and its link to perception remains unclear. We addressed these questions in mice, first showing that they can perceptually discriminate between textures and statistically simpler spectrally matched stimuli, and between texture types. Then, at the neural level, we found that the secondary visual area (LM) exhibited a higher degree of selectivity for textures compared to the primary visual area (V1). Furthermore, textures were represented in distinct neural activity subspaces whose relative distances were found to correlate with the statistical similarity of the images and the mice's ability to discriminate between them. Notably, these dependencies were more pronounced in LM, where the texture-related subspaces were smaller than in V1, resulting in superior stimulus decoding capabilities. Together, our results demonstrate texture vision in mice, finding a linking framework between stimulus statistics, neural representations, and perceptual sensitivity-a distinct hallmark of efficient coding computations.

Naturalistic Object Representations Depend on Distance and Size Cues

Egocentric distance and real-world size are important cues for object perception and action. Nevertheless, most studies of human vision rely on two-dimensional pictorial stimuli that convey ambiguous distance and size information. Here, we use fMRI to test whether pictures are represented differently in the human brain from real, tangible objects that convey unambiguous distance and size cues. Participants directly viewed stimuli in two display formats (real objects and matched printed pictures of those objects) presented at different egocentric distances (near and far). We measured the effects of format and distance on fMRI response amplitudes and response patterns. We found that fMRI response amplitudes in the lateral occipital and posterior parietal cortices were stronger overall for real objects than for pictures. In these areas and many others, including regions involved in action guidance, responses to real objects were stronger for near vs. far stimuli, whereas distance had little effect on responses to pictures-suggesting that distance determines relevance to action for real objects, but not for pictures. Although stimulus distance especially influenced response patterns in dorsal areas that operate in the service of visually guided action, distance also modulated representations in ventral cortex, where object responses are thought to remain invariant across contextual changes. We observed object size representations for both stimulus formats in ventral cortex but predominantly only for real objects in dorsal cortex. Together, these results demonstrate that whether brain responses reflect physical object characteristics depends on whether the experimental stimuli convey unambiguous information about those characteristics.

Significance Statement

Classic frameworks of vision attribute perception of inherent object characteristics, such as size, to the ventral visual pathway, and processing of spatial characteristics relevant to action, such as distance, to the dorsal visual pathway. However, these frameworks are based on studies that used projected images of objects whose actual size and distance from the observer were ambiguous. Here, we find that when object size and distance information in the stimulus is less ambiguous, these characteristics are widely represented in both visual pathways. Our results provide valuable new insights into the brain representations of objects and their various physical attributes in the context of naturalistic vision.

Object-oriented hand dexterity and grasping abilities, from the animal quarters to the neurosurgical OR: a systematic review of the underlying neural correlates in non-human, human primate and recent findings in awake brain surgery

Introduction

The sensorimotor integrations subserving object-oriented manipulative actions have been extensively investigated in non-human primates via direct approaches, as intracortical micro-stimulation (ICMS), cytoarchitectonic analysis and anatomical tracers. However, the understanding of the mechanisms underlying complex motor behaviors is yet to be fully integrated in brain mapping paradigms and the consistency of these findings with intraoperative data obtained during awake neurosurgical procedures for brain tumor removal is still largely unexplored. Accordingly, there is a paucity of systematic studies reviewing the cross-species analogies in neural activities during object-oriented hand motor tasks in primates and investigating the concordance with intraoperative findings during brain mapping. The current systematic review was designed to summarize the cortical and subcortical neural correlates of object-oriented fine hand actions, as revealed by fMRI and PET studies, in non-human and human primates and how those were translated into neurosurgical studies testing dexterous hand-movements during intraoperative brain mapping.

Methods

A systematic literature review was conducted following the PRISMA guidelines. PubMed, EMBASE and Web of Science databases were searched. Original articles were included if they: (1) investigated cortical activation sites on fMRI and/or PET during grasping task; (2) included humans or non-human primates. A second query was designed on the databases above to collect studies reporting motor, hand manipulation and dexterity tasks for intraoperative brain mapping in patients undergoing awake brain surgery for any condition. Due to the heterogeneity in neurosurgical applications, a qualitative synthesis was deemed more appropriate.

Results

We provided an updated overview of the current state of the art in translational neuroscience about the extended frontoparietal grasping-praxis network with a specific focus on the comparative functioning in non-human primates, healthy humans and how the latter knowledge has been implemented in the neurosurgical operating room during brain tumor resection.

Discussion

The anatomical and functional correlates we reviewed confirmed the evolutionary continuum from monkeys to humans, allowing a cautious but practical adoption of such evidence in intraoperative brain mapping protocols. Integrating the previous results in the surgical practice helps preserve complex motor abilities, prevent long-term disability and poor quality of life and allow the maximal safe resection of intrinsic brain tumors.

Explorations of using a convolutional neural network to understand brain activations during movie watching

Neuroimaging studies increasingly use naturalistic stimuli like video clips to trigger complex brain activations, but the complexity of such stimuli makes it difficult to assign specific functions to the resulting brain activations, particularly for higher-level content like social interactions. To address this challenge, researchers have turned to deep neural networks, e.g., convolutional neural networks (CNNs). CNNs have shown success in image recognition due to their different levels of features enabling high performance. In this study, we used pre-trained VGG-16, a popular CNN model, to analyze video data and extract hierarchical features from low-level shallow layers to high-level deeper layers, linking these activations to different levels of activation of the human brain. We hypothesized that activations in different layers of VGG-16 would be associated with different levels of brain activation and visual processing hierarchy in the brain. We were also curious about which brain regions would be associated with deeper convolutional layers in VGG-16. The study analyzed a functional MRI (fMRI) dataset where participants watched the cartoon movie Partly Cloudy. Frames of the videos were fed into VGG-16, and activation maps from different kernels and layers were extracted. Time series of the average activation patterns for each kernel were created and fed into a voxel-wise model to study brain activations. Results showed that lower convolutional layers (1st convolutional layer) were mostly associated with lower visual regions, but some kernels (6, 19, 24, 42, 55, and 58) surprisingly showed associations with activations in the posterior cingulate cortex, part of the default mode network. Deeper convolutional layers were associated with more anterior and lateral portions of the visual cortex (e.g., the lateral occipital complex) and the supramarginal gyrus. Analyzing activation features associated with different brain regions showed the promise and limitations of using CNNs to link video content to brain functions.

Functional decoding and meta-analytic connectivity modeling in thyroid-associated ophthalmopathy

Background

Thyroid-associated ophthalmopathy (TAO) is an orbital disease closely related to thyroid disease with a long-lasting duration that can be blinding and disabling. Recently, structural and functional neuroimaging studies have been performed in TAO patients, but studies have reported inconsistent results. This quantitative meta-analysis was conducted to identify convergent patterns of abnormal brain function among different studies in TAO.

Methods

We searched PubMed, EMBASE, Cochrane, and Web of Science, performed reference tracking, and retrieved 15 eligible studies. Peak coordinates were extracted from these studies and subsequently tested for convergence using activation likelihood estimation (ALE).

Results

Compared to healthy subjects, resting-state brain activity in the whole brain of TAO patients was significantly increased in the left superior frontal gyrus (SFG) and decreased in the left cuneus/precuneus. Functional decoding analysis of the BrainMap database revealed that these regions are predominantly associated with cognitive and emotional impairment. In this study, task-related meta-analytic connectivity modeling (MACM) analysis was used to describe the connectivity and function of the two seed regions. Significant coactivation of these regions was found primarily in the bilateral superior parietal lobule, medial frontal gyrus, left fusiform gyrus, left cingulate gyrus, supplementary motor area and thalamus.

Conclusion

Our findings underscore the role of the SFG and the cuneus/precuneus in the pathophysiology of TAO, highlighting the crucial impact of working memory deficits.

Structural, static, and dynamic functional MRI predictors for conversion from mild cognitive impairment to Alzheimer's disease: Inter-cohort validation of Shanghai Memory Study and ADNI

Mild cognitive impairment (MCI) is a critical prodromal stage of Alzheimer's disease (AD), and the mechanism underlying the conversion is not fully explored. Construction and inter-cohort validation of imaging biomarkers for predicting MCI conversion is of great challenge at present, due to lack of longitudinal cohorts and poor reproducibility of various study-specific imaging indices. We proposed a novel framework for inter-cohort MCI conversion prediction, involving comparison of structural, static, and dynamic functional brain features from structural magnetic resonance imaging (sMRI) and resting-state functional MRI (fMRI) between MCI converters (MCI_C) and non-converters (MCI_NC), and support vector machine for construction of prediction models. A total of 218 MCI patients with 3-year follow-up outcome were selected from two independent cohorts: Shanghai Memory Study cohort for internal cross-validation, and Alzheimer's Disease Neuroimaging Initiative (ADNI) cohort for external validation. In comparison with MCI_NC, MCI_C were mainly characterized by atrophy, regional hyperactivity and inter-network hypo-connectivity, and dynamic alterations characterized by regional and connectional instability, involving medial temporal lobe (MTL), posterior parietal cortex (PPC), and occipital cortex. All imaging-based prediction models achieved an area under the curve (AUC) > 0.7 in both cohorts, with the multi-modality MRI models as the best with excellent performances of AUC > 0.85. Notably, the combination of static and dynamic fMRI resulted in overall better performance as relative to static or dynamic fMRI solely, supporting the contribution of dynamic features. This inter-cohort validation study provides a new insight into the mechanisms of MCI conversion involving brain dynamics, and paves a way for clinical use of structural and functional MRI biomarkers in future.

Feasibility study of functional near-infrared spectroscopy in the ventral visual pathway for real-life applications

Significance

fNIRS-based neuroenhancement depends on the feasible detection of hemodynamic responses in target brain regions. Using the lateral occipital complex (LOC) and the fusiform face area (FFA) in the ventral visual pathway as neurofeedback targets boosts performance in visual recognition. However, the feasibility of utilizing fNIRS to detect LOC and FFA activity in adults remains to be validated as the depth of these regions may exceed the detection limit of fNIRS.

Aim

This study aims to investigate the feasibility of using fNIRS to measure hemodynamic responses in the ventral visual pathway, specifically in the LOC and FFA, in adults.

Approach

We recorded the hemodynamic activities of the LOC and FFA regions in 35 subjects using a portable eight-channel fNIRS instrument. A standard one-back object and face recognition task was employed to elicit selective brain responses in the LOC and FFA regions. The placement of fNIRS optodes for LOC and FFA detection was guided by our group's transcranial brain atlas (TBA).

Results

Our findings revealed selective activation of the LOC target channel (CH2) in response to objects, whereas the FFA target channel (CH7) did not exhibit selective activation in response to faces.

Conclusions

Our findings indicate that, although fNIRS detection has limitations in capturing FFA activity, the LOC region emerges as a viable target for fNIRS-based detection. Furthermore, our results advocate for the adoption of the TBA-based method for setting the LOC target channel, offering a promising solution for optrode placement. This feasibility study stands as the inaugural validation of fNIRS for detecting cortical activity in the ventral visual pathway, underscoring its ecological validity. We suggest that our findings establish a pivotal technical groundwork for prospective real-life applications of fNIRS-based research.

A focus on the multiple interfaces between action and perception and their neural correlates

Successful behaviour relies on the appropriate interplay between action and perception. The well-established dorsal and ventral stream theories depicted two distinct functional pathways for the processes of action and perception, respectively. In physiological conditions, the two pathways closely cooperate in order to produce successful adaptive behaviour. As the coupling between perception and action exists, this requires an interface that is responsible for a common reading of the two functions. Several studies have proposed different types of perception and action interfaces, suggesting their role in the creation of the shared interaction channel. In the present review, we describe three possible perception and action interfaces: i) the motor code, including common coding approaches, ii) attention, and iii) object affordance; we highlight their potential neural correlates. From this overview, a recurrent neural substrate that underlies all these interface functions appears to be crucial: the parieto-frontal circuit. This network is involved in the mirror mechanism which underlies the perception and action interfaces identified as common coding and motor code theories. The same network is also involved in the spotlight of attention and in the encoding of potential action towards objects; these are manifested in the perception and action interfaces for common attention and object affordance, respectively. Within this framework, most studies were dedicated to the description of the role of the inferior parietal lobule; growing evidence, however, suggests that the superior parietal lobule also plays a crucial role in the interplay between action and perception. The present review proposes a novel model that is inclusive of the superior parietal regions and their relative contribution to the different action and perception interfaces.

Visual neuroscience: A specialised neural pathway for social perception

Humans are an intensely social species. Our daily lives depend on understanding the behaviour and intentions of the people around us. A new study identifies a neural pathway specialised for interpreting the physical actions that we use to understand others.

Personal familiarity of faces, animals, objects, and scenes: Distinct perceptual and overlapping conceptual representations

While face, object, and scene recognition are often studied at a basic categorization level (e.g. "a face", "a car", "a kitchen"), we frequently recognise individual items of these categories as unique entities (e.g. "my mother", "my car", "my kitchen"). This recognition of individual identity is essential to appropriate behaviour in our world. However, relatively little is known about how we recognise individually familiar visual stimuli. Using event-related brain potentials, the present study examined whether and to what extent the underlying neural representations of personally familiar items are similar or different across different categories. In three experiments, we examined the recognition of personally highly familiar faces, animals, indoor scenes, and objects. We observed relatively distinct familiarity effects in an early time window (200-400 ms), with a clearly right-lateralized occipito-temporal scalp distribution for human faces and more bilateral and posterior distributions for other stimulus categories, presumably reflecting access to at least partly discrete visual long-term representations. In contrast, we found clearly overlapping familiarity effects in a later time window (starting 400 to 500 ms after stimulus onset), again with a mainly right occipito-temporal scalp distribution, for all stimulus categories. These later effects appear to reflect the sustained activation of conceptual properties relevant to any potential interaction. We conclude that familiarity for items from the various visual stimulus categories tested here is represented differently at the perceptual level, while relatively overlapping conceptual mechanisms allow for the preparation of impending potential interaction with the environment.

Hands off, brain off? A meta-analysis of neuroimaging data during active and passive driving

BACKGROUND

Car driving is more and more automated, to such an extent that driving without active steering control is becoming a reality. Although active driving requires the use of visual information to guide actions (i.e., steering the vehicle), passive driving only requires looking at the driving scene without any need to act (i.e., the human is passively driven).

MATERIALS & METHODS

After a careful search of the scientific literature, 11 different studies, providing 17 contrasts, were used to run a comprehensive meta-analysis contrasting active driving with passive driving.

RESULTS

Two brain regions were recruited more consistently for active driving compared to passive driving, the left precentral gyrus (BA3 and BA4) and the left postcentral gyrus (BA4 and BA3/40), whereas a set of brain regions was recruited more consistently in passive driving compared to active driving: the left middle frontal gyrus (BA6), the right anterior lobe and the left posterior lobe of the cerebellum, the right sub-lobar thalamus, the right anterior prefrontal cortex (BA10), the right inferior occipital gyrus (BA17/18/19), the right inferior temporal gyrus (BA37), and the left cuneus (BA17).

DISCUSSION

From a theoretical perspective, these findings support the idea that the output requirement of the visual scanning process engaged for the same activity can trigger different cerebral pathways, associated with different cognitive processes. A dorsal stream dominance was found during active driving, whereas a ventral stream dominance was obtained during passive driving. From a practical perspective, and contrary to the dominant position in the Human Factors community, our findings support the idea that a transition from passive to active driving would remain challenging as passive and active driving engage distinct neural networks.