Have you been there before? Decoding recognition of spatial scenes from fMRI signals in precuneus

One potential application of forensic "brain reading" is to test whether a suspect has previously experienced a crime scene. Here, we investigated whether it is possible to decode real life autobiographic exposure to spatial locations using fMRI. In the first session, participants visited four out of eight possible rooms on a university campus. During a subsequent scanning session, subjects passively viewed pictures and videos from these eight possible rooms (four old, four novel) without giving any responses. A multivariate searchlight analysis was employed that trained a classifier to distinguish between "seen" versus "unseen" stimuli from a subset of six rooms. We found that bilateral precuneus encoded information that can be used to distinguish between previously seen and unseen rooms and that also generalized to the two stimuli left out from training. We conclude that activity in bilateral precuneus is associated with the memory of previously visited rooms, irrespective of the identity of the room, thus supporting a parietal contribution to episodic memory for spatial locations. Importantly, we could decode whether a room was visited in real life without the need of explicit judgments about the rooms. This suggests that recognition is an automatic response that can be decoded from fMRI data, thus potentially supporting forensic applications of concealed information tests for crime scene recognition.

Erratum: NeuroDecodeR: a package for neural decoding in R

[This corrects the article DOI: 10.3389/fninf.2023.1275903.].

Choice overload interferes with early processing and necessitates late compensation: Evidence from electroencephalogram

Having a multitude of choices can be advantageous, yet an abundance of options can be detrimental to the decision-making process. Based on existing research, the present study combined electroencephalogram and self-reported methodologies to investigate the neural mechanisms underlying the phenomenon of choice overload. Behavioural data suggested that an increase in the number of options led to negative evaluations and avoidance of choice tendencies, even in the absence of time pressure. Event-related potential results indicated that the large choice set interfered with the early visual process, as evidenced by the small P1 amplitude, and failed to attract more attentional resources in the early stage, as evidenced by the small amplitude of P2 and N2. However, the LPC amplitude was increased in the late stage, suggesting greater investment of attentional resources and higher emotional arousal. Multivariate pattern analysis revealed that the difference between small and large choice set began at around 120 ms, and the early and late stages were characterised by opposite activation patterns. This suggested that too many options interfered with early processing and necessitate continued processing at a later stage. In summary, both behavioural and event-related potential (ERP) results confirm the choice overload effect, and it was observed that individuals tend to subjectively exaggerate the choice overload effect.

Decoding neural reactivation of threat during fear learning, extinction, and recall in a randomized clinical trial of L-DOPA among women with PTSD

BACKGROUND

Laboratory paradigms are widely used to study fear learning in posttraumatic stress disorder (PTSD). Recent basic science models demonstrate that, during fear learning, patterns of activity in large neuronal ensembles for the conditioned stimuli (CS) begin to reinstate neural activity patterns for the unconditioned stimuli (US), suggesting a direct way of quantifying fear memory strength for the CS. Here, we translate this concept to human neuroimaging and test the impact of post-learning dopaminergic neurotransmission on fear memory strength during fear acquisition, extinction, and recall among women with PTSD in a re-analysis of previously reported data.

METHODS

Participants (N = 79) completed a context-dependent fear acquisition and extinction task on day 1 and extinction recall tests 24 h later. We decoded activity patterns in large-scale functional networks for the US, then applied this decoder to activity patterns toward the CS on day 1 and day 2.

RESULTS

US decoder output for the CS+ increased during acquisition and decreased during extinction in networks traditionally implicated in human fear learning. The strength of US neural reactivation also predicted individuals skin conductance responses. Participants randomized to receive L-DOPA (n = 43) following extinction on day 1 demonstrated less US neural reactivation on day 2 relative to the placebo group (n = 28).

CONCLUSION

These results support neural reactivation as a measure of memory strength between competing memories of threat and safety and further demonstrate the role of dopaminergic neurotransmission in the consolidation of fear extinction memories.

Coherent categorical information triggers integration-related alpha dynamics

To create coherent visual experiences, the brain spatially integrates the complex and dynamic information it receives from the environment. We previously demonstrated that feedback-related alpha activity carries stimulus-specific information when two spatially and temporally coherent naturalistic inputs can be integrated into a unified percept. In this study, we sought to determine whether such integration-related alpha dynamics are triggered by categorical coherence in visual inputs. In an EEG experiment, we manipulated the degree of coherence by presenting pairs of videos from the same or different categories through two apertures in the left and right visual hemifields. Critically, video pairs could be video-level coherent (i.e., stem from the same video), coherent in their basic-level category, coherent in their superordinate category, or incoherent (i.e., stem from videos from two entirely different categories). We conducted multivariate classification analyses on rhythmic EEG responses to decode between the video stimuli in each condition. As the key result, we significantly decoded the video-level coherent and basic-level coherent stimuli, but not the superordinate coherent and incoherent stimuli, from cortical alpha rhythms. This suggests that alpha dynamics play a critical role in integrating information across space, and that cortical integration processes are flexible enough to accommodate information from different exemplars of the same basic-level category.NEW & NOTEWORTHY Our brain integrates dynamic inputs across the visual field to create coherent visual experiences. Such integration processes have previously been linked to cortical alpha dynamics. In this study, the integration-related alpha activity was observed not only when snippets from the same video were presented, but also when different video snippets from the same basic-level category were presented, highlighting the flexibility of neural integration processes.

Beyond peaks and troughs: Multiplexed performance monitoring signals in the EEG

With the discovery of event-related potentials elicited by errors more than 30 years ago, a new avenue of research on performance monitoring, cognitive control, and decision making emerged. Since then, the field has developed and expanded fulminantly. After a brief overview on the EEG correlates of performance monitoring, this article reviews recent advancements based on single-trial analyses using independent component analysis, multiple regression, and multivariate pattern classification. Given the close interconnection between performance monitoring and reinforcement learning, computational modeling and model-based EEG analyses have made a particularly strong impact. The reviewed findings demonstrate that error- and feedback-related EEG dynamics represent variables reflecting how performance-monitoring signals are weighted and transformed into an adaptation signal that guides future decisions and actions. The model-based single-trial analysis approach goes far beyond conventional peak-and-trough analyses of event-related potentials and enables testing mechanistic theories of performance monitoring, cognitive control, and decision making.

Voxel- and tensor-based morphometry with machine learning techniques identifying characteristic brain impairment in patients with cervical spondylotic myelopathy

Aim

The diagnosis of cervical spondylotic myelopathy (CSM) relies on several methods, including x-rays, computed tomography, and magnetic resonance imaging (MRI). Although MRI is the most useful diagnostic tool, strategies to improve the precise and independent diagnosis of CSM using novel MRI imaging techniques are urgently needed. This study aimed to explore potential brain biomarkers to improve the precise diagnosis of CSM through the combination of voxel-based morphometry (VBM) and tensor-based morphometry (TBM) with machine learning techniques.

Methods

In this retrospective study, 57 patients with CSM and 57 healthy controls (HCs) were enrolled. The structural changes in the gray matter volume and white matter volume were determined by VBM. Gray and white matter deformations were measured by TBM. The support vector machine (SVM) was used for the classification of CSM patients from HCs based on the structural features of VBM and TBM.

Results

CSM patients exhibited characteristic structural abnormalities in the sensorimotor, visual, cognitive, and subcortical regions, as well as in the anterior corona radiata and the corpus callosum [P < 0.05, false discovery rate (FDR) corrected]. A multivariate pattern classification analysis revealed that VBM and TBM could successfully identify CSM patients and HCs [classification accuracy: 81.58%, area under the curve (AUC): 0.85; P < 0.005, Bonferroni corrected] through characteristic gray matter and white matter impairments.

Conclusion

CSM may cause widespread and remote impairments in brain structures. This study provided a valuable reference for developing novel diagnostic strategies to identify CSM.

NeuroDecodeR: a package for neural decoding in R

Neural decoding is a powerful method to analyze neural activity. However, the code needed to run a decoding analysis can be complex, which can present a barrier to using the method. In this paper we introduce a package that makes it easy to perform decoding analyses in the R programing language. We describe how the package is designed in a modular fashion which allows researchers to easily implement a range of different analyses. We also discuss how to format data to be able to use the package, and we give two examples of how to use the package to analyze real data. We believe that this package, combined with the rich data analysis ecosystem in R, will make it significantly easier for researchers to create reproducible decoding analyses, which should help increase the pace of neuroscience discoveries.

Grey matter structure within the visual networks in migraine with aura: multivariate and univariate analyses

BACKGROUND

The visual cortex is involved in the generation of migraine aura. Voxel-based multivariate analyses applied to this region may provide complementary information about aura mechanisms relative to the commonly used mass-univariate analyses.

METHODS

Structural images constrained within the functional resting-state visual networks were obtained in migraine patients with (n = 50) and without (n = 50) visual aura and healthy controls (n = 50). The masked images entered a multivariate analysis in which Gaussian process classification was used to generate pairwise models. Generalizability was assessed by five-fold cross-validation and non-parametric permutation tests were used to estimate significance levels. A univariate voxel-based morphometry analysis was also performed.

RESULTS

A multivariate pattern of grey matter voxels within the ventral medial visual network contained significant information related to the diagnosis of migraine with visual aura (aura vs. healthy controls: classification accuracy = 78%, p < 0.001; area under the curve = 0.84, p < 0.001; migraine with aura vs. without aura: classification accuracy = 71%, p < 0.001; area under the curve = 0.73, p < 0.003). Furthermore, patients with visual aura exhibited increased grey matter volume in the medial occipital cortex compared to the two other groups.

CONCLUSIONS

Migraine with visual aura is characterized by multivariate and univariate patterns of grey matter changes within the medial occipital cortex that have discriminative power and may reflect pathological mechanisms.

Prolonged neural encoding of visual information in autism

Autism spectrum disorder (ASD) is associated with a hyper-focused visual attentional style, impacting higher-order social and affective domains. The understanding of such peculiarity can benefit from the use of multivariate pattern analysis (MVPA) of high-resolution electroencephalography (EEG) data, which has proved to be a powerful technique to investigate the hidden neural dynamics orchestrating sensory and cognitive processes. Here, we recorded EEG in typically developing (TD) children and in children with ASD during a visuo-spatial attentional task where attention was exogenously captured by a small (zoom-in) or large (zoom-out) cue in the visual field before the appearance of a target at different eccentricities. MVPA was performed both in the cue-locked period, to reveal potential differences in the modulation of the attentional focus, and in the target-locked period, to reveal potential cascade effects on stimulus processing. Cue-locked MVPA revealed that while in the TD group the pattern of neural activity contained information about the cue mainly before the target appearance, the ASD group showed a temporally sustained and topographically diffuse significant decoding of the cue neural response even after the target onset, suggesting a delayed extinction of cue-related neural activity. Crucially, this delayed extinction positively correlated with behavioral measures of attentional hyperfocusing. Results of target-locked MVPA were coherent with a hyper-focused attentional profile, highlighting an earlier and stronger decoding of target neural responses in small cue trials in the ASD group. The present findings document a spatially and temporally overrepresented encoding of visual information in ASD, which can constitute one of the main reasons behind their peculiar cognitive style.

Representational similarity analysis of self- versus other-processing: Effect of trait aggressiveness

In this study, using the self/other adjective judgment task, we aimed to explore how people perceive themselves in comparison to various other people, including friends, strangers, and those they dislike. Next, using representational similarity analysis, we sought to elucidate how these perceptual similarities and differences are represented in brain activity and how aggressiveness is related to these representations. Behavioral ratings show that, on average, people tend to consider themselves more like their friends than neutral strangers, and least like people they dislike. This pattern of similarity is positively correlated with neural representation in social and cognitive circuits of the brain and negatively correlated with neural representation in emotional centers that may represent emotional arousal associated with various social objects. Aggressiveness seems to predispose a person to a pattern of behavior that is the opposite of the average pattern, that is, a tendency to think of oneself as less like one's friends and more like one's enemies. This corresponds to an increase in the similarity of the behavioral representation with the representation in the emotional centers and a decrease in its similarity with the representation in the social and cognitive centers. This can be seen as evidence that in individuals prone to aggression, behavior in the social environment may depend to a greater extent on the representation of social objects in the emotional rather than social and cognitive brain circuits.

Lifelong olfactory deprivation-dependent cortical reorganization restricted to orbitofrontal cortex

Prolonged sensory deprivation has repeatedly been linked to cortical reorganization. We recently demonstrated that individuals with congenital anosmia (CA, complete olfactory deprivation since birth) have seemingly normal morphology in piriform (olfactory) cortex despite profound morphological deviations in the orbitofrontal cortex (OFC), a finding contradictory to both the known effects of blindness on visual cortex and to the sparse literature on brain morphology in anosmia. To establish whether these unexpected findings reflect the true brain morphology in CA, we first performed a direct replication of our previous study to determine if lack of results was due to a deviant control group, a confound in cross sectional studies. Individuals with CA (n = 30) were compared to age and sex matched controls (n = 30) using voxel- and surface-based morphometry. The replication results were near identical to the original study: bilateral clusters of group differences in the OFC, including CA atrophy around the olfactory sulci and volume increases in the medial orbital gyri. Importantly, no group differences in piriform cortex were detected. Subsequently, to assess any subtle patterns of group differences not detectable by our mass-univariate analysis, we explored the data from a multivariate perspective. Combining the newly collected data with data from the replicated study (CA = 49, control = 49), we performed support vector machine classification based on gray matter volume. In line with the mass-univariate analyses, the multivariate analysis could accurately differentiate between the groups in bilateral OFC, whereas the classification accuracy in piriform cortex was at chance level. Our results suggest that despite lifelong olfactory deprivation, piriform (olfactory) cortex is morphologically unaltered and the morphological deviations in CA are confined to the OFC.

Your place or mine? The neural dynamics of personally familiar scene recognition suggests category independent familiarity encoding

Recognizing a stimulus as familiar is an important capacity in our everyday life. Recent investigation of visual processes has led to important insights into the nature of the neural representations of familiarity for human faces. Still, little is known about how familiarity affects the neural dynamics of non-face stimulus processing. Here we report the results of an EEG study, examining the representational dynamics of personally familiar scenes. Participants viewed highly variable images of their own apartments and unfamiliar ones, as well as personally familiar and unfamiliar faces. Multivariate pattern analyses were used to examine the time course of differential processing of familiar and unfamiliar stimuli. Time-resolved classification revealed that familiarity is decodable from the EEG data similarly for scenes and faces. The temporal dynamics showed delayed onsets and peaks for scenes as compared to faces. Familiarity information, starting at 200 ms, generalized across stimulus categories and led to a robust familiarity effect. In addition, familiarity enhanced category representations in early (250-300 ms) and later (>400 ms) processing stages. Our results extend previous face familiarity results to another stimulus category and suggest that familiarity as a construct can be understood as a general, stimulus-independent processing step during recognition.

Orientation representation in human visual cortices: contributions of non-visual information and action-related process

Orientation processing in the human brain plays a crucial role in guiding grasping actions toward an object. Remarkably, despite the absence of visual input, the human visual cortex can still process orientation information. Instead of visual input, non-visual information, including tactile and proprioceptive sensory input from the hand and arm, as well as feedback from action-related processes, may contribute to orientation processing. However, the precise mechanisms by which the visual cortices process orientation information in the context of non-visual sensory input and action-related processes remain to be elucidated. Thus, our study examined the orientation representation within the visual cortices by analyzing the blood-oxygenation-level-dependent (BOLD) signals under four action conditions: direct grasp (DG), air grasp (AG), non-grasp (NG), and uninformed grasp (UG). The images of the cylindrical object were shown at +45° or - 45° orientations, corresponding to those of the real object to be grasped with the whole-hand gesture. Participants judged their orientation under all conditions. Grasping was performed without online visual feedback of the hand and object. The purpose of this design was to investigate the visual areas under conditions involving tactile feedback, proprioception, and action-related processes. To address this, a multivariate pattern analysis was used to examine the differences among the cortical patterns of the four action conditions in orientation representation by classification. Overall, significant decoding accuracy over chance level was discovered for the DG; however, during AG, only the early visual areas showed significant accuracy, suggesting that the object's tactile feedback influences the orientation process in higher visual areas. The NG showed no statistical significance in any area, indicating that without the grasping action, visual input does not contribute to cortical pattern representation. Interestingly, only the dorsal and ventral divisions of the third visual area (V3d and V3v) showed significant decoding accuracy during the UG despite the absence of visual instructions, suggesting that the orientation representation was derived from action-related processes in V3d and visual recognition of object visualization in V3v. The processing of orientation information during non-visually guided grasping of objects relies on other non-visual sources and is specifically divided by the purpose of action or recognition.

Distributed attribute representation in the superior parietal lobe during probabilistic decision-making

Several studies have examined the neural substrates of probabilistic decision-making, but few have systematically investigated the neural representations of the two objective attributes of probabilistic rewards, that is, the reward amount and the probability. Specifically, whether there are common or distinct neural activity patterns to represent the objective attributes and their association with the neural representation of the subjective valuation remains largely underexplored. We conducted two studies (nStudy1  = 34, nStudy2  = 41) to uncover distributed neural representations of the objective attributes and subjective value as well as their association with individual probability discounting rates. The amount and probability were independently manipulated to better capture brain signals sensitive to these two attributes and were presented simultaneously in Study 1 and successively in Study 2. Both univariate and multivariate pattern analyses showed that the brain activities in the superior parietal lobule (SPL), including the postcentral gyrus, were modulated by the amount of rewards and probability in both studies. Further, representational similarity analysis revealed a similar neural representation between these two objective attributes and between the attribute and valuation. Moreover, the SPL tracked the subjective value integrated by the hyperbolic function. Probability-related brain activations in the inferior parietal lobule were associated with the variability in individual discounting rates. These findings provide novel insights into a similar neural representation of the two attributes during probabilistic decision-making and perhaps support the common neural coding of stimulus objective properties and subjective value in the field of probabilistic discounting.

Identification and effective connections of core networks in patients with temporal lobe epilepsy and cognitive impairment: Granger causality analysis and multivariate pattern analysis

OBJECTIVE

This study aimed to explore effective connectivity (EC) of the core networks in cognition impairment associated with temporal lobe epilepsy (CI-TLE) by applying resting state and Granger causality analysis (REST-GCA). The specific brain regions that played a critical role in classification were assessed using multivariate pattern analysis (MVPA).

METHODS

Thirty-two patients with CI-TLE and 29 healthy controls who were matched based on age and gender underwent functional magnetic resonance imaging (fMRI).

RESULTS

REST-GCA revealed that patients with CI-TLE displayed decreased GC values in the following brain areas: from the posterior cingulate cortex (PCC) to the left fusiform gyrus (lFFG) and the right parahippocampal gyrus (rPPG); from the right dorsal prefrontal cortex (rDPFC) to the left superior parietal lobule (lSPL); from the left amygdala (lAG) to the PCC. Inhibitory EC was observed from the rDPFC to the PCC compared to HCs. The GC values increased from the right dorsal prefrontal cingulate cortex (rdACC) to the PCC and from the right dorsal forebrain insula (rDAI) to the right middle temporal gyrus (rMTG) in the CI-TLE patients. MVPA showed that the classification yielded an accuracy of 81.91% (78.12%, specificity =85.71%).

CONCLUSION

Our observations indicated that the abnormal EC between the frontal and parietal regions might be associated with the pathophysiological mechanism of CI-TLE. These results also indicated that EC might be play a role as a potential discriminative pattern to detect CI-TLE in patients.

Decoding neural patterns for the processing of fearful faces under different visual awareness conditions: A multivariate pattern analysis

Previous studies have provided mixed findings regarding the nonconscious processing of fearful faces. Here, we used multivariate pattern analysis on electroencephalography data from three backward masking experiments to examine the processing of fearful faces under different visual awareness conditions. Three groups of participants were shown pairs of face images presented very briefly (for 16 ms) or for sufficiently long (for 266 ms), and completed tasks where the faces were either relevant to the experimental task (Experiment 1) or not (Experiments 2 and 3). Three main decoding analyses were performed. First, in the visual awareness decoding, the visibility of the faces, and hence participants' awareness of them, was maximally decodable in three time windows: 158-168 ms, 235-260 ms and 400-600 ms where the earlier neural patterns were generalized to the later stage activity. Second, we found that the spatial location of a fearful face in the face pairs was decodable, however only when the faces were consciously seen and task-relevant. Finally, we successfully decoded distinct neural patterns associated with the fearful-face-present conditions, compared to the fearful-face-absent conditions, and these patterns were decodable during both short and long presentations of the faces. Together, our results suggest that, while the processing of the spatial location of fearful faces requires awareness and task-relevancy, the mere presence of fearful faces can be processed even when visual awareness is highly restricted.

Brain Representation of Animal and Non-Animal Images in Patients with Mild Cognitive Impairment and Alzheimer's Disease

Background

In early Alzheimer's disease (AD), high-level visual functions and processing speed are impacted. Few functional magnetic resonance imaging (fMRI) studies have investigated high-level visual deficits in AD, yet none have explored brain activity patterns during rapid animal/non-animal categorization tasks.

Objective

To address this, we utilized the previously known Integrated Cognitive Assessment (ICA) to collect fMRI data and compare healthy controls (HC) to individuals with mild cognitive impairment (MCI) and mild AD.

Methods

The ICA encompasses a rapid visual categorization task that involves distinguishing between animals and non-animals within natural scenes. To comprehensively explore variations in brain activity levels and patterns, we conducted both univariate and multivariate analyses of fMRI data.

Results

The ICA task elicited activation across a range of brain regions, encompassing the temporal, parietal, occipital, and frontal lobes. Univariate analysis, which compared responses to animal versus non-animal stimuli, showed no significant differences in the regions of interest (ROIs) across all groups, with the exception of the left anterior supramarginal gyrus in the HC group. In contrast, multivariate analysis revealed that in both HC and MCI groups, several regions could differentiate between animals and non-animals based on distinct patterns of activity. Notably, such differentiation was absent within the mild AD group.

Conclusions

Our study highlights the ICA task's potential as a valuable cognitive assessment tool designed for MCI and AD. Additionally, our use of fMRI pattern analysis provides valuable insights into the complex changes in brain function associated with AD. This approach holds promise for enhancing our understanding of the disease's progression.

The role of distractors in rapid serial visual presentation reveals the mechanism of attentional blink by EEG-based univariate and multivariate analyses

Attentional blink pertains to the performance of participants with a severe decline in identifying the second target presented after the first target reported correctly within 200-500 ms in a rapid serial visual presentation. The current study was conducted to investigate the neural mechanism of the effect of the distractor (D1) that immediately follows first target to attentional blink by altering whether D1 was substituted with a blank with electroencephalography recording. The results showed that D1 interfered with the attentional enhancement and working memory encoding in both single-target rapid serial visual presentation task and dual-target rapid serial visual presentation task, which were mainly manifested in delayed and attenuated P3a and diminished P3b of first target. Single-trial analysis indicated that first target and second target will compete with each other for working memory encoding resources in short lag, but not in the long lag. In addition, D1 interfered with the working memory encoding of second target under short lag rather than long lag in the dual-target rapid serial visual presentation task. These results suggested that attentional blink can be attributed to the limited working memory encoding resource, whereas the amount of available resources is subject to modulation by attention. The D1 hinders the attention enhancement of first target, thereby exacerbating attentional blink.

Subjectively preferred octave size is resolved at the late stages of cerebral auditory processing

Human listeners prefer octave intervals slightly above the exact 2:1 frequency ratio. To study the neural underpinnings of this subjective preference, called the octave enlargement phenomenon, we compared neural responses between exact, slightly enlarged, oversized, and compressed octaves (or their multiples). The first experiment (n = 20) focused on the N1 and P2 event-related potentials (ERPs) elicited in EEG 50-250 ms after the second tone onset during passive listening of one-octave intervals. In the second experiment (n = 20) applying four-octave intervals, musician participants actively rated the different octave types as 'low', 'good' and 'high'. The preferred slightly enlarged octave was individually determined prior to the second experiment. In both experiments, N1-P2 peak-to-peak amplitudes attenuated for the exact and slightly enlarged octave intervals compared with compressed and oversized intervals, suggesting overlapping neural representations of tones an octave (or its multiples) apart. While there were no differences between the N1-P2 amplitudes to the exact and preferred enlarged octaves, ERP amplitudes differed after 500 ms from onset of the second tone of the pair. In the multivariate pattern analysis (MVPA) of the second experiment, the different octave types were distinguishable (spatial classification across electroencephalography [EEG] channels) 200 ms after second tone onset. Temporal classification within channels suggested two separate discrimination processes peaking around 300 and 700 ms. These findings appear to be related to active listening, as no multivariate results were found in the first, passive listening experiment. The present results suggest that the subjectively preferred octave size is resolved at the late stages of auditory processing.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

The neural dynamics of familiarity-dependent face identity representation

Recognizing a face as belonging to a given identity is essential in our everyday life. Clearly, the correct identification of a face is only possible for familiar people, but 'familiarity' covers a wide range-from people we see every day to those we barely know. Although several studies have shown that the processing of familiar and unfamiliar faces is substantially different, little is known about how the degree of familiarity affects the neural dynamics of face identity processing. Here, we report the results of a multivariate EEG analysis, examining the representational dynamics of face identity across several familiarity levels. Participants viewed highly variable face images of 20 identities, including the participants' own face, personally familiar (PF), celebrity and unfamiliar faces. Linear discriminant classifiers were trained and tested on EEG patterns to discriminate pairs of identities of the same familiarity level. Time-resolved classification revealed that the neural representations of identity discrimination emerge around 100 ms post-stimulus onset, relatively independently of familiarity level. In contrast, identity decoding between 200 and 400 ms is determined to a large extent by familiarity: it can be recovered with higher accuracy and for a longer duration in the case of more familiar faces. In addition, we found no increased discriminability for faces of PF persons compared to those of highly familiar celebrities. One's own face benefits from processing advantages only in a relatively late time-window. Our findings provide new insights into how the brain represents face identity with various degrees of familiarity and show that the degree of familiarity modulates the available identity-specific information at a relatively early time window.

Differential processing of intrinsic vs. extrinsic coordinates in wrist movement: connectivity and chronometry perspectives

This study explores brain-network differences between the intrinsic and extrinsic motor coordinate frames. A connectivity model showing the coordinate frames difference was obtained using brain fMRI data of right wrist isometric flexions and extensions movements, performed in two forearm postures. The connectivity model was calculated by machine-learning-based neural representation and effective functional connectivity using psychophysiological interaction and dynamic causal modeling analyses. The model indicated the network difference wherein the inferior parietal lobule receives extrinsic information from the rostral lingual gyrus through the superior parietal lobule and transmits intrinsic information to the Handknob, whereas extrinsic information is transmitted to the Handknob directly from the rostral lingual gyrus. A behavioral experiment provided further evidence on the difference between motor coordinate frames showing onset timing delay of muscle activity of intrinsic coordinate-directed wrist movement compared to extrinsic one. These results suggest that, if the movement is externally directed, intrinsic coordinate system information is bypassed to reach the primary motor area.

Multivariate physical activity association patterns for fundamental motor skills and physical fitness in preschool children aged 3-5 years

Physical activity (PA) is important for children's development of fundamental motor skills (FMS) and physical fitness (FIT) but evidence regarding which intensities are associated with these outcomes in early childhood is limited. The aim of this study was to determine the cross-sectional multivariate PA intensity signatures associated with FMS and FIT in children aged 3-5 years. We used a sample of 952 Norwegian preschoolers (4.3 years, 51% boys) who provided data on PA (ActiGraph GT3X+), at least one FMS (locomotor, object control and/or balance skills) or FIT (speed agility, standing long jump, and/or handgrip strength) outcome, body mass index, and socioeconomic status in 2019-2020. We created 17 PA intensity variables (0-99 to ≥15000 counts per minute) from the vertical axis and used multivariate pattern analysis for analyses. The PA intensity spectrum (including sedentary time) was significantly associated with all outcomes. Associations for PA intensities were positive (negative for sedentary time), strongest for moderate and vigorous intensities, and were significant across sex and age groups. Our findings show that the PA intensity spectrum is associated with FMS and FIT in young children and that promotion of PA, in particular moderate- and vigorous-intensity activity, from an early age benefits children's physical development.

Erratum: Attentional capture by fearful faces requires consciousness and is modulated by task-relevancy: a dot-probe EEG study

[This corrects the article DOI: 10.3389/fnins.2023.1152220.].

Cognitive Dynamics Estimation: A whole-brain spatial regression paradigm for extracting the temporal dynamics of cognitive processes

Functional MRI (fMRI) has been instrumental in understanding how cognitive processes are spatially mapped in the brain, yielding insights about brain regions and functions. However, in case the orthogonality of behavioral or stimulus timing is not guaranteed, the estimated brain maps fail to dissociate each cognitive process, and the resultant maps become unstable. Also, the brain mapping exercise can not provide temporal information on the cognitive process. Here we propose a qualitatively different approach to fMRI analysis, named Cognitive Dynamics Estimation (CDE), that estimates how multiple cognitive processes change over time even when behavior or stimulus logs are unavailable. This method transposes the conventional brain mapping; the brain activity pattern at each time point is subject to regression analysis with data-driven maps of cognitive processes as regressors, resulting in the time series of cognitive processes. The estimated time series captured the fluctuation of intensity and timing of cognitive processes on a trial-by-trial basis, which conventional analysis could not capture. Notably, the estimated time series predicted participants' cognitive ability to perform each psychological task. As an addition to our fMRI analytic toolkit, these results suggest the potential for CDE to elucidate underexplored cognitive phenomena, especially in the temporal domain.

Decoding of Working Memory Contents in Auditory Cortex Is Not Distractor-Resistant

Working memory enables the temporary storage of relevant information in the service of behavior. Neuroimaging studies have suggested that sensory cortex is involved in maintaining contents in working memory. This raised the question of how sensory regions maintain memory representations during the exposure to distracting stimuli. Multivariate pattern analysis of fMRI signals in visual cortex has shown that the contents of visual working memory could be decoded concurrently with passively viewed distractors. The present fMRI study tested whether this finding extends to auditory working memory and to active distractor processing. We asked participants to memorize the pitch of a target sound and to compare it with a probe sound presented after a 13 s delay period. In separate conditions, we compared a blank delay phase (no distraction) with either passive listening to, or active processing of, an auditory distractor presented throughout the memory delay. Consistent with previous reports, pitch-specific memory information could be decoded in auditory cortex during the delay in trials without distraction. In contrast, decoding of target sounds in early auditory cortex dropped to chance level during both passive and active distraction. This was paralleled by memory performance decrements under distraction. Extending the analyses beyond sensory cortex yielded some evidence for memory content-specific activity in inferior frontal and superior parietal cortex during active distraction. In summary, while our findings question the involvement of early auditory cortex in the maintenance of distractor-resistant working memory contents, further research should elucidate the role of hierarchically higher regions.SIGNIFICANCE STATEMENT Information about sensory features held in working memory can be read out from hemodynamic activity recorded in human sensory cortices. Moreover, visual cortex can in parallel store visual content and process newly incoming, task-irrelevant visual input. The present study investigated the role of auditory cortex for working memory maintenance under distraction. While memorized sound frequencies could be decoded in auditory cortex in the absence of distraction, auditory distraction during the delay phase impaired memory performance and prevented decoding of information stored in working memory. Apparently, early auditory cortex is not sufficient to represent working memory contents under distraction that impairs performance. However, exploratory analyses indicated that, under distraction, higher-order frontal and parietal regions might contribute to content-specific working memory storage.

Neural Basis of Social Influence of Observing Other's Perception in Dot-Number Estimation

Our perceptions and decisions are often implicitly influenced by observing another's actions. However, it is unclear how observing other people's perceptual decisions without interacting with them can engage the processing of self-other discrepancies and change the observer's decisions. In this study, we employed functional magnetic resonance imaging and a computational model to investigate the neural basis of how unilaterally observing the other's perceptual decisions modulated one's own decisions. The experimental task was to discriminate whether the number of presented dots was higher or lower than a reference number. The participants performed the task solely while unilaterally observing the performance of another "participant," who produced overestimations and underestimations in the same task in separate sessions. Results of the behavioral analysis showed that the participants' decisions were modulated to resemble those of the other. Image analysis based on computational model revealed that the activation in the medial prefrontal cortex was associated with the discrepancy between the inferred participant's and the presented other's decisions. In addition, the number-sensitive region in the superior parietal region showed altered activation patterns after observing the other's overestimations and underestimations. The activity of the superior parietal region was not involved in assessing the observation of other's perceptual decisions, but it was engaged in plain numerosity perception. These results suggest that computational modeling can capture the neuro-behavioral processing of self-other discrepancies in perception followed by the activity modulation in the number-sensitive region in the task of dot-number estimation.

Modality-specific and modality-independent neural representations work in concert in predictive processes during sequence learning

Probabilistic sequence learning supports the development of skills and enables predictive processing. It remains contentious whether visuomotor sequence learning is driven by the representation of the visual sequence (perceptual coding) or by the representation of the response sequence (motor coding). Neurotypical adults performed a visuomotor sequence learning task. Learning occurred incidentally as it was evidenced by faster responses to high-probability than to low-probability targets. To uncover the neurophysiology of the learning process, we conducted both univariate analyses and multivariate pattern analyses (MVPAs) on the temporally decomposed EEG signal. Univariate analyses showed that sequence learning modulated the amplitudes of the motor code of the decomposed signal but not in the perceptual and perceptual-motor signals. However, MVPA revealed that all 3 codes of the decomposed EEG contribute to the neurophysiological representation of the learnt probabilities. Source localization revealed the involvement of a wider network of frontal and parietal activations that were distinctive across coding levels. These findings suggest that perceptual and motor coding both contribute to the learning of sequential regularities rather than to a neither-nor distinction. Moreover, modality-specific encoding worked in concert with modality-independent representations, which suggests that probabilistic sequence learning is nonunitary and encompasses a set of encoding principles.

Functional organization of social perception in the human brain

Humans rapidly extract diverse and complex information from ongoing social interactions, but the perceptual and neural organization of the different aspects of social perception remains unresolved. We showed short movie clips with rich social content to 97 healthy participants while their haemodynamic brain activity was measured with fMRI. The clips were annotated moment-to-moment for a large set of social features and 45 of the features were evaluated reliably between annotators. Cluster analysis of the social features revealed that 13 dimensions were sufficient for describing the social perceptual space. Three different analysis methods were used to map the social perceptual processes in the human brain. Regression analysis mapped regional neural response profiles for different social dimensions. Multivariate pattern analysis then established the spatial specificity of the responses and intersubject correlation analysis connected social perceptual processing with neural synchronization. The results revealed a gradient in the processing of social information in the brain. Posterior temporal and occipital regions were broadly tuned to most social dimensions and the classifier revealed that these responses showed spatial specificity for social dimensions; in contrast Heschl gyri and parietal areas were also broadly associated with different social signals, yet the spatial patterns of responses did not differentiate social dimensions. Frontal and subcortical regions responded only to a limited number of social dimensions and the spatial response patterns did not differentiate social dimension. Altogether these results highlight the distributed nature of social processing in the brain.

Left frontal eye field encodes sound locations during passive listening

Previous studies reported that auditory cortices (AC) were mostly activated by sounds coming from the contralateral hemifield. As a result, sound locations could be encoded by integrating opposite activations from both sides of AC ("opponent hemifield coding"). However, human auditory "where" pathway also includes a series of parietal and prefrontal regions. It was unknown how sound locations were represented in those high-level regions during passive listening. Here, we investigated the neural representation of sound locations in high-level regions by voxel-level tuning analysis, regions-of-interest-level (ROI-level) laterality analysis, and ROI-level multivariate pattern analysis. Functional magnetic resonance imaging data were collected while participants listened passively to sounds from various horizontal locations. We found that opponent hemifield coding of sound locations not only existed in AC, but also spanned over intraparietal sulcus, superior parietal lobule, and frontal eye field (FEF). Furthermore, multivariate pattern representation of sound locations in both hemifields could be observed in left AC, right AC, and left FEF. Overall, our results demonstrate that left FEF, a high-level region along the auditory "where" pathway, encodes sound locations during passive listening in two ways: a univariate opponent hemifield activation representation and a multivariate full-field activation pattern representation.

Neural response and representation: Facial expressions in scenes

Previous studies have shown that the brain generates expectations based on scenes, which affects facial expression recognition. However, although facial expressions are known to interact with perception, the mechanism underlying this interaction remains poorly understood. Here, we used frequency labeling and decoding techniques to reveal the effects of scene-based expectation on the amplitude and representational strength of neural activity. We also reduced the relative reliability between expectation and sensory input by blurring facial expressions to further investigate the effects of this relative reliability on the pattern of neural activation and representation. Participants viewed emotional changes in unblurred or blurred facial expressions, which flickered at a rate of 6 Hz within a scene. We found that facial expressions that were congruent with the emotional significance of the scene elicited a larger steady-state visual evoked potential amplitude than did facial expressions that were incongruent with the emotional significance of a scene, in both unblurred and blurred conditions. We also found that expected facial expression representations were stronger than unexpected representations during the unblurred condition. In the blurred condition, unexpected representations were stronger than expected representations. Taken together, these results suggested that facial expression processing in the visual cortex is modulated by top-down signals. The relative reliability of expectation and sensory input moderated the influence of a scene on facial expression representation. Furthermore, our study showed that neural activation amplitudes did not correspond to representational strength.

Decoding semantic representations in mind and brain

A key goal for cognitive neuroscience is to understand the neurocognitive systems that support semantic memory. Recent multivariate analyses of neuroimaging data have contributed greatly to this effort, but the rapid development of these novel approaches has made it difficult to track the diversity of findings and to understand how and why they sometimes lead to contradictory conclusions. We address this challenge by reviewing cognitive theories of semantic representation and their neural instantiation. We then consider contemporary approaches to neural decoding and assess which types of representation each can possibly detect. The analysis suggests why the results are heterogeneous and identifies crucial links between cognitive theory, data collection, and analysis that can help to better connect neuroimaging to mechanistic theories of semantic cognition.

Optimizing multivariate pattern classification in rapid event-related designs

BACKGROUND

Multivariate pattern analysis (MVPA or pattern decoding) has attracted considerable attention as a sensitive analytic tool for investigations using functional magnetic resonance imaging (fMRI) data. With the introduction of MVPA, however, has come a proliferation of methodological choices confronting the researcher, with few studies to date offering guidance from the vantage point of controlled datasets detached from specific experimental hypotheses.

NEW METHOD

We investigated the impact of four data processing steps on support vector machine (SVM) classification performance aimed at maximizing information capture in the presence of common noise sources. The four techniques included: trial averaging (classifying on separate trial estimates versus condition-based averages), within-run mean centering (centering the data or not), method of cost selection (using a fixed or tuned cost value), and motion-related denoising approach (comparing no denoising versus a variety of nuisance regressions capturing motion-related reference signals). The impact of these approaches was evaluated on real fMRI data from two control ROIs, as well as on simulated pattern data constructed with carefully controlled voxel- and trial-level noise components.

RESULTS

We find significant improvements in classification performance across both real and simulated datasets with run-wise trial averaging and mean centering. When averaging trials within conditions of each run, we note a simultaneous increase in the between-subject variability of SVM classification accuracies which we attribute to the reduced size of the test set used to assess the classifier's prediction error. Therefore, we propose a hybrid technique whereby randomly sampled subsets of trials are averaged per run and demonstrate that it helps mitigate the tradeoff between improving signal-to-noise ratio by averaging and losing exemplars in the test set.

COMPARISON WITH EXISTING METHODS

Though a handful of empirical studies have employed run-based trial averaging, mean centering, or their combination, such studies have done so without theoretical justification or rigorous testing using control ROIs.

CONCLUSIONS

Therefore, we intend this study to serve as a practical guide for researchers wishing to optimize pattern decoding without risk of introducing spurious results.

A unifying model for discordant and concordant results in human neuroimaging studies of facial viewpoint selectivity

Our ability to recognize faces regardless of viewpoint is a key property of the primate visual system. Traditional theories hold that facial viewpoint is represented by view-selective mechanisms at early visual processing stages and that representations become increasingly tolerant to viewpoint changes in higher-level visual areas. Newer theories, based on single-neuron monkey electrophysiological recordings, suggest an additional intermediate processing stage invariant to mirror-symmetric face views. Consistent with traditional theories, human studies combining neuroimaging and multivariate pattern analysis (MVPA) methods have provided evidence of view-selectivity in early visual cortex. However, contradictory results have been reported in higher-level visual areas concerning the existence in humans of mirror-symmetrically tuned representations. We believe these results reflect low-level stimulus confounds and data analysis choices. To probe for low-level confounds, we analyzed images from two popular face databases. Analyses of mean image luminance and contrast revealed biases across face views described by even polynomials-i.e., mirror-symmetric. To explain major trends across human neuroimaging studies of viewpoint selectivity, we constructed a network model that incorporates three biological constraints: cortical magnification, convergent feedforward projections, and interhemispheric connections. Given the identified low-level biases, we show that a gradual increase of interhemispheric connections across network layers is sufficient to replicate findings of mirror-symmetry in high-level processing stages, as well as view-tuning in early processing stages. Data analysis decisions-pattern dissimilarity measure and data recentering-accounted for the variable observation of mirror-symmetry in late processing stages. The model provides a unifying explanation of MVPA studies of viewpoint selectivity. We also show how common analysis choices can lead to erroneous conclusions.

Two distinct networks containing position-tolerant representations of actions in the human brain

Humans can recognize others' actions in the social environment. This action recognition ability is rarely hindered by the movement of people in the environment. The neural basis of this position tolerance for observed actions is not fully understood. Here, we aimed to identify brain regions capable of generalizing representations of actions across different positions and investigate the representational content of these regions. In a functional magnetic resonance imaging experiment, participants viewed point-light displays of different human actions. Stimuli were presented in either the upper or the lower visual field. Multivariate pattern analysis and a surface-based searchlight approach were employed to identify brain regions that contain position-tolerant action representation: Classifiers were trained with patterns in response to stimuli presented in one position and were tested with stimuli presented in another position. Results showed above-chance classification in the left and right lateral occipitotemporal cortices, right intraparietal sulcus, and right postcentral gyrus. Further analyses exploring the representational content of these regions showed that responses in the lateral occipitotemporal regions were more related to subjective judgments, while those in the parietal regions were more related to objective measures. These results provide evidence for two networks that contain abstract representations of human actions with distinct representational content.

The non-specific nature of mental health and structural brain outcomes following childhood trauma

BACKGROUND

Childhood trauma (CT) is associated with an increased risk of mental health disorders; however, it is unknown whether this represents a diagnosis-specific risk factor for specific psychopathology mediated by structural brain changes. Our aim was to explore whether (i) a predictive CT pattern for transdiagnostic psychopathology exists, and whether (ii) CT can differentiate between distinct diagnosis-dependent psychopathology. Furthermore, we aimed to identify the association between CT, psychopathology and brain structure.

METHODS

We used multivariate pattern analysis in data from 643 participants of the Personalised Prognostic Tools for Early Psychosis Management study (PRONIA), including healthy controls (HC), recent onset psychosis (ROP), recent onset depression (ROD), and patients clinically at high-risk for psychosis (CHR). Participants completed structured interviews and self-report measures including the Childhood Trauma Questionnaire, SCID diagnostic interview, BDI-II, PANSS, Schizophrenia Proneness Instrument, Structured Interview for Prodromal Symptoms and structural MRI, analyzed by voxel-based morphometry.

RESULTS

(i) Patients and HC could be distinguished by their CT pattern with a reasonable precision [balanced accuracy of 71.2% (sensitivity = 72.1%, specificity = 70.4%, p ≤ 0.001]. (ii) Subdomains 'emotional neglect' and 'emotional abuse' were most predictive for CHR and ROP, while in ROD 'physical abuse' and 'sexual abuse' were most important. The CT pattern was significantly associated with the severity of depressive symptoms in ROD, ROP, and CHR, as well as with the PANSS total and negative domain scores in the CHR patients. No associations between group-separating CT patterns and brain structure were found.

CONCLUSIONS

These results indicate that CT poses a transdiagnostic risk factor for mental health disorders, possibly related to depressive symptoms. While differences in the quality of CT exposure exist, diagnostic differentiation was not possible suggesting a multi-factorial pathogenesis.

Attentional capture by fearful faces requires consciousness and is modulated by task-relevancy: A dot-probe EEG study

In the current EEG study, we used a dot-probe task in conjunction with backward masking to examine the neural activity underlying awareness and spatial processing of fearful faces and the neural processes for subsequent cued spatial targets. We presented face images under different viewing conditions (subliminal and supraliminal) and manipulated the relation between a fearful face in the pair and a subsequent target. Our mass univariate analysis showed that fearful faces elicit the N2-posterior-contralateral, indexing spatial attention capture, only when they are presented supraliminally. Consistent with this, the multivariate pattern analysis revealed a successful decoding of the location of the fearful face only in the supraliminal viewing condition. Additionally, the spatial attention capture by fearful faces modulated the processing of subsequent lateralised targets that were spatially congruent with the fearful face, in both al and electrophysiological data. There was no evidence for nonconscious processing of the fearful faces in the current paradigm. We conclude that spatial attentional capture by fearful faces requires visual awareness and it is modulated by top-down task demands.

Prediction of individual trigeminal pain sensitivity from gray matter structure within the sensorimotor network

OBJECTIVE

To determine whether multivariate pattern regression analysis based on gray matter (GM) images constrained to the sensorimotor network could accurately predict trigeminal heat pain sensitivity in healthy individuals.

BACKGROUND

Prediction of individual pain sensitivity is of clinical relevance as high pain sensitivity is associated with increased risks of postoperative pain, pain chronification, and a poor treatment response. However, as pain is a subjective experience accurate identification of such individuals can be difficult. GM structure of sensorimotor regions have been shown to vary with pain sensitivity. It is unclear whether GM structure within these regions can be used to predict pain sensitivity.

METHODS

In this cross-sectional study, structural magnetic resonance images and pain thresholds in response to contact heat stimulation of the left supraorbital area were obtained from 79 healthy participants. Voxel-based morphometry was used to extract segmented and normalized GM images. These were then constrained to a mask encompassing the functionally defined resting-state sensorimotor network. The masked images and pain thresholds entered a multivariate relevance vector regression analysis for quantitative prediction of the individual pain thresholds. The correspondence between predicted and actual pain thresholds was indexed by the Pearson correlation coefficient (r) and the mean squared error (MSE). The generalizability of the model was assessed by 10-fold and 5-fold cross-validation. Non-parametric permutation tests were used to estimate significance levels.

RESULTS

Trigeminal heat pain sensitivity could be predicted from GM structure within the sensorimotor network with significant accuracy (10-fold: r = 0.53, p < 0.001, MSE = 10.32, p = 0.001; 5-fold: r = 0.46, p = 0.001, MSE = 10.54, p < 0.001). The resulting multivariate weight maps revealed that accurate prediction relied on multiple widespread regions within the sensorimotor network.

CONCLUSION

A multivariate pattern of GM structure within the sensorimotor network could be used to make accurate predictions about trigeminal heat pain sensitivity at the individual level in healthy participants. Widespread regions within the sensorimotor network contributed to the predictive model.

Decoding subject's own name in the primary auditory cortex

Current studies have shown that perception of subject's own name (SON) involves multiple multimodal brain regions, while activities in unimodal sensory regions (i.e., primary auditory cortex) and their interaction with multimodal regions during the self-processing remain unclear. To answer this, we combined multivariate pattern analysis and dynamic causal modelling analysis to explore the regional activation pattern and inter-region effective connection during the perception of SON. We found that SON and other names could be decoded from the activation pattern in the primary auditory cortex. In addition, we found an excitatory effect of SON on connections from the anterior insula/inferior frontal gyrus to the primary auditory cortex, and to the temporoparietal junction. Our findings extended the current knowledge of self-processing by showing that primary auditory cortex could discriminate SON from other names. Furthermore, our findings highlighted the importance of influence of the insula on the primary auditory cortex during self-processing.

Dynamic changes in fractional amplitude of low-frequency fluctuations in patients with chronic insomnia

BACKGROUND

Previous neuroimaging studies have mostly focused on changes in static functional connectivity in patients with chronic insomnia (CI) . Features of dynamic brain activity in patients with CI have rarely been described in detail. The present study investigated changes in dynamic intrinsic brain activity in patients with CI by dynamic fractional amplitude of low-frequency fluctuation (dfALFF) analysis.

MATERIALS AND METHODS

A total of 30 patients with CI and 27 healthy controls (HCs) were enrolled. We compared dfALFF between these two groups, and examined the correlation between changes in dfALFF and clinical symptoms of CI. Multivariate pattern analysis was performed to differentiate patients with CI from HCs.

RESULTS

Compared with HC subjects, patients with CI showed significantly increased dfALFF in the left insula, right superior temporal gyrus, left parahippocampal gyrus, right amygdala, and bilateral posterior lobes of the cerebellum. Moreover, dfALFF values in the left insula and left parahippocampal gyrus showed a positive correlation with Pittsburgh Sleep Quality Index scores. A logistic regression model was constructed that had 96.7% sensitivity, 80.0% specificity, and 83.0% overall accuracy for distinguishing patients with CI from HCs.

CONCLUSION

Dynamic local brain activity showed increased instability in patients with CI. The variability in dfALFF in the limbic system and brain areas related to sleep/wakefulness was associated with insomnia symptoms. These findings may provide insight into the neuropathologic basis of CI.

Decoding continuous variables from event-related potential (ERP) data with linear support vector regression using the Decision Decoding Toolbox (DDTBOX)

Multivariate classification analysis for event-related potential (ERP) data is a powerful tool for predicting cognitive variables. However, classification is often restricted to categorical variables and under-utilises continuous data, such as response times, response force, or subjective ratings. An alternative approach is support vector regression (SVR), which uses single-trial data to predict continuous variables of interest. In this tutorial-style paper, we demonstrate how SVR is implemented in the Decision Decoding Toolbox (DDTBOX). To illustrate in more detail how results depend on specific toolbox settings and data features, we report results from two simulation studies resembling real EEG data, and one real ERP-data set, in which we predicted continuous variables across a range of analysis parameters. Across all studies, we demonstrate that SVR is effective for analysis windows ranging from 2 to 100 ms, and relatively unaffected by temporal averaging. Prediction is still successful when only a small number of channels encode true information, and the analysis is robust to temporal jittering of the relevant information in the signal. Our results show that SVR as implemented in DDTBOX can reliably predict continuous, more nuanced variables, which may not be well-captured by classification analysis. In sum, we demonstrate that linear SVR is a powerful tool for the investigation of single-trial EEG data in relation to continuous variables, and we provide practical guidance for users.

Predictive models for social functioning in healthy young adults: A machine learning study integrating neuroanatomical, cognitive, and behavioral data

Poor social functioning is an emerging public health problem associated with physical and mental health consequences. Developing prognostic tools is critical to identify individuals at risk for poor social functioning and guide interventions. We aimed to inform prediction models of social functioning by evaluating models relying on bio-behavioral data using machine learning. With data from the Human Connectome Project Healthy Young Adult sample (age 22-35, N = 1,101), we built Support Vector Regression models to estimate social functioning from variable sets of brain morphology to behavior with increasing complexity: 1) brain-only model, 2) brain-cognition model, 3) cognition-behavioral model, and 4) combined brain-cognition-behavioral model. Predictive accuracy of each model was assessed and the importance of individual variables for model performance was determined. The combined and cognition-behavioral models significantly predicted social functioning, whereas the brain-only and brain-cognition models did not. Negative affect, psychological wellbeing, extraversion, withdrawal, and cortical thickness of the rostral middle-frontal and superior-temporal regions were the most important predictors in the combined model. Results demonstrate that social functioning can be accurately predicted using machine learning methods. Behavioral markers may be more significant predictors of social functioning than brain measures for healthy young adults and may represent important leverage points for preventative intervention.

The role of ascending arousal network in patients with chronic insomnia disorder

The ascending arousal system plays a crucial role in individuals' consciousness. Recently, advanced functional magnetic resonance imaging (fMRI) has made it possible to investigate the ascending arousal network (AAN) in vivo. However, the role of AAN in the neuropathology of human insomnia remains unclear. Our study aimed to explore alterations in AAN and its connections with cortical networks in chronic insomnia disorder (CID). Resting-state fMRI data were acquired from 60 patients with CID and 60 good sleeper controls (GSCs). Changes in the brain's functional connectivity (FC) between the AAN and eight cortical networks were detected in patients with CID and GSCs. Multivariate pattern analysis (MVPA) was employed to differentiate CID patients from GSCs and predict clinical symptoms in patients with CID. Finally, these MVPA findings were further verified using an external data set (32 patients with CID and 33 GSCs). Compared to GSCs, patients with CID exhibited increased FC within the AAN, as well as increased FC between the AAN and default mode, cerebellar, sensorimotor, and dorsal attention networks. These AAN-related FC patterns and the MVPA classification model could be used to differentiate CID patients from GSCs with 88% accuracy in the first cohort and 77% accuracy in the validation cohort. Moreover, the MVPA prediction models could separately predict insomnia (data set 1, R²  = .34; data set 2, R²  = .15) and anxiety symptoms (data set 1, R²  = .35; data set 2, R²  = .34) in the two independent cohorts of patients. Our findings indicated that AAN contributed to the neurobiological mechanism of insomnia and highlighted that fMRI-based markers and machine learning techniques might facilitate the evaluation of insomnia and its comorbid mental symptoms.

Decoding the Specificity of Post-Error Adjustments Using EEG-Based Multivariate Pattern Analysis

Errors can elicit post-error adjustments that serve to optimize performance by preventing further errors. An essential but unsolved issue is that whether post-error adjustments are domain-general or domain-specific, which was investigated in the present study through eliciting different types of errors. Behavioral and electrophysiological data were recorded when male and female subjects performed the Eriksen flanker task. For this study, we examined the aforementioned issue by combining event-related potential and multivariate pattern analysis. The results indicated that post-error slowing, error-related negativity, and error positivity were comparable between congruent and incongruent errors, indicating that errors triggered domain-general interference mechanisms, whereas post-error accuracy and late positive potential elicited by incongruent errors were larger than those elicited by congruent errors, exhibiting domain-specific control adjustment mechanisms. Importantly, no successful decoding soon after errors was found between congruent and incongruent errors, but above-chance decoding was observed between these two types of errors with increasing time, which further support that domain-general adjustments occurred in the early stage, whereas domain-specific adjustments appeared in the late stage. Furthermore, brain-behavior correlation results suggested that the late post-error adjustments predicted subsequent behavior performance. Together, this study revealed that early domain-general interference adjustments induced by errors are reflected in error detection and error awareness, which are independent of error types; on the contrary, late domain-specific control adjustments are reflected in attentional adjustments, which are modulated by error types.SIGNIFICANCE STATEMENT To date, clear evidence on the specificity of post-error adjustments is lacking. The present study provides neurophysiological evidence that post-error adjustments simultaneously rely on both domain-general and domain-specific mechanisms. Event-related potential results indicated that domain-general adjustments were accompanied by the interference of error detection and error awareness. In contrast, domain-specific adjustments were associated with attentional adjustments. Multivariate pattern analysis further decoded the two features of post-error adjustments in the early stage matching the time patterns of error-related negativity and error positivity and in the late stage corresponding to the late positive potential. Temporal generalization analysis showed that domain-specific processing appeared stably in late post-error adjustments. Hence, we propose that post-error different stages may determine the specificity of post-error adjustments.

Semantic Scene-Object Consistency Modulates N300/400 EEG Components, but Does Not Automatically Facilitate Object Representations

During natural vision, objects rarely appear in isolation, but often within a semantically related scene context. Previous studies reported that semantic consistency between objects and scenes facilitates object perception and that scene-object consistency is reflected in changes in the N300 and N400 components in EEG recordings. Here, we investigate whether these N300/400 differences are indicative of changes in the cortical representation of objects. In two experiments, we recorded EEG signals, while participants viewed semantically consistent or inconsistent objects within a scene; in Experiment 1, these objects were task-irrelevant, while in Experiment 2, they were directly relevant for behavior. In both experiments, we found reliable and comparable N300/400 differences between consistent and inconsistent scene-object combinations. To probe the quality of object representations, we performed multivariate classification analyses, in which we decoded the category of the objects contained in the scene. In Experiment 1, in which the objects were not task-relevant, object category could be decoded from ~100 ms after the object presentation, but no difference in decoding performance was found between consistent and inconsistent objects. In contrast, when the objects were task-relevant in Experiment 2, we found enhanced decoding of semantically consistent, compared with semantically inconsistent, objects. These results show that differences in N300/400 components related to scene-object consistency do not index changes in cortical object representations but rather reflect a generic marker of semantic violations. Furthermore, our findings suggest that facilitatory effects between objects and scenes are task-dependent rather than automatic.

Decoding the temporal dynamics of affective scene processing

Natural images containing affective scenes are used extensively to investigate the neural mechanisms of visual emotion processing. Functional fMRI studies have shown that these images activate a large-scale distributed brain network that encompasses areas in visual, temporal, and frontal cortices. The underlying spatial and temporal dynamics, however, remain to be better characterized. We recorded simultaneous EEG-fMRI data while participants passively viewed affective images from the International Affective Picture System (IAPS). Applying multivariate pattern analysis to decode EEG data, and representational similarity analysis to fuse EEG data with simultaneously recorded fMRI data, we found that: (1) ∼80 ms after picture onset, perceptual processing of complex visual scenes began in early visual cortex, proceeding to ventral visual cortex at ∼100 ms, (2) between ∼200 and ∼300 ms (pleasant pictures: ∼200 ms; unpleasant pictures: ∼260 ms), affect-specific neural representations began to form, supported mainly by areas in occipital and temporal cortices, and (3) affect-specific neural representations were stable, lasting up to ∼2 s, and exhibited temporally generalizable activity patterns. These results suggest that affective scene representations in the brain are formed temporally in a valence-dependent manner and may be sustained by recurrent neural interactions among distributed brain areas.

Decoding violated sensory expectations from the auditory cortex of anaesthetised mice: Hierarchical recurrent neural network depicts separate 'danger' and 'safety' units

The ability to respond appropriately to sensory information received from the external environment is among the most fundamental capabilities of central nervous systems. In the auditory domain, processes underlying this behaviour are studied by measuring auditory‐evoked electrophysiology during sequences of sounds with predetermined regularities. Identifying neural correlates of ensuing auditory novelty responses is supported by research in experimental animals. In the present study, we reanalysed epidural field potential recordings from the auditory cortex of anaesthetised mice during frequency and intensity oddball stimulation. Multivariate pattern analysis (MVPA) and hierarchical recurrent neural network (RNN) modelling were adopted to explore these data with greater resolution than previously considered using conventional methods. Time‐wise and generalised temporal decoding MVPA approaches revealed previously underestimated asymmetry between responses to sound‐level transitions in the intensity oddball paradigm, in contrast with tone frequency changes. After training, the cross‐validated RNN model architecture with four hidden layers produced output waveforms in response to simulated auditory inputs that were strongly correlated with grand‐average auditory‐evoked potential waveforms (r² > .9). Units in hidden layers were classified based on their temporal response properties and characterised using principal component analysis and sample entropy. These demonstrated spontaneous alpha rhythms, sound onset and offset responses and putative ‘safety’ and ‘danger’ units activated by relatively inconspicuous and salient changes in auditory inputs, respectively. The hypothesised existence of corresponding biological neural sources is naturally derived from this model. If proven, this could have significant implications for prevailing theories of auditory processing.

A resting-state fMRI study of temporal lobe epilepsy using multivariate pattern analysis and Granger causality analysis

BACKGROUND AND PURPOSE

Understanding the pathogenesis of temporal lobe epilepsy (TLE) is essential for its diagnosis and treatment. The study aimed to explore regional homogeneity (ReHo) and changes in effective connectivity (EC) between brain regions in TLE patients, hoping to discover potential abnormalities in certain brain regions in TLE patients.

METHODS

Resting-state functional magnetic resonance data were collected from 23 TLE patients and 32 normal controls (NC). ReHo was used as a feature of multivariate pattern analysis (MVPA) to explore the ability of its alterations in identifying TLE. Based on the results of the MVPA, certain brain regions were selected as seed points to further explore alterations in EC between brain regions using Granger causality analysis.

RESULTS

MVPA results showed that the classification accuracy for the TLE and NC groups was 87.27%, and the right posterior cerebellum lobe, right lingual gyrus (LING_R), right cuneus (CUN_R), and left superior temporal gyrus (STG_L) provided significant contributions. Moreover, the EC from STG_L to right fusiform gyrus (FFG_R) and LING_R and the EC from CUN_R to the right occipital superior gyrus (SOG_R) and right occipital middle gyrus (MOG_R) were altered compared to the NC group.

CONCLUSION

The MVPA results indicated that ReHo abnormalities in brain regions may be an important feature in the identification of TLE. The enhanced EC from STG_L to FFG_R and LING_R indicates a shift in language processing to the right hemisphere, and the weakened EC from SOG_R and MOG_R to CUN_R may reveal an underlying mechanism of TLE.

Feature-reweighted representational similarity analysis: A method for improving the fit between computational models, brains, and behavior

Representational Similarity Analysis (RSA) has emerged as a popular method for relating representational spaces from human brain activity, behavioral data, and computational models. RSA is based on the comparison of representational (dis-)similarity matrices (RDM or RSM), which characterize the pairwise (dis-)similarities of all conditions across all features (e.g. fMRI voxels or units of a model). However, classical RSA treats each feature as equally important. This 'equal weights' assumption contrasts with the flexibility of multivariate decoding, which reweights individual features for predicting a target variable. As a consequence, classical RSA may lead researchers to underestimate the correspondence between a model and a brain region and, in case of model comparison, may lead them to select an inferior model. The aim of this work is twofold: First, we sought to broadly test feature-reweighted RSA (FR-RSA) applied to computational models and reveal the extent to which reweighting model features improves RSM correspondence and affects model selection. Previous work suggested that reweighting can improve model selection in RSA but it has remained unclear to what extent these results generalize across datasets and data modalities. To draw more general conclusions, we utilized a range of publicly available datasets and three popular deep neural networks (DNNs). Second, we propose voxel-reweighted RSA, a novel use case of FR-RSA that reweights fMRI voxels, mirroring the rationale of multivariate decoding of optimally combining voxel activity patterns. We found that reweighting individual model units markedly improved the fit between model RSMs and target RSMs derived from several fMRI and behavioral datasets and affected model selection, highlighting the importance of considering FR-RSA. For voxel-reweighted RSA, improvements in RSM correspondence were even more pronounced, demonstrating the utility of this novel approach. We additionally show that classical noise ceilings can be exceeded when FR-RSA is applied and propose an updated approach for their computation. Taken together, our results broadly validate the use of FR-RSA for improving the fit between computational models, brain, and behavioral data, possibly allowing us to better adjudicate between competing computational models. Further, our results suggest that FR-RSA applied to brain measurement channels could become an important new method to assess the correspondence between representational spaces.