A hierarchy of responses to auditory regularities in the macaque brain

Erroneous predictive coding across brain hierarchies in a non-human primate model of autism spectrum disorder

In autism spectrum disorder (ASD), atypical sensory experiences are often associated with irregularities in predictive coding, which proposes that the brain creates hierarchical sensory models via a bidirectional process of predictions and prediction errors. However, it remains unclear how these irregularities manifest across different functional hierarchies in the brain. To address this, we study a marmoset model of ASD induced by valproic acid (VPA) treatment. We record high-density electrocorticography (ECoG) during an auditory task with two layers of temporal control, and applied a quantitative model to quantify the integrity of predictive coding across two distinct hierarchies. Our results demonstrate a persistent pattern of sensory hypersensitivity and unstable predictions across two brain hierarchies in VPA-treated animals, and reveal the associated spatio-spectro-temporal neural signatures. Despite the regular occurrence of imprecise predictions in VPA-treated animals, we observe diverse configurations of underestimation or overestimation of sensory regularities within the hierarchies. Our results demonstrate the coexistence of the two primary Bayesian accounts of ASD: overly-precise sensory observations and weak prior beliefs, and offer a potential multi-layered biomarker for ASD, which could enhance our understanding of its diverse symptoms.

Perceptual Awareness in Human Infants: What is the Evidence?

Perceptual awareness in infants during the first year of life is understudied, despite the philosophical, scientific, and clinical importance of understanding how and when consciousness emerges during human brain development. Although parents are undoubtedly convinced that their infant is conscious, the lack of adequate experimental paradigms to address this question in preverbal infants has been a hindrance to research on this topic. However, recent behavioral and brain imaging studies have shown that infants are engaged in complex learning from an early age and that their brains are more structured than traditionally thought. I will present a rapid overview of these results, which might provide indirect evidence of early perceptual awareness and then describe how a more systematic approach to this question could stand within the framework of global workspace theory, which identifies specific signatures of conscious perception in adults. Relying on these brain signatures as a benchmark for conscious perception, we can deduce that it exists in the second half of the first year, whereas the evidence before the age of 5 months is less solid, mainly because of the paucity of studies. The question of conscious perception before term remains open, with the possibility of short periods of conscious perception, which would facilitate early learning. Advances in brain imaging and growing interest in this subject should enable us to gain a better understanding of this important issue in the years to come.

Chronic pain - A maladaptive compensation to unbalanced hierarchical predictive processing

The ability to perceive pain presents an interesting evolutionary advantage to adapt to an ever-changing environment. However, in the case of chronic pain (CP), pain perception hinders the capacity of the system to adapt to changing sensory environments. Similar to other chronic perceptual disorders, CP is also proposed to be a maladaptive compensation to aberrant sensory predictive processing. The local-global oddball paradigm relies on learning hierarchical rules and processing environmental irregularities at a local and global level. Prediction errors (PE) between actual and predicted input typically trigger an update of the forward model to limit the probability of encountering future PEs. It has been hypothesised that CP hinders forward model updating, reflected in increased local deviance and decreased global deviance. In the present study, we used the local-global paradigm to examine how CP influences hierarchical learning relative to healthy controls. As hypothesised, we observed that deviance in the stimulus characteristics evoked heightened local deviance and decreased global deviance of the stimulus-driven PE. This is also accompanied by respective changes in theta phase locking that is correlated with the subjective pain perception. Changes in the global deviant in the stimulus-driven-PE could also be explained by dampened attention-related responses. Changing the context of the auditory stimulus did not however show a difference in the context-driven PE. These findings suggest that CP is accompanied by maladaptive forward model updating where the constant presence of pain perception disrupts local deviance in non-nociceptive domains. Furthermore, we hypothesise that the auditory-processing based biomarker identified here could be a marker of domain-general dysfunction that could be confirmed by future research.

Transient brain activity dynamics discriminate levels of consciousness during anesthesia

The awake mammalian brain is functionally organized in terms of large-scale distributed networks that are constantly interacting. Loss of consciousness might disrupt this temporal organization leaving patients unresponsive. We hypothesize that characterizing brain activity in terms of transient events may provide a signature of consciousness. For this, we analyze temporal dynamics of spatiotemporally overlapping functional networks obtained from fMRI transient activity across different anesthetics and levels of anesthesia. We first show a striking homology in spatial organization of networks between monkeys and humans, indicating cross-species similarities in resting-state fMRI structure. We then track how network organization shifts under different anesthesia conditions in macaque monkeys. While the spatial aspect of the networks is preserved, their temporal dynamics are highly affected by anesthesia. Networks express for longer durations and co-activate in an anesthetic-specific configuration. Additionally, hierarchical brain organization is disrupted with a consciousness-level-signature role of the default mode network. In conclusion, large-scale brain network temporal dynamics capture differences in anesthetic-specific consciousness-level, paving the way towards a clinical translation of these cortical signature.

An integrative view of the role of prefrontal cortex in consciousness

The involvement of the prefrontal cortex (PFC) in consciousness is an ongoing focus of intense investigation. An important question is whether representations of conscious contents and experiences in the PFC are confounded by post-perceptual processes related to cognitive functions. Here, I review recent findings suggesting that neuronal representations of consciously perceived contents-in the absence of post-perceptual processes-can indeed be observed in the PFC. Slower ongoing fluctuations in the electrophysiological state of the PFC seem to control the stability and updates of these prefrontal representations of conscious awareness. In addition to conscious perception, the PFC has been shown to play a critical role in controlling the levels of consciousness as observed during anesthesia, while prefrontal lesions can result in severe loss of perceptual awareness. Together, the convergence of these processes in the PFC suggests its integrative role in consciousness and highlights the complex nature of consciousness itself.

Spontaneously emerging internal models of visual sequences combine abstract and event-specific information in the prefrontal cortex

When exposed to sensory sequences, do macaque monkeys spontaneously form abstract internal models that generalize to novel experiences? Here, we show that neuronal populations in macaque ventrolateral prefrontal cortex jointly encode visual sequences by separate codes for the specific pictures presented and for their abstract sequential structure. We recorded prefrontal neurons while macaque monkeys passively viewed visual sequences and sequence mismatches in the local-global paradigm. Even without any overt task or response requirements, prefrontal populations spontaneously form representations of sequence structure, serial order, and image identity within distinct but superimposed neuronal subspaces. Representations of sequence structure rapidly update following single exposure to a mismatch sequence, while distinct populations represent mismatches for sequences of different complexity. Finally, those representations generalize across sequences following the same repetition structure but comprising different images. These results suggest that prefrontal populations spontaneously encode rich internal models of visual sequences reflecting both content-specific and abstract information.

Local orchestration of distributed functional patterns supporting loss and restoration of consciousness in the primate brain

A central challenge of neuroscience is to elucidate how brain function supports consciousness. Here, we combine the specificity of focal deep brain stimulation with fMRI coverage of the entire cortex, in awake and anaesthetised non-human primates. During propofol, sevoflurane, or ketamine anaesthesia, and subsequent restoration of responsiveness by electrical stimulation of the central thalamus, we investigate how loss of consciousness impacts distributed patterns of structure-function organisation across scales. We report that distributed brain activity under anaesthesia is increasingly constrained by brain structure across scales, coinciding with anaesthetic-induced collapse of multiple dimensions of hierarchical cortical organisation. These distributed signatures are observed across different anaesthetics, and they are reversed by electrical stimulation of the central thalamus, coinciding with recovery of behavioural markers of arousal. No such effects were observed upon stimulating the ventral lateral thalamus, demonstrating specificity. Overall, we identify consistent distributed signatures of consciousness that are orchestrated by specific thalamic nuclei.

Altered hierarchical auditory predictive processing after lesions to the orbitofrontal cortex

Orbitofrontal cortex (OFC) is classically linked to inhibitory control, emotion regulation, and reward processing. Recent perspectives propose that the OFC also generates predictions about perceptual events, actions, and their outcomes. We tested the role of the OFC in detecting violations of prediction at two levels of abstraction (i.e., hierarchical predictive processing) by studying the event-related potentials (ERPs) of patients with focal OFC lesions (n = 12) and healthy controls (n = 14) while they detected deviant sequences of tones in a local-global paradigm. The structural regularities of the tones were controlled at two hierarchical levels by rules defined at a local (i.e., between tones within sequences) and at a global (i.e., between sequences) level. In OFC patients, ERPs elicited by standard tones were unaffected at both local and global levels compared to controls. However, patients showed an attenuated mismatch negativity (MMN) and P3a to local prediction violation, as well as a diminished MMN followed by a delayed P3a to the combined local and global level prediction violation. The subsequent P3b component to conditions involving violations of prediction at the level of global rules was preserved in the OFC group. Comparable effects were absent in patients with lesions restricted to the lateral PFC, which lends a degree of anatomical specificity to the altered predictive processing resulting from OFC lesion. Overall, the altered magnitudes and time courses of MMN/P3a responses after lesions to the OFC indicate that the neural correlates of detection of auditory regularity violation are impacted at two hierarchical levels of rule abstraction.

The empirical status of predictive coding and active inference

Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.

Multiple Concurrent Predictions Inform Prediction Error in the Human Auditory Pathway

The key assumption of the predictive coding framework is that internal representations are used to generate predictions on how the sensory input will look like in the immediate future. These predictions are tested against the actual input by the so-called prediction error units, which encode the residuals of the predictions. What happens to prediction errors, however, if predictions drawn by different stages of the sensory hierarchy contradict each other? To answer this question, we conducted two fMRI experiments while female and male human participants listened to sequences of sounds: pure tones in the first experiment and frequency-modulated sweeps in the second experiment. In both experiments, we used repetition to induce predictions based on stimulus statistics (stats-informed predictions) and abstract rules disclosed in the task instructions to induce an orthogonal set of (task-informed) predictions. We tested three alternative scenarios: neural responses in the auditory sensory pathway encode prediction error with respect to (1) the stats-informed predictions, (2) the task-informed predictions, or (3) a combination of both. Results showed that neural populations in all recorded regions (bilateral inferior colliculus, medial geniculate body, and primary and secondary auditory cortices) encode prediction error with respect to a combination of the two orthogonal sets of predictions. The findings suggest that predictive coding exploits the non-linear architecture of the auditory pathway for the transmission of predictions. Such non-linear transmission of predictions might be crucial for the predictive coding of complex auditory signals like speech.Significance Statement Sensory systems exploit our subjective expectations to make sense of an overwhelming influx of sensory signals. It is still unclear how expectations at each stage of the processing pipeline are used to predict the representations at the other stages. The current view is that this transmission is hierarchical and linear. Here we measured fMRI responses in auditory cortex, sensory thalamus, and midbrain while we induced two sets of mutually inconsistent expectations on the sensory input, each putatively encoded at a different stage. We show that responses at all stages are concurrently shaped by both sets of expectations. The results challenge the hypothesis that expectations are transmitted linearly and provide for a normative explanation of the non-linear physiology of the corticofugal sensory system.

A multimodal cortical network of sensory expectation violation revealed by fMRI

The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities.

Brainstem fMRI signaling of surprise across different types of deviant stimuli

Detection of deviant stimuli is crucial to orient and adapt our behavior. Previous work shows that deviant stimuli elicit phasic activation of the locus coeruleus (LC), which releases noradrenaline and controls central arousal. However, it is unclear whether the detection of behaviorally relevant deviant stimuli selectively triggers LC responses or other neuromodulatory systems (dopamine, serotonin, and acetylcholine). We combine human functional MRI (fMRI) recordings optimized for brainstem imaging with pupillometry to perform a mapping of deviant-related responses in subcortical structures. Participants have to detect deviant items in a "local-global" paradigm that distinguishes between deviance based on the stimulus probability and the sequence structure. fMRI responses to deviant stimuli are distributed in many cortical areas. Both types of deviance elicit responses in the pupil, LC, and other neuromodulatory systems. Our results reveal that the detection of task-relevant deviant items recruits the same multiple subcortical systems across computationally different types of deviance.

Brain-imaging evidence for compression of binary sound sequences in human memory

According to the language-of-thought hypothesis, regular sequences are compressed in human memory using recursive loops akin to a mental program that predicts future items. We tested this theory by probing memory for 16-item sequences made of two sounds. We recorded brain activity with functional MRI and magneto-encephalography (MEG) while participants listened to a hierarchy of sequences of variable complexity, whose minimal description required transition probabilities, chunking, or nested structures. Occasional deviant sounds probed the participants' knowledge of the sequence. We predicted that task difficulty and brain activity would be proportional to the complexity derived from the minimal description length in our formal language. Furthermore, activity should increase with complexity for learned sequences, and decrease with complexity for deviants. These predictions were upheld in both fMRI and MEG, indicating that sequence predictions are highly dependent on sequence structure and become weaker and delayed as complexity increases. The proposed language recruited bilateral superior temporal, precentral, anterior intraparietal, and cerebellar cortices. These regions overlapped extensively with a localizer for mathematical calculation, and much less with spoken or written language processing. We propose that these areas collectively encode regular sequences as repetitions with variations and their recursive composition into nested structures.

Monkey Dorsolateral Prefrontal Cortex Represents Abstract Visual Sequences during a No-Report Task

Monitoring sequential information is an essential component of our daily lives. Many of these sequences are abstract, in that they do not depend on the individual stimuli, but do depend on an ordered set of rules (e.g., chop then stir when cooking). Despite the ubiquity and utility of abstract sequential monitoring, little is known about its neural mechanisms. Human rostrolateral prefrontal cortex (RLPFC) exhibits specific increases in neural activity (i.e., "ramping") during abstract sequences. Monkey dorsolateral prefrontal cortex (DLPFC) has been shown to represent sequential information in motor (not abstract) sequence tasks, and contains a subregion, area 46, with homologous functional connectivity to human RLPFC. To test the prediction that area 46 may represent abstract sequence information, and do so with parallel dynamics to those found in humans, we conducted functional magnetic resonance imaging (fMRI) in three male monkeys. When monkeys performed no-report abstract sequence viewing, we found that left and right area 46 responded to abstract sequential changes. Interestingly, responses to rule and number changes overlapped in right area 46 and left area 46 exhibited responses to abstract sequence rules with changes in ramping activation, similar to that observed in humans. Together, these results indicate that monkey DLPFC monitors abstract visual sequential information, potentially with a preference for different dynamics in the two hemispheres. More generally, these results show that abstract sequences are represented in functionally homologous regions across monkeys and humans.SIGNIFICANCE STATEMENT Daily, we complete sequences that are "abstract" because they depend on an ordered set of rules (e.g., chop then stir when cooking) rather than the identity of individual items. Little is known about how the brain tracks, or monitors, this abstract sequential information. Based on previous human work showing abstract sequence related dynamics in an analogous area, we tested whether monkey dorsolateral prefrontal cortex (DLPFC), specifically area 46, represents abstract sequential information using awake monkey functional magnetic resonance imaging (fMRI). We found that area 46 responded to abstract sequence changes, with a preference for more general responses on the right and dynamics similar to humans on the left. These results suggest that abstract sequences are represented in functionally homologous regions across monkeys and humans.

Temporal hierarchies in the predictive processing of melody - From pure tones to songs

Listening to musical melodies is a complex task that engages perceptual and memoryrelated processes. The processes underlying melody cognition happen simultaneously on different timescales, ranging from milliseconds to minutes. Although attempts have been made, research on melody perception is yet to produce a unified framework of how melody processing is achieved in the brain. This may in part be due to the difficulty of integrating concepts such as perception, attention and memory, which pertain to different temporal scales. Recent theories on brain processing, which hold prediction as a fundamental principle, offer potential solutions to this problem and may provide a unifying framework for explaining the neural processes that enable melody perception on multiple temporal levels. In this article, we review empirical evidence for predictive coding on the levels of pitch formation, basic pitch-related auditory patterns,more complex regularity processing extracted from basic patterns and long-term expectations related to musical syntax. We also identify areas that would benefit from further inquiry and suggest future directions in research on musical melody perception.

Hierarchical Deviant Processing in Auditory Cortex of Awake Mice

Detecting patterns, and noticing unexpected pattern changes, in the environment is a vital aspect of sensory processing. Adaptation and prediction error responses are two components of neural processing related to these tasks, and previous studies in the auditory system in rodents show that these two components are partially dissociable in terms of the topography and latency of neural responses to sensory deviants. However, many previous studies have focused on repetitions of single stimuli, such as pure tones, which have limited ecological validity. In this study, we tested whether the auditory cortical activity shows adaptation to repetition of more complex sound patterns (bisyllabic pairs). Specifically, we compared neural responses to violations of sequences based on single stimulus probability only, against responses to more complex violations based on stimulus order. We employed an auditory oddball paradigm and monitored the auditory cortex (ACtx) activity of awake mice (N=8) using wide-field calcium imaging. We found that cortical responses were sensitive both to single stimulus probabilities and to more global stimulus patterns, as mismatch signals were elicited following both substitution deviants and transposition deviants. Notably, A2 area elicited larger mismatch signaling to those deviants than primary ACtx (A1), which suggests a hierarchical gradient of prediction error signaling in the auditory cortex. Such a hierarchical gradient was observed for late but not early peaks of calcium transients to deviants, suggesting that the late part of the deviant response may reflect prediction error signaling in response to more complex sensory pattern violations.

Sensory Target Detection at Local and Global Timescales Reveals a Hierarchy of Supramodal Dynamics in the Human Cortex

To ensure survival in a dynamic environment, the human neocortex monitors input streams from different sensory organs for important sensory events. Which principles govern whether different senses share common or modality-specific brain networks for sensory target detection? We examined whether complex targets evoke sustained supramodal activity while simple targets rely on modality-specific networks with short-lived supramodal contributions. In a series of hierarchical multisensory target detection studies (n = 77, of either sex) using EEG, we applied a temporal cross-decoding approach to dissociate supramodal and modality-specific cortical dynamics elicited by rule-based global and feature-based local sensory deviations within and between the visual, somatosensory, and auditory modality. Our data show that each sense implements a cortical hierarchy orchestrating supramodal target detection responses, which operate at local and global timescales in successive processing stages. Across different sensory modalities, simple feature-based sensory deviations presented in temporal vicinity to a monotonous input stream triggered a mismatch negativity-like local signal which decayed quickly and early, whereas complex rule-based targets tracked across time evoked a P3b-like global neural response which generalized across a late time window. Converging results from temporal cross-modality decoding analyses across different datasets, we reveal that global neural responses are sustained in a supramodal higher-order network, whereas local neural responses canonically thought to rely on modality-specific regions evolve into short-lived supramodal activity. Together, our findings demonstrate that cortical organization largely follows a gradient in which short-lived modality-specific as well as supramodal processes dominate local responses, whereas higher-order processes encode temporally extended abstract supramodal information fed forward from modality-specific cortices.SIGNIFICANCE STATEMENT Each sense supports a cortical hierarchy of processes tracking deviant sensory events at multiple timescales. Conflicting evidence produced a lively debate around which of these processes are supramodal. Here, we manipulated the temporal complexity of auditory, tactile, and visual targets to determine whether cortical local and global ERP responses to sensory targets share cortical dynamics between the senses. Using temporal cross-decoding, we found that temporally complex targets elicit a supramodal sustained response. Conversely, local responses to temporally confined targets typically considered modality-specific rely on early short-lived supramodal activation. Our finding provides evidence for a supramodal gradient supporting sensory target detection in the cortex, with implications for multiple fields in which these responses are studied (e.g., predictive coding, consciousness, and attention).

A quantitative model reveals a frequency ordering of prediction and prediction-error signals in the human brain

The human brain is proposed to harbor a hierarchical predictive coding neuronal network underlying perception, cognition, and action. In support of this theory, feedforward signals for prediction error have been reported. However, the identification of feedback prediction signals has been elusive due to their causal entanglement with prediction-error signals. Here, we use a quantitative model to decompose these signals in electroencephalography during an auditory task, and identify their spatio-spectral-temporal signatures across two functional hierarchies. Two prediction signals are identified in the period prior to the sensory input: a low-level signal representing the tone-to-tone transition in the high beta frequency band, and a high-level signal for the multi-tone sequence structure in the low beta band. Subsequently, prediction-error signals dependent on the prior predictions are found in the gamma band. Our findings reveal a frequency ordering of prediction signals and their hierarchical interactions with prediction-error signals supporting predictive coding theory.

A computational framework can extract spatio-spectro-temporal neural signatures corresponding to hierarchical prediction and prediction errors in a local-global auditory task.

The current status and trend of the functional magnetic resonance combined with stimulation in animals

As a non-radiative, non-invasive imaging technique, functional magnetic resonance imaging (fMRI) has excellent effects on studying the activation of blood oxygen levels and functional connectivity of the brain in human and animal models. Compared with resting-state fMRI, fMRI combined with stimulation could be used to assess the activation of specific brain regions and the connectivity of specific pathways and achieve better signal capture with a clear purpose and more significant results. Various fMRI methods and specific stimulation paradigms have been proposed to investigate brain activation in a specific state, such as electrical, mechanical, visual, olfactory, and direct brain stimulation. In this review, the studies on animal brain activation using fMRI combined with different stimulation methods were retrieved. The instruments, experimental parameters, anesthesia, and animal models in different stimulation conditions were summarized. The findings would provide a reference for studies on estimating specific brain activation using fMRI combined with stimulation.

Caught in the ACTS: Defining Abstract Cognitive Task Sequences as an Independent Process

Cognitive neuroscience currently conflates the study of serial responses (e.g., delay match to sample/nonsample, n-back) with the study of sequential operations. In this essay, our goal is to define and disentangle the latter, termed abstract cognitive task sequences (ACTS). Existing literatures address tasks requiring serial events, including procedural learning of implicit motor responses, statistical learning of predictive relationships, and judgments of attributes. These findings do not describe the behavior and underlying mechanism required to succeed at remembering to evaluate color, then shape; or to multiply, then add. A new literature is needed to characterize these sorts of second-order cognitive demands of studying a sequence of operations. Our second goal is to characterize gaps in knowledge related to ACTS that merit further investigation. In the following sections, we define more precisely what we mean by ACTS and suggest research questions' further investigation would be positioned to address.

Deep brain stimulation of the thalamus restores signatures of consciousness in a nonhuman primate model

Loss of consciousness is associated with the disruption of long-range thalamocortical and corticocortical brain communication. We tested the hypothesis that deep brain stimulation (DBS) of central thalamus might restore both arousal and awareness following consciousness loss. We applied anesthesia to suppress consciousness in nonhuman primates. During anesthesia, central thalamic stimulation induced arousal in an on-off manner and increased functional magnetic resonance imaging activity in prefrontal, parietal, and cingulate cortices. Moreover, DBS restored a broad dynamic repertoire of spontaneous resting-state activity, previously described as a signature of consciousness. None of these effects were obtained during the stimulation of a control site in the ventrolateral thalamus. Last, DBS restored a broad hierarchical response to auditory violations that was disrupted under anesthesia. Thus, DBS restored the two dimensions of consciousness, arousal and conscious access, following consciousness loss, paving the way to its therapeutical translation in patients with disorders of consciousness.

Constructing the hierarchy of predictive auditory sequences in the marmoset brain

Our brains constantly generate predictions of sensory input that are compared with actual inputs, propagate the prediction-errors through a hierarchy of brain regions, and subsequently update the internal predictions of the world. However, the essential feature of predictive coding, the notion of hierarchical depth and its neural mechanisms, remains largely unexplored. Here, we investigated the hierarchical depth of predictive auditory processing by combining functional magnetic resonance imaging (fMRI) and high-density whole-brain electrocorticography (ECoG) in marmoset monkeys during an auditory local-global paradigm in which the temporal regularities of the stimuli were designed at two hierarchical levels. The prediction-errors and prediction updates were examined as neural responses to auditory mismatches and omissions. Using fMRI, we identified a hierarchical gradient along the auditory pathway: midbrain and sensory regions represented local, shorter-time-scale predictive processing followed by associative auditory regions, whereas anterior temporal and prefrontal areas represented global, longer-time-scale sequence processing. The complementary ECoG recordings confirmed the activations at cortical surface areas and further differentiated the signals of prediction-error and update, which were transmitted via putative bottom-up γ and top-down β oscillations, respectively. Furthermore, omission responses caused by absence of input, reflecting solely the two levels of prediction signals that are unique to the hierarchical predictive coding framework, demonstrated the hierarchical top-down process of predictions in the auditory, temporal, and prefrontal areas. Thus, our findings support the hierarchical predictive coding framework, and outline how neural networks and spatiotemporal dynamics are used to represent and arrange a hierarchical structure of auditory sequences in the marmoset brain.

The macaque ventral intraparietal area has expanded into three homologue human parietal areas

The macaque ventral intraparietal area (VIP) in the fundus of the intraparietal sulcus has been implicated in a diverse range of sensorimotor and cognitive functions such as motion processing, multisensory integration, processing of head peripersonal space, defensive behavior, and numerosity coding. Here, we exhaustively review macaque VIP function, cytoarchitectonics, and anatomical connectivity and integrate it with human studies that have attempted to identify a potential human VIP homologue. We show that human VIP research has consistently identified three, rather than one, bilateral parietal areas that each appear to subsume some, but not all, of the macaque area's functionality. Available evidence suggests that this human "VIP complex" has evolved as an expansion of the macaque area, but that some precursory specialization within macaque VIP has been previously overlooked. The three human areas are dominated, roughly, by coding the head or self in the environment, visual heading direction, and the peripersonal environment around the head, respectively. A unifying functional principle may be best described as prediction in space and time, linking VIP to state estimation as a key parietal sensorimotor function. VIP's expansive differentiation of head and self-related processing may have been key in the emergence of human bodily self-consciousness.

Cerebral representation of sequence patterns across multiple presentation formats

The ability to detect the abstract pattern underlying a temporal sequence of events is crucial to many human activities, including language and mathematics, but its cortical correlates remain poorly understood. It is also unclear whether repeated exposure to the same sequence of sensory stimuli is sufficient to induce the encoding of an abstract amodal representation of the pattern. Using functional MRI, we probed the existence of such abstract codes for sequential patterns, their localization in the human brain, and their relation to existing language and math-responsive networks. We used a passive sequence violation paradigm, in which a given sequence is repeatedly presented before rare deviant sequences are introduced. We presented two binary patterns, AABB and ABAB, in four presentation formats, either visual or auditory, and either cued by the identity of the stimuli or by their spatial location. Regardless of the presentation format, a habituation to the repeated pattern and a response to pattern violations were seen in a set of inferior frontal, intraparietal and temporal areas. Within language areas, such pattern-violation responses were only found in the inferior frontal gyrus (IFG), whereas all math-responsive regions responded to pattern changes. Most of these regions also responded whenever the modality or the cue changed, suggesting a general sensitivity to violation detection. Thus, the representation of sequence patterns appears to be distributed, yet to include a core set of abstract amodal regions, particularly the IFG.

Automatic Sensory Predictions: A Review of Predictive Mechanisms in the Brain and Their Link to Conscious Processing

The human brain has the astonishing capacity of integrating streams of sensory information from the environment and forming predictions about future events in an automatic way. Despite being initially developed for visual processing, the bulk of predictive coding research has subsequently focused on auditory processing, with the famous mismatch negativity signal as possibly the most studied signature of a surprise or prediction error (PE) signal. Auditory PEs are present during various consciousness states. Intriguingly, their presence and characteristics have been linked with residual levels of consciousness and return of awareness. In this review we first give an overview of the neural substrates of predictive processes in the auditory modality and their relation to consciousness. Then, we focus on different states of consciousness - wakefulness, sleep, anesthesia, coma, meditation, and hypnosis - and on what mysteries predictive processing has been able to disclose about brain functioning in such states. We review studies investigating how the neural signatures of auditory predictions are modulated by states of reduced or lacking consciousness. As a future outlook, we propose the combination of electrophysiological and computational techniques that will allow investigation of which facets of sensory predictive processes are maintained when consciousness fades away.

Common functional localizers to enhance NHP & cross-species neuroscience imaging research

Functional localizers are invaluable as they can help define regions of interest, provide cross-study comparisons, and most importantly, allow for the aggregation and meta-analyses of data across studies and laboratories. To achieve these goals within the non-human primate (NHP) imaging community, there is a pressing need for the use of standardized and validated localizers that can be readily implemented across different groups. The goal of this paper is to provide an overview of the value of localizer protocols to imaging research and we describe a number of commonly used or novel localizers within NHPs, and keys to implement them across studies. As has been shown with the aggregation of resting-state imaging data in the original PRIME-DE submissions, we believe that the field is ready to apply the same initiative for task-based functional localizers in NHP imaging. By coming together to collect large datasets across research group, implementing the same functional localizers, and sharing the localizers and data via PRIME-DE, it is now possible to fully test their robustness, selectivity and specificity. To do this, we reviewed a number of common localizers and we created a repository of well-established localizer that are easily accessible and implemented through the PRIME-RE platform.

Violation of rhythmic expectancies can elicit late frontal gamma activity nested in theta oscillations

Rhythm processing involves building expectations according to the hierarchical temporal structure of auditory events. Although rhythm processing has been addressed in the context of predictive coding, the properties of the oscillatory response in different cortical areas are still not clear. We explored the oscillatory properties of the neural response to rhythmic incongruence and the cross-frequency coupling between multiple frequencies to further investigate the mechanisms underlying rhythm perception. We designed an experiment to investigate the neural response to rhythmic deviations in which the tone either arrived earlier than expected or the tone in the same metrical position was omitted. These two manipulations modulate the rhythmic structure differently, with the former creating a larger violation of the general structure of the musical stimulus than the latter. Both deviations resulted in an MMN response, whereas only the rhythmic deviant resulted in a subsequent P3a. Rhythmic deviants due to the early occurrence of a tone, but not omission deviants, seemed to elicit a late high gamma response (60-80 Hz) at the end of the P3a over the left frontal region, which, interestingly, correlated with the P3a amplitude over the same region and was also nested in theta oscillations. The timing of the elicited high-frequency gamma oscillations related to rhythmic deviation suggests that it might be related to the update of the predictive neural model, corresponding to the temporal structure of the events in higher-level cortical areas.

Combining brain perturbation and neuroimaging in non-human primates

Brain perturbation studies allow detailed causal inferences of behavioral and neural processes. Because the combination of brain perturbation methods and neural measurement techniques is inherently challenging, research in humans has predominantly focused on non-invasive, indirect brain perturbations, or neurological lesion studies. Non-human primates have been indispensable as a neurobiological system that is highly similar to humans while simultaneously being more experimentally tractable, allowing visualization of the functional and structural impact of systematic brain perturbation. This review considers the state of the art in non-human primate brain perturbation with a focus on approaches that can be combined with neuroimaging. We consider both non-reversible (lesions) and reversible or temporary perturbations such as electrical, pharmacological, optical, optogenetic, chemogenetic, pathway-selective, and ultrasound based interference methods. Method-specific considerations from the research and development community are offered to facilitate research in this field and support further innovations. We conclude by identifying novel avenues for further research and innovation and by highlighting the clinical translational potential of the methods.

Brain States and Transitions: Insights from Computational Neuroscience

Within the field of computational neuroscience there are great expectations of finding new ways to rebalance the complex dynamic system of the human brain through controlled pharmacological or electromagnetic perturbation. Yet many obstacles remain between the ability to accurately predict how and where best to perturb to force a transition from one brain state to another. The foremost challenge is a commonly agreed definition of a given brain state. Recent progress in computational neuroscience has made it possible to robustly define brain states and force transitions between them. Here, we review the state of the art and propose a framework for determining the functional hierarchical organization describing any given brain state. We describe the latest advances in creating sophisticated whole-brain computational models with interacting neuronal and neurotransmitter systems that can be studied fully in silico to predict and design novel pharmacological and electromagnetic interventions to rebalance them in disease.

The evolution of hierarchical structure building capacity for language and music: a bottom-up perspective

A central property of human language is its hierarchical structure. Humans can flexibly combine elements to build a hierarchical structure expressing rich semantics. A hierarchical structure is also considered as playing a key role in many other human cognitive domains. In music, auditory-motor events are combined into hierarchical pitch and/or rhythm structure expressing affect. How did such a hierarchical structure building capacity evolve? This paper investigates this question from a bottom-up perspective based on a set of action-related components as a shared basis underlying cognitive capacities of nonhuman primates and humans. Especially, I argue that the evolution of hierarchical structure building capacity for language and music is tractable for comparative evolutionary study once we focus on the gradual elaboration of shared brain architecture: the cortico-basal ganglia-thalamocortical circuits for hierarchical control of goal-directed action and the dorsal pathways for hierarchical internal models. I suggest that this gradual elaboration of the action-related brain architecture in the context of vocal control and tool-making went hand in hand with amplification of working memory, and made the brain ready for hierarchical structure building in language and music.

Disentangling perceptual awareness from nonconscious processing in rhesus monkeys (Macaca mulatta)

Scholars have long debated whether animals, which display impressive intelligent behaviors, are consciously aware or not. Yet, because many complex human behaviors and high-level functions can be performed without conscious awareness, it was long considered impossible to untangle whether animals are aware or just conditionally or nonconsciously behaving. Here, we developed an empirical approach to address this question. We harnessed a well-established cross-over double dissociation between nonconscious and conscious processing, in which people perform in completely opposite ways when they are aware of stimuli versus when they are not. To date, no one has explored if similar performance dissociations exist in a nonhuman species. In a series of seven experiments, we first established these signatures in humans using both known and newly developed nonverbal double-dissociation tasks and then identified similar signatures in rhesus monkeys (Macaca mulatta). These results provide robust evidence for two distinct modes of processing in nonhuman primates. This empirical approach makes it feasible to disentangle conscious visual awareness from nonconscious processing in nonhuman species; hence, it can be used to strip away ambiguity when exploring the processes governing intelligent behavior across the animal kingdom. Taken together, these results strongly support the existence of both nonconscious processing as well as functional human-like visual awareness in nonhuman animals.

Human Brain Ages With Hierarchy-Selective Attenuation of Prediction Errors

From the perspective of predictive coding, our brain embodies a hierarchical generative model to realize perception, which proactively predicts the statistical structure of sensory inputs. How are these predictive processes modified as we age? Recent research suggested that aging leads to decreased weighting of sensory inputs and increased reliance on predictions. Here we investigated whether this age-related shift from sensorium to predictions occurs at all levels of hierarchical message passing. We recorded the electroencephalography responses with an auditory local-global paradigm in a cohort of 108 healthy participants from 3 groups: seniors, adults, and adolescents. The detection of local deviancy seems largely preserved in older individuals at earlier latency (including the mismatch negativity followed by the P3a but not the reorienting negativity). In contrast, the detection of global deviancy is clearly compromised in older individuals, as they showed worse task performance and attenuated P3b. Our findings demonstrate that older brains show little decline in sensory (i.e., first-order) prediction errors but significant diminution in contextual (i.e., second-order) prediction errors. Age-related deficient maintenance of auditory information in working memory might affect whether and how lower-level prediction errors propagate to the higher level.

Using non-invasive neuroimaging to enhance the care, well-being and experimental outcomes of laboratory non-human primates (monkeys)

Over the past 10-20 years, neuroscience witnessed an explosion in the use of non-invasive imaging methods, particularly magnetic resonance imaging (MRI), to study brain structure and function. Simultaneously, with access to MRI in many research institutions, MRI has become an indispensable tool for researchers and veterinarians to guide improvements in surgical procedures and implants and thus, experimental as well as clinical outcomes, given that access to MRI also allows for improved diagnosis and monitoring for brain disease. As part of the PRIMEatE Data Exchange, we gathered expert scientists, veterinarians, and clinicians who treat humans, to provide an overview of the use of non-invasive imaging tools, primarily MRI, to enhance experimental and welfare outcomes for laboratory non-human primates engaged in neuroscientific experiments. We aimed to provide guidance for other researchers, scientists and veterinarians in the use of this powerful imaging technology as well as to foster a larger conversation and community of scientists and veterinarians with a shared goal of improving the well-being and experimental outcomes for laboratory animals.

A theory of memory for binary sequences: Evidence for a mental compression algorithm in humans

Working memory capacity can be improved by recoding the memorized information in a condensed form. Here, we tested the theory that human adults encode binary sequences of stimuli in memory using an abstract internal language and a recursive compression algorithm. The theory predicts that the psychological complexity of a given sequence should be proportional to the length of its shortest description in the proposed language, which can capture any nested pattern of repetitions and alternations using a limited number of instructions. Five experiments examine the capacity of the theory to predict human adults' memory for a variety of auditory and visual sequences. We probed memory using a sequence violation paradigm in which participants attempted to detect occasional violations in an otherwise fixed sequence. Both subjective complexity ratings and objective violation detection performance were well predicted by our theoretical measure of complexity, which simply reflects a weighted sum of the number of elementary instructions and digits in the shortest formula that captures the sequence in our language. While a simpler transition probability model, when tested as a single predictor in the statistical analyses, accounted for significant variance in the data, the goodness-of-fit with the data significantly improved when the language-based complexity measure was included in the statistical model, while the variance explained by the transition probability model largely decreased. Model comparison also showed that shortest description length in a recursive language provides a better fit than six alternative previously proposed models of sequence encoding. The data support the hypothesis that, beyond the extraction of statistical knowledge, human sequence coding relies on an internal compression using language-like nested structures.

Hierarchical disruption in the cortex of anesthetized monkeys as a new signature of consciousness loss

Anesthesia induces a reconfiguration of the repertoire of functional brain states leading to a high function-structure similarity. However, it is unclear how these functional changes lead to loss of consciousness. Here we suggest that the mechanism of conscious access is related to a general dynamical rearrangement of the intrinsic hierarchical organization of the cortex. To measure cortical hierarchy, we applied the Intrinsic Ignition analysis to resting-state fMRI data acquired in awake and anesthetized macaques. Our results reveal the existence of spatial and temporal hierarchical differences of neural activity within the macaque cortex, with a strong modulation by the depth of anesthesia and the employed anesthetic agent. Higher values of Intrinsic Ignition correspond to rich and flexible brain dynamics whereas lower values correspond to poor and rigid, structurally driven brain dynamics. Moreover, spatial and temporal hierarchical dimensions are disrupted in a different manner, involving different hierarchical brain networks. All together suggest that disruption of brain hierarchy is a new signature of consciousness loss.

Signature of consciousness in brain-wide synchronization patterns of monkey and human fMRI signals

During the sleep-wake cycle, the brain undergoes profound dynamical changes, which manifest subjectively as transitions between conscious experience and unconsciousness. Yet, neurophysiological signatures that can objectively distinguish different consciousness states based are scarce. Here, we show that differences in the level of brain-wide signals can reliably distinguish different stages of sleep and anesthesia from the awake state in human and monkey fMRI resting state data. Moreover, a whole-brain computational model can faithfully reproduce changes in global synchronization and other metrics such as functional connectivity, structure-function relationship, integration and segregation across vigilance states. We demonstrate that the awake brain is close to a Hopf bifurcation, which naturally coincides with the emergence of globally correlated fMRI signals. Furthermore, simulating lesions of individual brain areas highlights the importance of connectivity hubs in the posterior brain and subcortical nuclei for maintaining the model in the awake state, as predicted by graph-theoretical analyses of structural data.

Sedation Modulates Frontotemporal Predictive Coding Circuits and the Double Surprise Acceleration Effect

Two important theories in cognitive neuroscience are predictive coding (PC) and the global workspace (GW) theory. A key research task is to understand how these two theories relate to one another, and particularly, how the brain transitions from a predictive early state to the eventual engagement of a brain-scale state (the GW). To address this question, we present a source-localization of EEG responses evoked by the local-global task—an experimental paradigm that engages a predictive hierarchy, which encompasses the GW. The results of our source reconstruction suggest three phases of processing. The first phase involves the sensory (here auditory) regions of the superior temporal lobe and predicts sensory regularities over a short timeframe (as per the local effect). The third phase is brain-scale, involving inferior frontal, as well as inferior and superior parietal regions, consistent with a global neuronal workspace (GNW; as per the global effect). Crucially, our analysis suggests that there is an intermediate (second) phase, involving modulatory interactions between inferior frontal and superior temporal regions. Furthermore, sedation with propofol reduces modulatory interactions in the second phase. This selective effect is consistent with a PC explanation of sedation, with propofol acting on descending predictions of the precision of prediction errors; thereby constraining access to the GNW.

How does the brain learn environmental structure? Ten core principles for understanding the neurocognitive mechanisms of statistical learning

Despite a growing body of research devoted to the study of how humans encode environmental patterns, there is still no clear consensus about the nature of the neurocognitive mechanisms underpinning statistical learning nor what factors constrain or promote its emergence across individuals, species, and learning situations. Based on a review of research examining the roles of input modality and domain, input structure and complexity, attention, neuroanatomical bases, ontogeny, and phylogeny, ten core principles are proposed. Specifically, there exist two sets of neurocognitive mechanisms underlying statistical learning. First, a "suite" of associative-based, automatic, modality-specific learning mechanisms are mediated by the general principle of cortical plasticity, which results in improved processing and perceptual facilitation of encountered stimuli. Second, an attention-dependent system, mediated by the prefrontal cortex and related attentional and working memory networks, can modulate or gate learning and is necessary in order to learn nonadjacent dependencies and to integrate global patterns across time. This theoretical framework helps clarify conflicting research findings and provides the basis for future empirical and theoretical endeavors.

Evaluating the neurophysiological evidence for predictive processing as a model of perception

For many years, the dominant theoretical framework guiding research into the neural origins of perceptual experience has been provided by hierarchical feedforward models, in which sensory inputs are passed through a series of increasingly complex feature detectors. However, the long‐standing orthodoxy of these accounts has recently been challenged by a radically different set of theories that contend that perception arises from a purely inferential process supported by two distinct classes of neurons: those that transmit predictions about sensory states and those that signal sensory information that deviates from those predictions. Although these predictive processing (PP) models have become increasingly influential in cognitive neuroscience, they are also criticized for lacking the empirical support to justify their status. This limited evidence base partly reflects the considerable methodological challenges that are presented when trying to test the unique predictions of these models. However, a confluence of technological and theoretical advances has prompted a recent surge in human and nonhuman neurophysiological research seeking to fill this empirical gap. Here, we will review this new research and evaluate the degree to which its findings support the key claims of PP.

fMRI reveals language-specific predictive coding during naturalistic sentence comprehension

Much research in cognitive neuroscience supports prediction as a canonical computation of cognition across domains. Is such predictive coding implemented by feedback from higher-order domain-general circuits, or is it locally implemented in domain-specific circuits? What information sources are used to generate these predictions? This study addresses these two questions in the context of language processing. We present fMRI evidence from a naturalistic comprehension paradigm (1) that predictive coding in the brain's response to language is domain-specific, and (2) that these predictions are sensitive both to local word co-occurrence patterns and to hierarchical structure. Using a recently developed continuous-time deconvolutional regression technique that supports data-driven hemodynamic response function discovery from continuous BOLD signal fluctuations in response to naturalistic stimuli, we found effects of prediction measures in the language network but not in the domain-general multiple-demand network, which supports executive control processes and has been previously implicated in language comprehension. Moreover, within the language network, surface-level and structural prediction effects were separable. The predictability effects in the language network were substantial, with the model capturing over 37% of explainable variance on held-out data. These findings indicate that human sentence processing mechanisms generate predictions about upcoming words using cognitive processes that are sensitive to hierarchical structure and specialized for language processing, rather than via feedback from high-level executive control mechanisms.

Cortical mapping of mismatch responses to independent acoustic features

Predictive coding is an influential theory of neural processing underlying perceptual inference. However, it is unknown to what extent prediction violations of different sensory features are mediated in different regions in auditory cortex, with different dynamics, and by different mechanisms. This study investigates the neural responses to synthesized acoustic syllables, which could be expected or unexpected, along several features. By using electrocorticography (ECoG) in rat auditory cortex (subjects: adult female Wistar rats with normal hearing), we aimed at mapping regional differences in mismatch responses to different stimulus features. Continuous streams of morphed syllables formed roving oddball sequences in which each stimulus was repeated several times (thereby forming a standard) and subsequently replaced with a deviant stimulus which differed from the standard along one of several acoustic features: duration, pitch, interaural level differences (ILD), or consonant identity. Each of these features could assume one of several different levels, and the resulting change from standard to deviant could be larger or smaller. The deviant stimuli were then repeated to form new standards. We analyzed responses to the first repetition of a new stimulus (deviant) and its last repetition in a stimulus train (standard). For the ECoG recording, we implanted urethane-anaesthetized rats with 8 × 8 surface electrode arrays covering a 3 × 3 mm cortical patch encompassing primary and higher-order auditory cortex. We identified the response topographies and latencies of population activity evoked by acoustic stimuli in the rat auditory regions, and mapped their sensitivity to expectation violations along different acoustic features. For all features, the responses to deviant stimuli increased in amplitude relative to responses to standard stimuli. Deviance magnitude did not further modulate these mismatch responses. Mismatch responses to different feature violations showed a heterogeneous distribution across cortical areas, with no evidence for systematic topographic gradients for any of the tested features. However, within rats, the spatial distribution of mismatch responses varied more between features than the spatial distribution of tone-evoked responses. This result supports the notion that prediction error signaling along different stimulus features is subserved by different cortical populations, albeit with substantial heterogeneity across individuals.

Resting-state Dynamics as a Cortical Signature of Anesthesia in Monkeys

WHAT WE ALREADY KNOW ABOUT THIS TOPIC

WHAT THIS ARTICLE TELLS US THAT IS NEW: BACKGROUND:: The mechanism by which anesthetics induce a loss of consciousness remains a puzzling problem. We hypothesized that a cortical signature of anesthesia could be found in an increase in similarity between the matrix of resting-state functional correlations and the anatomical connectivity matrix of the brain, resulting in an increased function-structure similarity.

METHODS

We acquired resting-state functional magnetic resonance images in macaque monkeys during wakefulness (n = 3) or anesthesia with propofol (n = 3), ketamine (n = 3), or sevoflurane (n = 3). We used the k-means algorithm to cluster dynamic resting-state data into independent functional brain states. For each condition, we performed a regression analysis to quantify function-structure similarity and the repertoire of functional brain states.

RESULTS

Seven functional brain states were clustered and ranked according to their similarity to structural connectivity, with higher ranks corresponding to higher function-structure similarity and lower ranks corresponding to lower correlation between brain function and brain anatomy. Anesthesia shifted the brain state composition from a low rank (rounded rank [mean ± SD]) in the awake condition (awake rank = 4 [3.58 ± 1.03]) to high ranks in the different anesthetic conditions (ketamine rank = 6 [6.10 ± 0.32]; moderate propofol rank = 6 [6.15 ± 0.76]; deep propofol rank = 6 [6.16 ± 0.46]; moderate sevoflurane rank = 5 [5.10 ± 0.81]; deep sevoflurane rank = 6 [5.81 ± 1.11]; P < 0.0001).

CONCLUSIONS

Whatever the molecular mechanism, anesthesia led to a massive reconfiguration of the repertoire of functional brain states that became predominantly shaped by brain anatomy (high function-structure similarity), giving rise to a well-defined cortical signature of anesthesia-induced loss of consciousness.

Neuronal Encoding in a High-Level Auditory Area: From Sequential Order of Elements to Grammatical Structure

Sensitivity to the sequential structure of communication sounds is fundamental not only for language comprehension in humans but also for song recognition in songbirds. By quantifying single-unit responses, we first assessed whether the sequential order of song elements, called syllables, in conspecific songs is encoded in a secondary auditory cortex-like region of the zebra finch brain. Based on a habituation/dishabituation paradigm, we show that, after multiple repetitions of the same conspecific song, rearranging syllable order reinstated strong responses. A large proportion of neurons showed sensitivity to song context in which syllables occurred providing support for the nonlinear processing of syllable sequences. Sensitivity to the temporal order of items within a sequence should enable learning its underlying structure, an ability considered a core mechanism of the human language faculty. We show that repetitions of songs that were ordered according to a specific grammatical structure (i.e., ABAB or AABB structures; A and B denoting song syllables) led to different responses in both anesthetized and awake birds. Once responses were decreased due to song repetitions, the transition from one structure to the other could affect the firing rates and/or the spike patterns. Our results suggest that detection was based on local differences rather than encoding of the global song structure as a whole. Our study demonstrates that a high-level auditory region provides neuronal mechanisms to help discriminate stimuli that differ in their sequential structure.SIGNIFICANCE STATEMENT Sequence processing has been proposed as a potential precursor of language syntax. As a sequencing operation, the encoding of the temporal order of items within a sequence may help in recognition of relationships between adjacent items and in learning the underlying structure. Taking advantage of the stimulus-specific adaptation phenomenon observed in a high-level auditory region of the zebra finch brain, we addressed this question at the neuronal level. Reordering elements within conspecific songs reinstated robust responses. Neurons also detected changes in the structure of artificial songs, and this detection depended on local transitions between adjacent or nonadjacent syllables. These findings establish the songbird as a model system for deciphering the mechanisms underlying sequence processing at the single-cell level.

Auditory and Visual Sequence Learning in Humans and Monkeys using an Artificial Grammar Learning Paradigm

Language flexibly supports the human ability to communicate using different sensory modalities, such as writing and reading in the visual modality and speaking and listening in the auditory domain. Although it has been argued that nonhuman primate communication abilities are inherently multisensory, direct behavioural comparisons between human and nonhuman primates are scant. Artificial grammar learning (AGL) tasks and statistical learning experiments can be used to emulate ordering relationships between words in a sentence. However, previous comparative work using such paradigms has primarily investigated sequence learning within a single sensory modality. We used an AGL paradigm to evaluate how humans and macaque monkeys learn and respond to identically structured sequences of either auditory or visual stimuli. In the auditory and visual experiments, we found that both species were sensitive to the ordering relationships between elements in the sequences. Moreover, the humans and monkeys produced largely similar response patterns to the visual and auditory sequences, indicating that the sequences are processed in comparable ways across the sensory modalities. These results provide evidence that human sequence processing abilities stem from an evolutionarily conserved capacity that appears to operate comparably across the sensory modalities in both human and nonhuman primates. The findings set the stage for future neurobiological studies to investigate the multisensory nature of these sequencing operations in nonhuman primates and how they compare to related processes in humans.

Processing of auditory novelty across the cortical hierarchy: An intracranial electrophysiology study

Under the predictive coding hypothesis, specific spatiotemporal patterns of cortical activation are postulated to occur during sensory processing as expectations generate feedback predictions and prediction errors generate feedforward signals. Establishing experimental evidence for this information flow within cortical hierarchy has been difficult, especially in humans, due to spatial and temporal limitations of non-invasive measures of cortical activity. This study investigated cortical responses to auditory novelty using the local/global deviant paradigm, which engages the hierarchical network underlying auditory predictive coding over short ('local deviance'; LD) and long ('global deviance'; GD) time scales. Electrocorticographic responses to auditory stimuli were obtained in neurosurgical patients from regions of interest (ROIs) including auditory, auditory-related and prefrontal cortex. LD and GD effects were assayed in averaged evoked potential (AEP) and high gamma (70-150 Hz) signals, the former likely dominated by local synaptic currents and the latter largely reflecting local spiking activity. AEP LD effects were distributed across all ROIs, with greatest percentage of significant sites in core and non-core auditory cortex. High gamma LD effects were localized primarily to auditory cortex in the superior temporal plane and on the lateral surface of the superior temporal gyrus (STG). LD effects exhibited progressively longer latencies in core, non-core, auditory-related and prefrontal cortices, consistent with feedforward signaling. The spatial distribution of AEP GD effects overlapped that of LD effects, but high gamma GD effects were more restricted to non-core areas. High gamma GD effects had shortest latencies in STG and preceded AEP GD effects in most ROIs. This latency profile, along with the paucity of high gamma GD effects in the superior temporal plane, suggest that the STG plays a prominent role in initiating novelty detection signals over long time scales. Thus, the data demonstrate distinct patterns of information flow in human cortex associated with auditory novelty detection over multiple time scales.

Disruption of Boundary Encoding During Sensorimotor Sequence Learning: An MEG Study

Music performance relies on the ability to learn and execute actions and their associated sounds. The process of learning these auditory-motor contingencies depends on the proper encoding of the serial order of the actions and sounds. Among the different serial positions of a behavioral sequence, the first and last (boundary) elements are particularly relevant. Animal and patient studies have demonstrated a specific neural representation for boundary elements in prefrontal cortical regions and in the basal ganglia, highlighting the relevance of their proper encoding. The neural mechanisms underlying the encoding of sequence boundaries in the general human population remain, however, largely unknown. In this study, we examined how alterations of auditory feedback, introduced at different ordinal positions (boundary or within-sequence element), affect the neural and behavioral responses during sensorimotor sequence learning. Analysing the neuromagnetic signals from 20 participants while they performed short piano sequences under the occasional effect of altered feedback (AF), we found that at around 150-200 ms post-keystroke, the neural activities in the dorsolateral prefrontal cortex (DLPFC) and supplementary motor area (SMA) were dissociated for boundary and within-sequence elements. Furthermore, the behavioral data demonstrated that feedback alterations on boundaries led to greater performance costs, such as more errors in the subsequent keystrokes. These findings jointly support the idea that the proper encoding of boundaries is critical in acquiring sensorimotor sequences. They also provide evidence for the involvement of a distinct neural circuitry in humans including prefrontal and higher-order motor areas during the encoding of the different classes of serial order.