Sensitivity to structure in action sequences: An infant event-related potential study

Coregistration of EEG and eye-tracking in infants and developing populations

Infants cannot be instructed where to look; therefore, infant researchers rely on observation of their participant's gaze to make inferences about their cognitive processes. They therefore started studying infant attention in the real world from early on. Developmental researchers were early adopters of methods combining observations of gaze and behaviour with electroencephalography (EEG) to study attention and other cognitive functions. However, the direct combination of eye-tracking methods and EEG to test infants is still rare, as it includes specific challenges. The current article reviews the development of co-registration research in infancy. It points out specific challenges of co-registration in infant research and suggests ways to overcome them. It ends with recommendations for implementing the co-registration of EEG and eye-tracking in infant research to maximise the benefits of the two measures and their combination and to orient on Open Science principles while doing so. In summary, this work shows that the co-registration of EEG and eye-tracking in infant research can be beneficial to studying natural and real-world behaviour despite its challenges.

Theta Signal Transfer from Parietal to Prefrontal Cortex Ignites Conscious Awareness of Implicit Knowledge during Sequence Learning

Unconscious acquisition of sequence structure from experienced events can lead to explicit awareness of the pattern through extended practice. Although the implicit-to-explicit transition has been extensively studied in humans using the serial reaction time (SRT) task, the subtle neural activity supporting this transition remains unclear. Here, we investigated whether frequency-specific neural signal transfer contributes to this transition. A total of 208 participants (107 females) learned a sequence pattern through a multisession SRT task, allowing us to observe the transitions. Session-by-session measures of participants' awareness for sequence knowledge were conducted during the SRT task to identify the session when the transition occurred. By analyzing time course RT data using switchpoint modeling, we identified an increase in learning benefit specifically at the transition session. Electroencephalogram (EEG)/magnetoencephalogram (MEG) recordings revealed increased theta power in parietal (precuneus) regions one session before the transition (pretransition) and a prefrontal (superior frontal gyrus; SFG) one at the transition session. Phase transfer entropy (PTE) analysis confirmed that directional theta transfer from precuneus → SFG occurred at the pretransition session and its strength positively predicted learning improvement at the subsequent transition session. Furthermore, repetitive transcranial magnetic stimulation (TMS) modulated precuneus theta power and altered transfer strength from precuneus to SFG, resulting in changes in both transition rate and learning benefit at that specific point of transition. Our brain-stimulation evidence supports a role for parietal → prefrontal theta signal transfer in igniting conscious awareness of implicitly acquired knowledge.SIGNIFICANCE STATEMENT There exists a pervasive phenomenon wherein individuals unconsciously acquire sequence patterns from their environment, gradually becoming aware of the underlying regularities through repeated practice. While previous studies have established the robustness of this implicit-to-explicit transition in humans, the refined neural mechanisms facilitating conscious access to implicit knowledge remain poorly understood. Here, we demonstrate that prefrontal activity, known to be crucial for conscious awareness, is triggered by neural signal transfer originating from the posterior brain region, specifically the precuneus. By employing brain stimulation techniques, we establish a causal link between neural signal transfer and the occurrence of awareness. Our findings unveil a mechanism by which implicit knowledge becomes consciously accessible in human cognition.

Electrophysiological Evidence for A Number-Action Mapping in Infancy

In the last decades, a growing body of literature has focused on the link between number and action. Many studies conducted on adult participants have provided evidence for a bidirectional influence between numerosity processing and grasping or reaching actions. However, it is not yet clear whether this link is functional in early infancy. Here, we used the event-related potential (ERP) technique to record electrical activity of the brain in response to number-hand pairings. We implemented a cueing paradigm where 3- to 4-month-old infants observed images showing either congruency (e.g., a large numerosity primed by a large hand opening) or incongruency (e.g., a large numerosity primed by a small hand opening). Infants' brain activity was modulated by the congruency of the pairings: amplitudes recorded over frontal and parietal-occipital scalp positions differed for congruent versus incongruent pairings. These findings suggest that the association between number and hand action processing is already functional early in life.

Beyond Infant's Looking: The Neural Basis for Infant Prediction Errors

Contemporary conceptualizations on infant cognitive development focus on predictive processes; the basic idea is that the brain continuously creates predictions about what is expected and that the divergence between predicted and actual perceived data yields a prediction error. This prediction error updates the model from which the predictions are generated and therefore is a basic mechanism for learning and adaptation to the dynamics of the ever-changing environment. In this article, we review the types of available empirical evidence supporting the idea that predictive processes can be found in infancy, especially emphasizing the contribution of electrophysiology as a potential method for testing the similarity of the brain mechanisms for processing prediction errors in infants to those of adults. In infants, as with older children, adolescents, and adults, predictions involve synchronization bursts of middle-central theta reflecting brain activity in the anterior cingulate cortex. We discuss how early in development such brain mechanisms develop and open questions that still remain to be empirically investigated.

Visual statistical learning in deaf and hearing infants and toddlers

Congenital hearing loss offers a unique opportunity to examine the role of sound in cognitive, social, and linguistic development. Children with hearing loss demonstrate atypical performance across a range of general cognitive skills. For instance, research has shown that deaf school-age children underperform on visual statistical learning (VSL) tasks. However, the evidence for these deficits has been challenged, with mixed findings emerging in recent years. Here, we used a novel approach to examine VSL in the action domain early in development. We compared learning between deaf and hearing infants, prior to cochlear implantation (pre-CI), and a group of toddlers post implantation (post-CI). Findings revealed a significant difference between deaf and hearing infants pre-CI, with evidence for learning only in the hearing infants. However, there were no significant group differences between deaf and hearing toddlers post-CI, with both groups demonstrating learning. Further, VSL performance was positively correlated with language scores for the deaf toddlers, adding to the body of evidence suggesting that statistical learning is associated with language abilities. We discuss these findings in the context of previous evidence for group differences in VSL skills, and the role that auditory experiences play in infant cognitive development.

Understanding the causes and consequences of variability in infant ERP editing practices

The current study examined the effects of variability on infant event-related potential (ERP) data editing methods. A widespread approach for analyzing infant ERPs is through a trial-by-trial editing process. Researchers identify electroencephalogram (EEG) channels containing artifacts and reject trials that are judged to contain excessive noise. This process can be performed manually by experienced researchers, partially automated by specialized software, or completely automated using an artifact-detection algorithm. Here, we compared the editing process from four different editors-three human experts and an automated algorithm-on the final ERP from an existing infant EEG dataset. Findings reveal that agreement between editors was low, for both the numbers of included trials and of interpolated channels. Critically, variability resulted in differences in the final ERP morphology and in the statistical results of the target ERP that each editor obtained. We also analyzed sources of disagreement by estimating the EEG characteristics that each human editor considered for accepting an ERP trial. In sum, our study reveals significant variability in ERP data editing pipelines, which has important consequences for the final ERP results. These findings represent an important step toward developing best practices for ERP editing methods in infancy research.

Kinematic boundary cues modulate 12-month-old infants' segmentation of action sequences: An ERP study

Human infants can segment action sequences into their constituent actions already during the first year of life. However, work to date has almost exclusively examined the role of infants' conceptual knowledge of actions and their outcomes in driving this segmentation. The present study examined electrophysiological correlates of infants' processing of lower-level perceptual cues that signal a boundary between two actions of an action sequence. Specifically, we tested the effect of kinematic boundary cues (pre-boundary lengthening and pause) on 12-month-old infants' (N = 27) processing of a sequence of three arbitrary actions, performed by an animated figure. Using the Event-Related Potential (ERP) approach, evidence of a positivity following the onset of the boundary cues was found, in line with previous work that has found an ERP positivity (Closure Positive Shift, CPS) related to boundary processing in auditory stimuli and action sequences in adults. Moreover, an ERP negativity (Negative Central, Nc) indicated that infants' encoding of the post-boundary action was modulated by the presence or absence of prior boundary cues. We therefore conclude that 12-month-old infants are sensitive to lower-level perceptual kinematic boundary cues, which can support segmentation of a continuous stream of movement into individual action units.

Learning non-adjacent rules and non-adjacent dependencies from human actions in 9-month-old infants

Seven month old infants can learn simple repetition patterns, such as we-fo-we, and generalize the rules to sequences of new syllables, such as ga-ti-ga. However, repetition rule learning in visual sequences seems more challenging, leading some researchers to claim that this type of rule learning applies preferentially to communicative stimuli. Here we demonstrate that 9-month-old infants can learn repetition rules in sequences of non-communicative dynamic human actions. We also show that when primed with these non-adjacent repetition patterns, infants can learn non-adjacent dependencies that involve memorizing the dependencies between specific human actions-patterns that prior research has shown to be difficult for infants in the visual domain and in speech. We discuss several possible mechanisms that account for the apparent advantage stimuli involving human action sequences has over other kinds of stimuli in supporting non-adjacent dependency learning. We also discuss possible implications for theories of language acquisition.

Learning From Others: The Effects of Agency on Event Memory in Young Children

Little is known about the influence of social context on children's event memory. Across four studies, we examined whether learning that could occur in the absence of a person was more robust when a person was present. Three-year-old children (N = 125) viewed sequential events that either included or excluded an acting agent. In Experiment 1, children who viewed an agent recalled more than children who did not. Experiments 2a and 2b utilized an eye tracker to demonstrate this effect was not due to differences in attention. Experiment 3 used a combined behavioral and event-related potential paradigm to show that condition effects were present in memory-related components. These converging results indicate a particular role for social knowledge in supporting memory for events.

Event Perception and Memory

Events make up much of our lived experience, and the perceptual mechanisms that represent events in experience have pervasive effects on action control, language use, and remembering. Event representations in both perception and memory have rich internal structure and connections one to another, and both are heavily informed by knowledge accumulated from previous experiences. Event perception and memory have been identified with specific computational and neural mechanisms, which show protracted development in childhood and are affected by language use, expertise, and brain disorders and injuries. Current theoretical approaches focus on the mechanisms by which events are segmented from ongoing experience, and emphasize the common coding of events for perception, action, and memory. Abetted by developments in eye-tracking, neuroimaging, and computer science, research on event perception and memory is moving from small-scale laboratory analogs to the complexity of events in the wild.

Tonality Tunes the Statistical Characteristics in Music: Computational Approaches on Statistical Learning

Statistical learning is a learning mechanism based on transition probability in sequences such as music and language. Recent computational and neurophysiological studies suggest that the statistical learning contributes to production, action, and musical creativity as well as prediction and perception. The present study investigated how statistical structure interacts with tonalities in music based on various-order statistical models. To verify this in all 24 major and minor keys, the transition probabilities of the sequences containing the highest pitches in Bach's Well-Tempered Clavier, which is a collection of two series (No. 1 and No. 2) of preludes and fugues in all of the 24 major and minor keys, were calculated based on nth-order Markov models. The transition probabilities of each sequence were compared among tonalities (major and minor), two series (No. 1 and No. 2), and music types (prelude and fugue). The differences in statistical characteristics between major and minor keys were detected in lower- but not higher-order models. The results also showed that statistical knowledge in music might be modulated by tonalities and composition periods. Furthermore, the principal component analysis detected the shared components of related keys, suggesting that the tonalities modulate statistical characteristics in music. The present study may suggest that there are at least two types of statistical knowledge in music that are interdependent on and independent of tonality, respectively.

Depth and the Uncertainty of Statistical Knowledge on Musical Creativity Fluctuate Over a Composer's Lifetime

Brain models music as a hierarchy of dynamical systems that encode probability distributions and complexity (i.e., entropy and uncertainty). Through musical experience over lifetime, a human is intrinsically motivated in optimizing the internalized probabilistic model for efficient information processing and the uncertainty resolution, which has been regarded as rewords. Human's behavior, however, appears to be not necessarily directing to efficiency but sometimes act inefficiently in order to explore a maximum rewards of uncertainty resolution. Previous studies suggest that the drive for novelty seeking behavior (high uncertain phenomenon) reflects human's curiosity, and that the curiosity rewards encourage humans to create and learn new regularities. That is to say, although brain generally minimizes uncertainty of music structure, we sometimes derive pleasure from music with uncertain structure due to curiosity for novelty seeking behavior by which we anticipate the resolution of uncertainty. Few studies, however, investigated how curiosity for uncertain and novelty seeking behavior modulates musical creativity. The present study investigated how the probabilistic model and the uncertainty in music fluctuate over a composer's lifetime (all of the 32 piano sonatas by Ludwig van Beethoven). In the late periods of the composer's lifetime, the transitional probabilities (TPs) of sequential patterns that ubiquitously appear in all of his music (familiar phrase) were decreased, whereas the uncertainties of the whole structure were increased. Furthermore, these findings were prominent in higher-, rather than lower-, order models of TP distribution. This may suggest that the higher-order probabilistic model is susceptible to experience and psychological phenomena over the composer's lifetime. The present study first suggested the fluctuation of uncertainty of musical structure over a composer's lifetime. It is suggested that human's curiosity for uncertain and novelty seeking behavior may modulate optimization and creativity in human's brain.

Entropy, Uncertainty, and the Depth of Implicit Knowledge on Musical Creativity: Computational Study of Improvisation in Melody and Rhythm

Recent neurophysiological and computational studies have proposed the hypothesis that our brain automatically codes the nth-order transitional probabilities (TPs) embedded in sequential phenomena such as music and language (i.e., local statistics in nth-order level), grasps the entropy of the TP distribution (i.e., global statistics), and predicts the future state based on the internalized nth-order statistical model. This mechanism is called statistical learning (SL). SL is also believed to contribute to the creativity involved in musical improvisation. The present study examines the interactions among local statistics, global statistics, and different levels of orders (mutual information) in musical improvisation interact. Interactions among local statistics, global statistics, and hierarchy were detected in higher-order SL models of pitches, but not lower-order SL models of pitches or SL models of rhythms. These results suggest that the information-theoretical phenomena of local and global statistics in each order may be reflected in improvisational music. The present study proposes novel methodology to evaluate musical creativity associated with SL based on information theory.

Shared neural and cognitive mechanisms in action and language: The multiscale information transfer framework

This review compares how humans process action and language sequences produced by other humans. On the one hand, we identify commonalities between action and language processing in terms of cognitive mechanisms (e.g., perceptual segmentation, predictive processing, integration across multiple temporal scales), neural resources (e.g., the left inferior frontal cortex), and processing algorithms (e.g., comprehension based on changes in signal entropy). On the other hand, drawing on sign language with its particularly strong motor component, we also highlight what differentiates (both oral and signed) linguistic communication from nonlinguistic action sequences. We propose the multiscale information transfer framework (MSIT) as a way of integrating these insights and highlight directions into which future empirical research inspired by the MSIT framework might fruitfully evolve. This article is categorized under: Psychology > Language Linguistics > Language in Mind and Brain Psychology > Motor Skill and Performance Psychology > Prediction.

Neurophysiological Markers of Statistical Learning in Music and Language: Hierarchy, Entropy, and Uncertainty

Statistical learning (SL) is a method of learning based on the transitional probabilities embedded in sequential phenomena such as music and language. It has been considered an implicit and domain-general mechanism that is innate in the human brain and that functions independently of intention to learn and awareness of what has been learned. SL is an interdisciplinary notion that incorporates information technology, artificial intelligence, musicology, and linguistics, as well as psychology and neuroscience. A body of recent study has suggested that SL can be reflected in neurophysiological responses based on the framework of information theory. This paper reviews a range of work on SL in adults and children that suggests overlapping and independent neural correlations in music and language, and that indicates disability of SL. Furthermore, this article discusses the relationships between the order of transitional probabilities (TPs) (i.e., hierarchy of local statistics) and entropy (i.e., global statistics) regarding SL strategies in human's brains; claims importance of information-theoretical approaches to understand domain-general, higher-order, and global SL covering both real-world music and language; and proposes promising approaches for the application of therapy and pedagogy from various perspectives of psychology, neuroscience, computational studies, musicology, and linguistics.

Infants' Motor Proficiency and Statistical Learning for Actions

Prior research has shown that infants learn statistical regularities in action sequences better than they learn non-action event sequences. This is consistent with current theories claiming that the same mechanism guides action observation and action execution. The current eye-tracking study tested the prediction, based on these theories, that infants' ability to learn statistical regularities in action sequences is modulated by their own motor abilities. Eight- to eleven-month-old infants observed an action sequence containing two deterministic action pairs (i.e., action A always followed by action B) embedded within an otherwise random sequence. One pair was performed with a whole-hand grasp. The second pair was performed with a pincer grasp, a fine motor skill that emerges around 9 months of age. Infants were then categorized into groups according to which grasp was dominant in their motor repertoire. Predictive looks to correct upcoming actions during the deterministic pairs were analyzed to measure whether infants learned and anticipated the sequence regularities. Findings indicate that infants learned the statistical regularities: across motor groups, they made more correct than incorrect predictive fixations to upcoming actions. Overall, learning was not significantly modulated by their dominant grasping abilities. However, infants with a dominant pincer grasp showed an earlier increase in correct predictions for the pincer grasp pair and not the whole-hand grasp. Likewise, infants with a dominant whole-hand grasp showed an early increase in correct predictions for the pair performed with a whole-hand grasp, and not the pincer grasp. Together, these findings suggest that infants' ability to learn action sequences is facilitated when the observed action matches their own action repertoire. However, findings cannot be explained entirely by motor accounts, as infants also learned the actions less congruent with their own abilities. Findings are discussed in terms of the interplay between the motor system and additional non-motor resources during the acquisition of new motor skills in infancy.