Neural tracking in infants - An analytical tool for multisensory social processing in development

Effects and prediction of cognitive load on encoding model of brain response to auditory and linguistic stimuli in educational multimedia

Multimedia is extensively used for educational purposes. However, certain types of multimedia lack proper design, which could impose a cognitive load on the user. Therefore, it is essential to predict cognitive load and understand how it impairs brain functioning. Participants watched a version of educational multimedia that applied Mayer's principles, followed by a version that did not. Meanwhile, their electroencephalography (EEG) was recorded. Subsequently, they participated in a post-test and completed a self-reported cognitive load questionnaire. The audio envelope and word frequency were extracted from the multimedia, and the temporal response functions (TRFs) were obtained using a linear encoding model. We observed that the behavioral data are different between the two groups and the TRFs of the two multimedia versions were different. We saw changes in the amplitude and latencies of both early and late components. In addition, correlations were found between behavioral data and the amplitude and latencies of TRF components. Cognitive load decreased participants' attention to the multimedia, and semantic processing of words also occurred with a delay and smaller amplitude. Hence, encoding models provide insights into the temporal and spatial mapping of the cognitive load activity, which could help us detect and reduce cognitive load in potential environments such as educational multimedia or simulators for different purposes.

Decoding speech information from EEG data with 4-, 7- and 11-month-old infants: Using convolutional neural network, mutual information-based and backward linear models

BACKGROUND

Computational models that successfully decode neural activity into speech are increasing in the adult literature, with convolutional neural networks (CNNs), backward linear models, and mutual information (MI) models all being applied to neural data in relation to speech input. This is not the case in the infant literature.

NEW METHOD

Three different computational models, two novel for infants, were applied to decode low-frequency speech envelope information. Previously-employed backward linear models were compared to novel CNN and MI-based models. Fifty infants provided EEG recordings when aged 4, 7, and 11 months, while listening passively to natural speech (sung or chanted nursery rhymes) presented by video with a female singer.

RESULTS

Each model computed speech information for these nursery rhymes in two different low-frequency bands, delta and theta, thought to provide different types of linguistic information. All three models demonstrated significant levels of performance for delta-band neural activity from 4 months of age, with two of three models also showing significant performance for theta-band activity. All models also demonstrated higher accuracy for the delta-band neural responses. None of the models showed developmental (age-related) effects.

COMPARISONS WITH EXISTING METHODS

The data demonstrate that the choice of algorithm used to decode speech envelope information from neural activity in the infant brain determines the developmental conclusions that can be drawn.

CONCLUSIONS

The modelling shows that better understanding of the strengths and weaknesses of each modelling approach is fundamental to improving our understanding of how the human brain builds a language system.

Sing to me, baby: Infants show neural tracking and rhythmic movements to live and dynamic maternal singing

Infant-directed singing has unique acoustic characteristics that may allow even very young infants to respond to the rhythms carried through the caregiver's voice. The goal of this study was to examine neural and movement responses to live and dynamic maternal singing in 7-month-old infants and their relation to linguistic development. In total, 60 mother-infant dyads were observed during two singing conditions (playsong and lullaby). In Study 1 (n = 30), we measured infant EEG and used an encoding approach utilizing ridge regressions to measure neural tracking. In Study 2 (n =40), we coded infant rhythmic movements. In both studies, we assessed children's vocabulary when they were 20 months old. In Study 1, we found above-threshold neural tracking of maternal singing, with superior tracking of lullabies than playsongs. We also found that the acoustic features of infant-directed singing modulated tracking. In Study 2, infants showed more rhythmic movement to playsongs than lullabies. Importantly, neural coordination (Study 1) and rhythmic movement (Study 2) to playsongs were positively related to infants' expressive vocabulary at 20 months. These results highlight the importance of infants' brain and movement coordination to their caregiver's musical presentations, potentially as a function of musical variability.

Phonological acquisition depends on the timing of speech sounds: Deconvolution EEG modeling across the first five years

The late development of fast brain activity in infancy restricts initial processing abilities to slow information. Nevertheless, infants acquire the short-lived speech sounds of their native language during their first year of life. Here, we trace the early buildup of the infant phoneme inventory with naturalistic electroencephalogram. We apply the recent method of deconvolution modeling to capture the emergence of the feature-based phoneme representation that is known to govern speech processing in the mature brain. Our cross-sectional analysis uncovers a gradual developmental increase in neural responses to native phonemes. Critically, infants appear to acquire those phoneme features first that extend over longer time intervals-thus meeting infants' slow processing abilities. Shorter-lived phoneme features are added stepwise, with the shortest acquired last. Our study shows that the ontogenetic acceleration of electrophysiology shapes early language acquisition by determining the duration of the acquired units.

Early social communication through music: State of the art and future perspectives

A growing body of research shows that the universal capacity for music perception and production emerges early in development. Possibly building on this predisposition, caregivers around the world often communicate with infants using songs or speech entailing song-like characteristics. This suggests that music might be one of the earliest developing and most accessible forms of interpersonal communication, providing a platform for studying early communicative behavior. However, little research has examined music in truly communicative contexts. The current work aims to facilitate the development of experimental approaches that rely on dynamic and naturalistic social interactions. We first review two longstanding lines of research that examine musical interactions by focusing either on the caregiver or the infant. These include defining the acoustic and non-acoustic features that characterize infant-directed (ID) music, as well as behavioral and neurophysiological research examining infants' processing of musical timing and pitch. Next, we review recent studies looking at early musical interactions holistically. This research focuses on how caregivers and infants interact using music to achieve co-regulation, mutual engagement, and increase affiliation and prosocial behavior. We conclude by discussing methodological, technological, and analytical advances that might empower a comprehensive study of musical communication in early childhood.

Proactive or reactive? Neural oscillatory insight into the leader-follower dynamics of early infant-caregiver interaction

We know that infants' ability to coordinate attention with others toward the end of the first year is fundamental to language acquisition and social cognition. Yet, we understand little about the neural and cognitive mechanisms driving infant attention in shared interaction: do infants play a proactive role in creating episodes of joint attention? Recording electroencephalography (EEG) from 12-mo-old infants while they engaged in table-top play with their caregiver, we examined the communicative behaviors and neural activity preceding and following infant- vs. adult-led joint attention. Infant-led episodes of joint attention appeared largely reactive: they were not associated with increased theta power, a neural marker of endogenously driven attention, and infants did not increase their ostensive signals before the initiation. Infants were, however, sensitive to whether their initiations were responded to. When caregivers joined their attentional focus, infants showed increased alpha suppression, a pattern of neural activity associated with predictive processing. Our results suggest that at 10 to 12 mo, infants are not routinely proactive in creating joint attention episodes yet. They do, however, anticipate behavioral contingency, a potentially foundational mechanism for the emergence of intentional communication.

Behaviour recommendations with a deep learning model and genetic algorithm for health debt characterisation

Human behaviour is a dense longitudinal multi-featured measure that directly impacts the health of individuals in the short and long terms. Therefore, issues usually emerge from the insistence on performing risky behaviours, such as smoking or eating fast foods, which continuously increase the gap between current and beneficial health states. This paper introduces the term "health debt" as an economic metaphor to represent the quantification of this gap in domains such as sleep, contributing to physical and mental health states. Then, we present a theoretical framework that relies on behaviour change recommendations to quantify this debt. The practical instantiation of this framework relies on passively assessed sleep related data via personal wearable devices, and uses of an attention-based predictive model as the fitness function of a genetic algorithm that acts as a recommender. We evaluate this proposal by means of a case example aimed at improving the sleep duration of individuals. Results show, for example, that the use of individual rather than generic datasets produces more accurate models. At the same time, the use of constraints on the variability of behaviours features generates more feasible recommendations. These foundations open new research opportunities to support the adoption of preventive medicine based on longitudinal wearable passive data analysis.

Three simple steps to improve the interpretability of EEG-SVM studies

Machine-learning systems that classify electroencephalography (EEG) data offer important perspectives for the diagnosis and prognosis of a wide variety of neurological and psychiatric conditions, but their clinical adoption remains low. We propose here that much of the difficulties translating EEG-machine-learning research to the clinic result from consistent inaccuracies in their technical reporting, which severely impair the interpretability of their often-high claims of performance. Taking example from a major class of machine-learning algorithms used in EEG research, the support-vector machine (SVM), we highlight three important aspects of model development (normalization, hyperparameter optimization, and cross-validation) and show that, while these three aspects can make or break the performance of the system, they are left entirely undocumented in a shockingly vast majority of the research literature. Providing a more systematic description of these aspects of model development constitute three simple steps to improve the interpretability of EEG-SVM research and, in fine, its clinical adoption.

Studying the Developing Brain in Real-World Contexts: Moving From Castles in the Air to Castles on the Ground

Most current research in cognitive neuroscience uses standardized non-ecological experiments to study the developing brain. But these approaches do a poor job of mimicking the real-world, and thus can only provide a distorted picture of how cognitive operations and brain development unfold outside of the lab. Here we consider future research avenues which may lead to a better appreciation of how developing brains dynamically interact with a complex real-world environment, and how cognition develops over time. We raise several problems faced by current mainstream methods in the field, before briefly reviewing novel promising approaches that alleviate some of these issues. First, we consider research that examines perception by measuring entrainment between brain activity and temporal patterns in naturalistic stimuli. Second, we consider research that examines our ability to parse our continuous experience into discrete events, and how this ability develops over time. Third, we consider the role of children as active agents in selecting what they sample from the environment from one moment to the next. Fourth, we consider new approaches that measure how mutual influences between children and others are instantiated in suprapersonal brain networks. Finally, we discuss how we may reduce adult biases when designing developmental studies. Together, these approaches have great potential to further our understanding of how the developing brain learns to process information, and to control complex real-world behaviors.

The power of rhythms: how steady-state evoked responses reveal early neurocognitive development

Electroencephalography (EEG) is a non-invasive and painless recording of cerebral activity, particularly well-suited for studying young infants, allowing the inspection of cerebral responses in a constellation of different ways. Of particular interest for developmental cognitive neuroscientists is the use of rhythmic stimulation, and the analysis of steady-state evoked potentials (SS-EPs) - an approach also known as frequency tagging. In this paper we rely on the existing SS-EP early developmental literature to illustrate the important advantages of SS-EPs for studying the developing brain. We argue that (1) the technique is both objective and predictive: the response is expected at the stimulation frequency (and/or higher harmonics), (2) its high spectral specificity makes the computed responses particularly robust to artifacts, and (3) the technique allows for short and efficient recordings, compatible with infants' limited attentional spans. We additionally provide an overview of some recent inspiring use of the SS-EP technique in adult research, in order to argue that (4) the SS-EP approach can be implemented creatively to target a wide range of cognitive and neural processes. For all these reasons, we expect SS-EPs to play an increasing role in the understanding of early cognitive processes. Finally, we provide practical guidelines for implementing and analyzing SS-EP studies.

Cortical Tracking of Sung Speech in Adults vs Infants: A Developmental Analysis

Here we duplicate a neural tracking paradigm, previously published with infants (aged 4 to 11 months), with adult participants, in order to explore potential developmental similarities and differences in entrainment. Adults listened and watched passively as nursery rhymes were sung or chanted in infant-directed speech. Whole-head EEG (128 channels) was recorded, and cortical tracking of the sung speech in the delta (0.5–4 Hz), theta (4–8 Hz) and alpha (8–12 Hz) frequency bands was computed using linear decoders (multivariate Temporal Response Function models, mTRFs). Phase-amplitude coupling (PAC) was also computed to assess whether delta and theta phases temporally organize higher-frequency amplitudes for adults in the same pattern as found in the infant brain. Similar to previous infant participants, the adults showed significant cortical tracking of the sung speech in both delta and theta bands. However, the frequencies associated with peaks in stimulus-induced spectral power (PSD) in the two populations were different. PAC was also different in the adults compared to the infants. PAC was stronger for theta- versus delta- driven coupling in adults but was equal for delta- versus theta-driven coupling in infants. Adults also showed a stimulus-induced increase in low alpha power that was absent in infants. This may suggest adult recruitment of other cognitive processes, possibly related to comprehension or attention. The comparative data suggest that while infant and adult brains utilize essentially the same cortical mechanisms to track linguistic input, the operation of and interplay between these mechanisms may change with age and language experience.

Natural Infant-Directed Speech Facilitates Neural Tracking of Prosody

Infants prefer to be addressed with infant-directed speech (IDS). IDS benefits language acquisition through amplified low-frequency amplitude modulations. It has been reported that this amplification increases electrophysiological tracking of IDS compared to adult-directed speech (ADS). It is still unknown which particular frequency band triggers this effect. Here, we compare tracking at the rates of syllables and prosodic stress, which are both critical to word segmentation and recognition. In mother-infant dyads (n=30), mothers described novel objects to their 9-month-olds while infants' EEG was recorded. For IDS, mothers were instructed to speak to their children as they typically do, while for ADS, mothers described the objects as if speaking with an adult. Phonetic analyses confirmed that pitch features were more prototypically infant-directed in the IDS-condition compared to the ADS-condition. Neural tracking of speech was assessed by speech-brain coherence, which measures the synchronization between speech envelope and EEG. Results revealed significant speech-brain coherence at both syllabic and prosodic stress rates, indicating that infants track speech in IDS and ADS at both rates. We found significantly higher speech-brain coherence for IDS compared to ADS in the prosodic stress rate but not the syllabic rate. This indicates that the IDS benefit arises primarily from enhanced prosodic stress. Thus, neural tracking is sensitive to parents' speech adaptations during natural interactions, possibly facilitating higher-level inferential processes such as word segmentation from continuous speech.