Temporal dynamics of visual representations in the infant brain

Infant neuroscience: how to measure brain activity in the youngest minds

The functional properties of the infant brain are poorly understood. Recent advances in cognitive neuroscience are opening new avenues for measuring brain activity in human infants. These include novel uses of existing technologies such as electroencephalography (EEG) and magnetoencephalography (MEG), the availability of newer technologies including functional near-infrared spectroscopy (fNIRS) and optically pumped magnetometry (OPM), and innovative applications of functional magnetic resonance imaging (fMRI) in awake infants during cognitive tasks. In this review article we catalog these available non-invasive methods, discuss the challenges and opportunities encountered when applying them to human infants, and highlight the potential they may ultimately hold for advancing our understanding of the youngest minds.

Decoding speech sounds from neurophysiological data: Practical considerations and theoretical implications

Machine learning techniques have proven to be a useful tool in cognitive neuroscience. However, their implementation in scalp-recorded electroencephalography (EEG) is relatively limited. To address this, we present three analyses using data from a previous study that examined event-related potential (ERP) responses to a wide range of naturally-produced speech sounds. First, we explore which features of the EEG signal best maximize machine learning accuracy for a voicing distinction, using a support vector machine (SVM). We manipulate three dimensions of the EEG signal as input to the SVM: number of trials averaged, number of time points averaged, and polynomial fit. We discuss the trade-offs in using different feature sets and offer some recommendations for researchers using machine learning. Next, we use SVMs to classify specific pairs of phonemes, finding that we can detect differences in the EEG signal that are not otherwise detectable using conventional ERP analyses. Finally, we characterize the timecourse of phonetic feature decoding across three phonological dimensions (voicing, manner of articulation, and place of articulation), and find that voicing and manner are decodable from neural activity, whereas place of articulation is not. This set of analyses addresses both practical considerations in the application of machine learning to EEG, particularly for speech studies, and also sheds light on current issues regarding the nature of perceptual representations of speech.

Longitudinal development of category representations in ventral temporal cortex predicts word and face recognition

Regions in ventral temporal cortex that are involved in visual recognition of categories like words and faces undergo differential development during childhood. However, categories are also represented in distributed responses across high-level visual cortex. How distributed category representations develop and if this development relates to behavioral changes in recognition remains largely unknown. Here, we used functional magnetic resonance imaging to longitudinally measure the development of distributed responses across ventral temporal cortex to 10 categories in school-age children over several years. Our results reveal both strengthening and weakening of category representations with age, which was mainly driven by changes across category-selective voxels. Representations became particularly more distinct for words in the left hemisphere and for faces bilaterally. Critically, distinctiveness for words and faces across category-selective voxels in left and right lateral ventral temporal cortex, respectively, predicted individual children's word and face recognition performance. These results suggest that the development of distributed representations in ventral temporal cortex has behavioral ramifications and advance our understanding of prolonged cortical development during childhood.

Visual Representations: Insights from Neural Decoding

Patterns of brain activity contain meaningful information about the perceived world. Recent decades have welcomed a new era in neural analyses, with computational techniques from machine learning applied to neural data to decode information represented in the brain. In this article, we review how decoding approaches have advanced our understanding of visual representations and discuss efforts to characterize both the complexity and the behavioral relevance of these representations. We outline the current consensus regarding the spatiotemporal structure of visual representations and review recent findings that suggest that visual representations are at once robust to perturbations, yet sensitive to different mental states. Beyond representations of the physical world, recent decoding work has shone a light on how the brain instantiates internally generated states, for example, during imagery and prediction. Going forward, decoding has remarkable potential to assess the functional relevance of visual representations for human behavior, reveal how representations change across development and during aging, and uncover their presentation in various mental disorders.

Distributed and hierarchical neural encoding of multidimensional biological motion attributes in the human brain

The human visual system can efficiently extract distinct physical, biological, and social attributes (e.g. facing direction, gender, and emotional state) from biological motion (BM), but how these attributes are encoded in the brain remains largely unknown. In the current study, we used functional magnetic resonance imaging to investigate this issue when participants viewed multidimensional BM stimuli. Using multiple regression representational similarity analysis, we identified distributed brain areas, respectively, related to the processing of facing direction, gender, and emotional state conveyed by BM. These brain areas are governed by a hierarchical structure in which the respective neural encoding of facing direction, gender, and emotional state is modulated by each other in descending order. We further revealed that a portion of the brain areas identified in representational similarity analysis was specific to the neural encoding of each attribute and correlated with the corresponding behavioral results. These findings unravel the brain networks for encoding BM attributes in consideration of their interactions, and highlight that the processing of multidimensional BM attributes is recurrently interactive.

Capacity for movement is an organisational principle in object representations

The ability to perceive moving objects is crucial for threat identification and survival. Recent neuroimaging evidence has shown that goal-directed movement is an important element of object processing in the brain. However, prior work has primarily used moving stimuli that are also animate, making it difficult to disentangle the effect of movement from aliveness or animacy in representational categorisation. In the current study, we investigated the relationship between how the brain processes movement and aliveness by including stimuli that are alive but still (e.g., plants), and stimuli that are not alive but move (e.g., waves). We examined electroencephalographic (EEG) data recorded while participants viewed static images of moving or non-moving objects that were either natural or artificial. Participants classified the images according to aliveness, or according to capacity for movement. Movement explained significant variance in the neural data over and above that of aliveness, showing that capacity for movement is an important dimension in the representation of visual objects in humans.

Cognitive development: Looking for perceptual awareness in human infants

The extent to which young human infants are conscious, in the sense of being perceptually aware of their environment, has been long debated. A new study has revealed that infants do exhibit a key signature of consciousness - the attentional blink - but this early consciousness changes with age.

A practical guide to applying machine learning to infant EEG data

Electroencephalography (EEG) has been widely adopted by the developmental cognitive neuroscience community, but the application of machine learning (ML) in this domain lags behind adult EEG studies. Applying ML to infant data is particularly challenging due to the low number of trials, low signal-to-noise ratio, high inter-subject variability, and high inter-trial variability. Here, we provide a step-by-step tutorial on how to apply ML to classify cognitive states in infants. We describe the type of brain attributes that are widely used for EEG classification and also introduce a Riemannian geometry based approach for deriving connectivity estimates that account for inter-trial and inter-subject variability. We present pipelines for learning classifiers using trials from a single infant and from multiple infants, and demonstrate the application of these pipelines on a standard infant EEG dataset of forty 12-month-old infants collected under an auditory oddball paradigm. While we classify perceptual states induced by frequent versus rare stimuli, the presented pipelines can be easily adapted for other experimental designs and stimuli using the associated code that we have made publicly available.

Time-resolved multivariate pattern analysis of infant EEG data: A practical tutorial

Time-resolved multivariate pattern analysis (MVPA), a popular technique for analyzing magneto- and electro-encephalography (M/EEG) neuroimaging data, quantifies the extent and time-course by which neural representations support the discrimination of relevant stimuli dimensions. As EEG is widely used for infant neuroimaging, time-resolved MVPA of infant EEG data is a particularly promising tool for infant cognitive neuroscience. MVPA has recently been applied to common infant imaging methods such as EEG and fNIRS. In this tutorial, we provide and describe code to implement time-resolved, within-subject MVPA with infant EEG data. An example implementation of time-resolved MVPA based on linear SVM classification is described, with accompanying code in Matlab and Python. Results from a test dataset indicated that in both infants and adults this method reliably produced above-chance accuracy for classifying stimuli images. Extensions of the classification analysis are presented including both geometric- and accuracy-based representational similarity analysis, implemented in Python. Common choices of implementation are presented and discussed. As the amount of artifact-free EEG data contributed by each participant is lower in studies of infants than in studies of children and adults, we also explore and discuss the impact of varying participant-level inclusion thresholds on resulting MVPA findings in these datasets.

Peak selection and latency jitter correction in developmental event-related potentials

Event-related potentials (ERPs) provide great insight into neural responses, yet developmental ERP work is plagued with inconsistent approaches to identifying and quantifying component latency. In this analytical review, we describe popular conventions for the selection of time windows for ERP analysis and assert that a data-driven strategy should be applied to the identification of component latency within individual participants' data. This may overcome weaknesses of more general approaches to peak selection; however, it does not account for trial-by-trial variability within a participant. This issue, known as ERP latency jitter, may blur the average ERP, misleading the interpretation of neural mechanisms. Recently, the ReSync MATLAB toolbox has been made available for correction of latency jitter. Although not created specifically for pediatric ERP data, this approach can be adapted for developmental researchers. We have demonstrated the use of the ReSync toolbox with individual infant and child datasets to illustrate its utility. Details about our peak detection script and the ReSync toolbox are provided. The adoption of data processing procedures that allow for accurate, study-specific component selection and reduce trial-by-trial asynchrony strengthens developmental ERP research by decreasing noise included in ERP analyses and improving the representation of the neural response.