Multisensory Processes: A Balancing Act across the Lifespan

Impoverished Inhibitory Control Exacerbates Multisensory Impairments in Older Fallers

Impaired temporal perception of multisensory cues is a common phenomenon observed in older adults that can lead to unreliable percepts of the external world. For instance, the sound induced flash illusion (SIFI) can induce an illusory percept of a second flash by presenting a beep close in time to an initial flash-beep pair. Older adults that have enhanced susceptibility to a fall demonstrate significantly stronger illusion percepts during the SIFI task compared to those older adults without any history of falling. We hypothesize that a global inhibitory deficit may be driving the impairments across both postural stability and multisensory function in older adults with a fall history (FH). We investigated oscillatory activity and perceptual performance during the SIFI task, to understand how active sensory processing, measured by gamma (30–80 Hz) power, was regulated by alpha activity (8–13 Hz), oscillations that reflect inhibitory control. Compared to young adults (YA), the FH and non-faller (NF) groups demonstrated enhanced susceptibility to the SIFI. Further, the FH group had significantly greater illusion strength compared to the NF group. The FH group also showed significantly impaired performance relative to YA during congruent trials (2 flash-beep pairs resulting in veridical perception of 2 flashes). In illusion compared to non-illusion trials, the NF group demonstrated reduced alpha power (or diminished inhibitory control). Relative to YA and NF, the FH group showed reduced phase-amplitude coupling between alpha and gamma activity in non-illusion trials. This loss of inhibitory capacity over sensory processing in FH compared to NF suggests a more severe change than that consequent of natural aging.

Directional biases in whole hand motion perception revealed by mid-air tactile stimulation

Many emerging technologies are attempting to leverage the tactile domain to convey complex spatiotemporal information translated directly from the visual domain, such as shape and motion. Despite the intuitive appeal of touch for communication, we do not know to what extent the hand can substitute for the retina in this way. Here we ask whether the tactile system can be used to perceive complex whole hand motion stimuli, and whether it exhibits the same kind of established perceptual biases as reported in the visual domain. Using ultrasound stimulation, we were able to project complex moving dot percepts onto the palm in mid-air, over 30 cm above an emitter device. We generated dot kinetogram stimuli involving motion in three different directional axes ('Horizontal', 'Vertical', and 'Oblique') on the ventral surface of the hand. Using Bayesian statistics, we found clear evidence that participants were able to discriminate tactile motion direction. Furthermore, there was a marked directional bias in motion perception: participants were both better and more confident at discriminating motion in the vertical and horizontal axes of the hand, compared to those stimuli moving obliquely. This pattern directly mirrors the perceptional biases that have been robustly reported in the visual field, termed the 'Oblique Effect'. These data demonstrate the existence of biases in motion perception that transcend sensory modality. Furthermore, we extend the Oblique Effect to a whole hand scale, using motion stimuli presented on the broad and relatively low acuity surface of the palm, away from the densely innervated and much studied fingertips. These findings highlight targeted ultrasound stimulation as a versatile method to convey potentially complex spatial and temporal information without the need for a user to wear or touch a device.

Multisensory Facilitation of Working Memory Training

Research suggests that memorization of multisensory stimuli benefits performance compared to memorization of unisensory stimuli; however, little is known about multisensory facilitation in the context of working memory (WM) training and transfer. To investigate this, 240 adults were randomly assigned to an N-back training task that consisted of visual-only stimuli, alternating visual and auditory blocks, or audio-visual (multisensory) stimuli, or to a passive control group. Participants in the active groups completed 13 sessions of N-back training (6.7 hours in total) and all groups completed a battery of WM tasks: untrained N-back tasks, Corsi Blocks, Sequencing, and Symmetry Span. The Multisensory group showed similar training N-level gain compared to the Visual Only group, and both of these groups outperformed the Alternating group on the training task. As expected, all three active groups significantly improved on untrained visual N-back tasks compared to the Control group. In contrast, the Multisensory group showed significantly greater gains on the Symmetry Span task and to a certain extent on the Sequencing task compared to other groups. These results tentatively suggest that incorporating multisensory objects in a WM training protocol can benefit performance on the training task and potentially facilitate transfer to complex WM span tasks.

A reachable probability approach for the analysis of spatio-temporal dynamics in the human functional network

The dynamic architecture of the human brain has been consistently observed. However, there is still limited modeling work to elucidate how neuronal circuits are hierarchically and flexibly organized in functional systems. Here we proposed a reachable probability approach based on non-homogeneous Markov chains, to characterize all possible connectivity flows and the hierarchical structure of brain functional systems at the dynamic level. We proved at the theoretical level the convergence of the functional brain network system, and demonstrated that this approach is able to detect network steady states across connectivity structure, particularly in areas of the default mode network. We further explored the dynamically hierarchical functional organization centered at the primary sensory cortices. We observed smaller optimal reachable steps to their local functional regions, and differentiated patterns in larger optimal reachable steps for primary perceptual modalities. The reachable paths with the largest and second largest transition probabilities between primary sensory seeds via multisensory integration regions were also tracked to explore the flexibility and plasticity of the multisensory integration. The present work provides a novel approach to depict both the stable and flexible hierarchical connectivity organization of the human brain.

Perception and Motion in Real and Virtual Environments: A Narrative Review of Autism Spectrum Disorders

Atypical sensorimotor developmental trajectories greatly contribute to the profound heterogeneity that characterizes Autism Spectrum Disorders (ASD). Individuals with ASD manifest deviations in sensorimotor processing with early markers in the use of sensory information coming from both the external world and the body, as well as motor difficulties. The cascading effect of these impairments on the later development of higher-order abilities (e.g., executive functions and social communication) underlines the need for interventions that focus on the remediation of sensorimotor integration skills. One of the promising technologies for such stimulation is Immersive Virtual Reality (IVR). In particular, head-mounted displays (HMDs) have unique features that fully immerse the user in virtual realities which disintegrate and otherwise manipulate multimodal information. The contribution of each individual sensory input and of multisensory integration to perception and motion can be evaluated and addressed according to a user’s clinical needs. HMDs can therefore be used to create virtual environments aimed at improving people’s sensorimotor functioning, with strong potential for individualization for users. Here we provide a narrative review of the sensorimotor atypicalities evidenced by children and adults with ASD, alongside some specific relevant features of IVR technology. We discuss how individuals with ASD may interact differently with IVR versus real environments on the basis of their specific atypical sensorimotor profiles and describe the unique potential of HMD-delivered immersive virtual environments to this end.

Wiring of higher-order cortical areas: Spatiotemporal development of cortical hierarchy

A hierarchical development of cortical areas was suggested over a century ago, but the diversity and complexity of cortical hierarchy properties have so far prevented a formal demonstration. The aim of this review is to clarify the similarities and differences in the developmental processes underlying cortical development of primary and higher-order areas. We start by recapitulating the historical and recent advances underlying the biological principle of cortical hierarchy in adults. We then revisit the arguments for a hierarchical maturation of cortical areas, and further integrate the principles of cortical areas specification during embryonic and postnatal development. We highlight how the dramatic expansion in cortical size might have contributed to the increased number of association areas sustaining cognitive complexification in evolution. Finally, we summarize the recent observations of an alteration of cortical hierarchy in neuropsychiatric disorders and discuss their potential developmental origins.

Typical Crossmodal Numerosity Perception in Preterm Newborns

Premature birth is associated with a high risk of damage in the parietal cortex, a key area for numerical and non-numerical magnitude perception and mathematical reasoning. Children born preterm have higher rates of learning difficulties for school mathematics. In this study, we investigated how preterm newborns (born at 28-34 weeks of gestation age) and full-term newborns respond to visual numerosity after habituation to auditory stimuli of different numerosities. The results show that the two groups have a similar preferential looking response to visual numerosity, both preferring the incongruent set after crossmodal habituation. These results suggest that the numerosity system is resistant to prematurity.

The multisensory cocktail party problem in adults: Perceptual segregation of talking faces on the basis of audiovisual temporal synchrony

Social interactions often involve a cluttered multisensory scene consisting of multiple talking faces. We investigated whether audiovisual temporal synchrony can facilitate perceptual segregation of talking faces. Participants either saw four identical or four different talking faces producing temporally jittered versions of the same visible speech utterance and heard the audible version of the same speech utterance. The audible utterance was either synchronized with the visible utterance produced by one of the talking faces or not synchronized with any of them. Eye tracking indicated that participants exhibited a marked preference for the synchronized talking face, that they gazed more at the mouth than the eyes overall, that they gazed more at the eyes of an audiovisually synchronized than a desynchronized talking face, and that they gazed more at the mouth when all talking faces were audiovisually desynchronized. These findings demonstrate that audiovisual temporal synchrony plays a major role in perceptual segregation of multisensory clutter and that adults rely on differential scanning strategies of a talker's eyes and mouth to discover sources of multisensory coherence.

A Dynamic Systems Framework for Gender/Sex Development: From Sensory Input in Infancy to Subjective Certainty in Toddlerhood

From birth to 15 months infants and caregivers form a fundamentally intersubjective, dyadic unit within which the infant's ability to recognize gender/sex in the world develops. Between about 18 and 36 months the infant accumulates an increasingly clear and subjective sense of self as female or male. We know little about how the precursors to gender/sex identity form during the intersubjective period, nor how they transform into an independent sense of self by 3 years of age. In this Theory and Hypothesis article I offer a general framework for thinking about this problem. I propose that through repetition and patterning, the dyadic interactions in which infants and caregivers engage imbue the infant with an embodied, i.e., sensori-motor understanding of gender/sex. During this developmental period (which I label Phase 1) gender/sex is primarily an intersubjective project. From 15 to 18 months (which I label Phase 2) there are few reports of newly appearing gender/sex behavioral differences, and I hypothesize that this absence reflects a period of developmental instability during which there is a transition from gender/sex as primarily inter-subjective to gender/sex as primarily subjective. Beginning at 18 months (i.e., the start of Phase 3), a toddler's subjective sense of self as having a gender/sex emerges, and it solidifies by 3 years of age. I propose a dynamic systems perspective to track how infants first assimilate gender/sex information during the intersubjective period (birth to 15 months); then explore what changes might occur during a hypothesized phase transition (15 to 18 months), and finally, review the emergence and initial stabilization of individual subjectivity-the period from 18 to 36 months. The critical questions explored focus on how to model and translate data from very different experimental disciplines, especially neuroscience, physiology, developmental psychology and cognitive development. I close by proposing the formation of a research consortium on gender/sex development during the first 3 years after birth.

The development of visuotactile congruency effects for sequences of events

Sensitivity to the temporal coherence of visual and tactile signals increases perceptual reliability and is evident during infancy. However, it is not clear how, or whether, bidirectional visuotactile interactions change across childhood. Furthermore, no study has explored whether viewing a body modulates how children perceive visuotactile sequences of events. Here, children aged 5-7 years (n = 19), 8 and 9 years (n = 21), and 10-12 years (n = 24) and adults (n = 20) discriminated the number of target events (one or two) in a task-relevant modality (touch or vision) and ignored distractors (one or two) in the opposing modality. While participants performed the task, an image of either a hand or an object was presented. Children aged 5-7 years and 8 and 9 years showed larger crossmodal interference from visual distractors when discriminating tactile targets than the converse. Across age groups, this was strongest when two visual distractors were presented with one tactile target, implying a "fission-like" crossmodal effect (perceiving one event as two events). There was no influence of visual context (viewing a hand or non-hand image) on visuotactile interactions for any age group. Our results suggest robust interference from discontinuous visual information on tactile discrimination of sequences of events during early and middle childhood. These findings are discussed with respect to age-related changes in sensory dominance, selective attention, and multisensory processing.

Hearing Better with the Right Eye? The Lateralization of Multisensory Processing Affects Auditory Learning in Northern Bobwhite Quail (Colinus Virginianus) Chicks

Precocial avian species exhibit a high degree of lateralization of perceptual and motor abilities, including preferential eye use for tasks such as social recognition and predator detection. Such lateralization has been related, in part, to differential experience prior to hatch. That is, due to spatial and resulting postural constraints late in incubation, one eye and hemisphere-generally the right eye / left hemisphere-receive greater amounts of stimulation than the contralateral eye / hemisphere. This raises the possibility that the left hemisphere may specialize or show relative advantages in integrating information across visual and auditory modalities, given that it typically receives greater amounts of multimodal auditory and visual stimulation prior to hatch. The present study represents an initial investigation of this question in a precocial avian species, the Northern bobwhite quail (Colinus virginianus). Day-old bobwhite chicks received 5 min training sessions in which they vocalized to receive contingent playback of a bobwhite maternal call, presented with or without a light that flashed in synchrony with the notes of the call (i.e., bimodal versus unimodal exposure, respectively). Chicks were trained with or without eye patches that allowed them to experience the visual component of the bimodal stimulus with only the left eye (LE), right eye (RE), or both eyes (i.e., binocular; BIN). Finally, the light was placed in various positions relative to the speakers playing the maternal call across three experiments. 24 hrs later chicks were provided a simultaneous choice test between the familiarized and a novel bobwhite maternal call. Given that the right eye and ear typically face outward and are thus unoccluded by the body during late prenatal development, we hypothesized that RE chicks would show facilitated learning under bimodal conditions compared to all other training conditions. This hypothesis was partially confirmed in Experiment 1, when the light was positioned 40 cm above the source of the maternal call. However, we also observed evidence of suppressed learning in chicks provided BIN exposure to the bimodal audio-visual stimulus that was not present during auditory-only training. Experiments 2 and 3 demonstrated that this was likely related to activation of a left-hemisphere dependent fear response when the left eye was exposed to a visual stimulus that loomed above the auditory stimulus. These results indicate that multisensory processing is lateralized in a precocial bird and that these species may thus provide a unique model for studying experience-dependent plasticity of intersensory perception.

Ageing and multisensory integration: A review of the evidence, and a computational perspective

The processing of multisensory signals is crucial for effective interaction with the environment, but our ability to perform this vital function changes as we age. In the first part of this review, we summarise existing research into the effects of healthy ageing on multisensory integration. We note that age differences vary substantially with the paradigms and stimuli used: older adults often receive at least as much benefit (to both accuracy and response times) as younger controls from congruent multisensory stimuli, but are also consistently more negatively impacted by the presence of intersensory conflict. In the second part, we outline a normative Bayesian framework that provides a principled and computationally informed perspective on the key ingredients involved in multisensory perception, and how these are affected by ageing. Applying this framework to the existing literature, we conclude that changes to sensory reliability, prior expectations (together with attentional control), and decisional strategies all contribute to the age differences observed. However, we find no compelling evidence of any age-related changes to the basic inference mechanisms involved in multisensory perception.

The Importance of Complementary Collaboration of Researchers, Veterinarians, and Husbandry Staff in the Successful Training of Marmoset Behavioral Assays

Interest in marmosets as research models has seen exponential growth over the last decade, especially given that the research community is eager to improve on gaps with historical animal models for behavioral and cognitive disorders. The spectrum of human disease traits that present naturally in marmosets, as well as the range of analogous human behaviors that can be assessed in marmosets, makes them ideally suited as translational models for behavioral and cognitive disorders. Regardless of the specific research aims of any project, without close collaboration between researchers, veterinarians, and animal care staff, it would be impossible to meet these goals. Behavior is inherently variable, as are marmosets that are genetically and phenotypically diverse. Thus, to ensure rigor, reliability, and reproducibility in results, it is important that in the research environment, the animal's daily husbandry and veterinary needs are being met and align with the research goals while keeping the welfare of the animal the most critical and highest priority. Much of the information described herein provides details on key components for successful behavioral testing, based on a compendium of methods from peer-reviewed publications and our own experiences. Specific areas highlighted include habituation procedures, selection of appropriate rewards, optimization of testing environments, and ways to integrate regular veterinary and husbandry procedures into the research program with minimal disruptions to the behavioral testing plan. This article aims to provide a broad foundation for researchers new to establishing behavioral and cognitive testing paradigms in marmosets and especially for the veterinary and husbandry colleagues who are indispensable collaborators of these research projects.

Audiovisual interactions in speeded discrimination of a visual event

The integration of information from different senses is central to our perception of the external world. Audiovisual interactions have been particularly well studied in this context and various illusions have been developed to demonstrate strong influences of these interactions on the final percept. Using audiovisual paradigms, previous studies have shown that even task-irrelevant information provided by a secondary modality can change the detection and discrimination of a primary target. These modulations have been found to be significantly dependent on the relative timing between auditory and visual stimuli. Although these interactions in time have been commonly reported, we have still limited understanding of the relationship between the modulations of event-related potentials (ERPs) and final behavioral performance. Here, we aimed to shed light on this important issue by using a speeded discrimination paradigm combined with electroencephalogram (EEG). During the experimental sessions, the timing between an auditory click and a visual flash was varied over a wide range of stimulus onset asynchronies and observers were engaged in speeded discrimination of flash location. Behavioral reaction times were significantly changed by click timing. Furthermore, the modulations of evoked activities over medial parietal/parieto-occipital electrodes were associated with this effect. These modulations were within the 126-176 ms time range and more importantly, they were also correlated with the changes in reaction times. These results provide an important functional link between audiovisual interactions at early stages of sensory processing and reaction times. Together with previous research, they further suggest that early crossmodal interactions play a critical role in perceptual performance.

The Role of Audiovisual Speech in Fast-Mapping and Novel Word Retention in Monolingual and Bilingual 24-Month-Olds

Three experiments examined the role of audiovisual speech on 24-month-old monolingual and bilinguals’ performance in a fast-mapping task. In all three experiments, toddlers were exposed to familiar trials which tested their knowledge of known word–referent pairs, disambiguation trials in which novel word–referent pairs were indirectly learned, and retention trials which probed their recognition of the newly-learned word–referent pairs. In Experiment 1 (n = 48), lip movements were present during familiar and disambiguation trials, but not retention trials. In Experiment 2 (n = 48), lip movements were present during all three trial types. In Experiment 3 (bilinguals only, n = 24), a still face with no lip movements was present in all three trial types. While toddlers succeeded in the familiar and disambiguation trials of every experiment, success in the retention trials was only found in Experiment 2. This work suggests that the extra-linguistic support provided by lip movements improved the learning and recognition of the novel words.

Choice-dependent cross-modal interaction in the medial prefrontal cortex of rats

Cross-modal interaction (CMI) could significantly influence the perceptional or decision-making process in many circumstances. However, it remains poorly understood what integrative strategies are employed by the brain to deal with different task contexts. To explore it, we examined neural activities of the medial prefrontal cortex (mPFC) of rats performing cue-guided two-alternative forced-choice tasks. In a task requiring rats to discriminate stimuli based on auditory cue, the simultaneous presentation of an uninformative visual cue substantially strengthened mPFC neurons' capability of auditory discrimination mainly through enhancing the response to the preferred cue. Doing this also increased the number of neurons revealing a cue preference. If the task was changed slightly and a visual cue, like the auditory, denoted a specific behavioral direction, mPFC neurons frequently showed a different CMI pattern with an effect of cross-modal enhancement best evoked in information-congruent multisensory trials. In a choice free task, however, the majority of neurons failed to show a cross-modal enhancement effect and cue preference. These results indicate that CMI at the neuronal level is context-dependent in a way that differs from what has been shown in previous studies.

Aging Effects on the Neuroplastic Attributes of Multisensory Cortical Networks as Triggered by a Computerized Music Reading Training Intervention

The constant increase in the graying population is the result of a great expansion of life expectancy. A smaller expansion of healthy cognitive and brain functioning diminishes the gains achieved by longevity. Music training, as a special case of multisensory learning, may induce restorative neuroplasticity in older ages. The current study aimed to explore aging effects on the cortical network supporting multisensory cognition and to define aging effects on the network's neuroplastic attributes. A computer-based music reading protocol was developed and evaluated via electroencephalography measurements pre- and post-training on young and older adults. Results revealed that multisensory integration is performed via diverse strategies in the two groups: Older adults employ higher-order supramodal areas to a greater extent than lower level perceptual regions, in contrast to younger adults, indicating an age-related shift in the weight of each processing strategy. Restorative neuroplasticity was revealed in the left inferior frontal gyrus and right medial temporal gyrus, as a result of the training, while task-related reorganization of cortical connectivity was obstructed in the group of older adults, probably due to systemic maturation mechanisms. On the contrary, younger adults significantly increased functional connectivity among the regions supporting multisensory integration.

Improvements and Degradation to Spatial Tactile Acuity Among Blind and Deaf Individuals

Cross-modal reorganization takes place for sensory cortices when there is no more primary input. For instance, the visual cortex in blind individuals which receives no visual input starts responding to auditory and tactile stimuli. Reorganization may improve or degrade processing of other modality inputs, via bottom-up compensational processes and top-down updating. In two experiments, we measured the spatial tactile response in a large sample of early- (N = 49) and late-blind (N = 51) individuals with varying levels of Braille proficiencies, and early-deaf (N = 69) with varying levels of hearing devices against separate hearing and sighted controls. Spatial tactile responses were measured using a standard gradient orientation task on two locations, the finger and tongue. Experiments show limited to no advantage in passive tactile response for blind individuals and degradation for deaf individuals at the finger. However, the use of hearing devices decreased the tactile impairment in early-deaf individuals. Also, no differences in age-related decline in both sensory-impaired groups were shown. Results show less tactile acuity differences between blind and sighted than previously reported, but supports recent reports of tactile impairment among the early-deaf.

Audio-Visual Training in Older Adults: 2-Interval-Forced Choice Task Improves Performance

A growing interest in ameliorating multisensory perception deficits in older adults arises from recent evidence showing that impaired multisensory processing, particularly in the temporal domain, may be associated with cognitive and functional impairments. Perceptual training has proved successful in improving multisensory temporal processing in young adults, but few studies have investigated this training approach in older adults. In the present study we used a simultaneity (or synchronicity) judgement task with feedback, to train the audio-visual abilities of community-dwelling, cognitively healthy older adults. We recruited 23 older adults (M = 74.17, SD = 6.23) and a group of 20 young adults (M = 24.20, SD = 4.23) who served as a comparison. Participants were tested before and after perceptual training using a 2-Interval Forced Choice Task (2-IFC); and the Sound-Induced Flash Illusion (SIFI). After 3 days of training, participants improved on the 2-IFC task, with a significant narrowing of the temporal window of integration (TWI) found for both groups. Generalization of training effects was not found, with no post-training differences in perceptual sensitivity to the SIFI for either group. These findings provide evidence perceptual narrowing can be achieved in older as well as younger adults after 3 days of perceptual training. These results provide useful information for future studies attempting to improve audio-visual temporal discrimination abilities in older people.

Electrophysiological Dynamics of Visual-Tactile Temporal Order Perception in Early Deaf Adults

Studies of compensatory plasticity in early deaf (ED) individuals have mainly focused on unisensory processing, and on spatial rather than temporal coding. However, precise discrimination of the temporal relationship between stimuli is imperative for successful perception of and interaction with the complex, multimodal environment. Although the properties of cross-modal temporal processing have been extensively studied in neurotypical populations, remarkably little is known about how the loss of one sense impacts the integrity of temporal interactions among the remaining senses. To understand how auditory deprivation affects multisensory temporal interactions, ED and age-matched normal hearing (NH) controls performed a visual-tactile temporal order judgment task in which visual and tactile stimuli were separated by varying stimulus onset asynchronies (SOAs) and subjects had to discern the leading stimulus. Participants performed the task while EEG data were recorded. Group averaged event-related potential waveforms were compared between groups in occipital and fronto-central electrodes. Despite similar temporal order sensitivities and performance accuracy, ED had larger visual P100 amplitudes for all SOA levels and larger tactile N140 amplitudes for the shortest asynchronous (± 30 ms) and synchronous SOA levels. The enhanced signal strength reflected in these components from ED adults are discussed in terms of compensatory recruitment of cortical areas for visual-tactile processing. In addition, ED adults had similar tactile P200 amplitudes as NH but longer P200 latencies suggesting reduced efficiency in later processing of tactile information. Overall, these results suggest that greater responses by ED for early processing of visual and tactile signals are likely critical for maintained performance in visual-tactile temporal order discrimination.

DNA Methylation-Dependent Dysregulation of GABAergic Interneuron Functionality in Neuropsychiatric Diseases

Neuropsychiatric diseases, such as mood disorders, schizophrenia, and autism, represent multifactorial disorders, differing in causes, disease onset, severity, and symptoms. A common feature of numerous neuropsychiatric conditions are defects in the cortical inhibitory GABAergic system. The balance of excitation and inhibition is fundamental for proper and efficient information processing in the cerebral cortex. Thus, altered inhibition is suggested to account for pathological symptoms like cognitive impairments and dysfunctional multisensory integration. While it became apparent that most of these diseases have a clear genetic component, environmental influences emerged as an impact of disease manifestation, onset, and severity. Epigenetic mechanisms of transcriptional control, such as DNA methylation, are known to be responsive to external stimuli, and are suspected to be implicated in the functional impairments of GABAergic interneurons, and hence, the pathophysiology of neuropsychiatric diseases. Here, we provide an overview about the multifaceted functional implications of DNA methylation and DNA methyltransferases in cortical interneuron development and function in health and disease. Apart from the regulation of gamma-aminobutyric acid-related genes and genes relevant for interneuron development, we discuss the role of DNA methylation-dependent regulation of synaptic transmission by the modulation of endocytosis-related genes as potential pathophysiological mechanisms underlying neuropsychiatric conditions. Deciphering the hierarchy and mechanisms of changes in epigenetic signatures is crucial to develop effective strategies for treatment and prevention.

Neonatal Multisensory Processing in Preterm and Term Infants Predicts Sensory Reactivity and Internalizing Tendencies in Early Childhood

Multisensory processes include the capacity to combine information from the different senses, often improving stimulus representations and behavior. The extent to which multisensory processes are an innate capacity or instead require experience with environmental stimuli remains debated. We addressed this knowledge gap by studying multisensory processes in prematurely born and full-term infants. We recorded 128-channel event-related potentials (ERPs) from a cohort of 55 full-term and 61 preterm neonates (at an equivalent gestational age) in response to auditory, somatosensory, and combined auditory-somatosensory multisensory stimuli. Data were analyzed within an electrical neuroimaging framework, involving unsupervised topographic clustering of the ERP data. Multisensory processing in full-term infants was characterized by a simple linear summation of responses to auditory and somatosensory stimuli alone, which furthermore shared common ERP topographic features. We refer to the ERP topography observed in full-term infants as "typical infantile processing" (TIP). In stark contrast, preterm infants exhibited non-linear responses and topographies less-often characterized by TIP; there were distinct patterns of ERP topographies to multisensory and summed unisensory conditions. We further observed that the better TIP characterized an infant's ERPs, independently of prematurity, the more typical was the score on the Infant/Toddler Sensory Profile (ITSP) at 12 months of age and the less likely was the child to the show internalizing tendencies at 24 months of age. Collectively, these results highlight striking differences in the brain's responses to multisensory stimuli in children born prematurely; differences that relate to later sensory and internalizing functions.

The Effect of Alpha tACS on the Temporal Resolution of Visual Perception

We experience the world around us as a smooth and continuous flow. However, there is growing evidence that the stream of sensory inputs is not elaborated in an analog way but is instead organized in discrete or quasi-discrete temporal processing windows. These discrete windows are suggested to depend on rhythmic neural activity in the alpha (and theta) frequency bands, which in turn reflect changes in neural activity within, and coupling between, cortical areas. In the present study, we investigated a possible causal link between oscillatory brain activity in the alpha range (8-12 Hz) and the temporal resolution of visual perception, which determines whether sequential stimuli are perceived as distinct entities or combined into a single representation. To this aim, we employed a two-flash fusion task while participants received focal transcranial alternating current stimulation (tACS) in extra-striate visual regions including V5/MT of the right hemisphere. Our findings show that 10-Hz tACS, as opposed to a placebo (sham tACS), reduces the temporal resolution of perception, inducing participants to integrate the two stimuli into a unique percept more often. This pattern was observed only in the contralateral visual hemifield, providing further support for a specific effect of alpha tACS. The present findings corroborate the idea of a causal link between temporal windows of integration/segregation and oscillatory alpha activity in V5/MT and extra-striate visual regions. They also stimulate future research on possible ways to shape the temporal resolution of human vision in an individualized manner.

An Eye-Tracking Study on Audiovisual Speech Perception Strategies Adopted by Normal-Hearing and Deaf Adults Under Different Language Familiarities

Purpose The primary purpose of this study was to explore the audiovisual speech perception strategies.80.23.47 adopted by normal-hearing and deaf people in processing familiar and unfamiliar languages. Our primary hypothesis was that they would adopt different perception strategies due to different sensory experiences at an early age, limitations of the physical device, and the developmental gap of language, and others. Method Thirty normal-hearing adults and 33 prelingually deaf adults participated in the study. They were asked to perform judgment and listening tasks while watching videos of a Uygur-Mandarin bilingual speaker in a familiar language (Standard Chinese) or an unfamiliar language (Modern Uygur) while their eye movements were recorded by eye-tracking technology. Results Task had a slight influence on the distribution of selective attention, whereas subject and language had significant influences. To be specific, the normal-hearing and the d10eaf participants mainly gazed at the speaker's eyes and mouth, respectively, in the experiment; moreover, while the normal-hearing participants had to stare longer at the speaker's mouth when they confronted with the unfamiliar language Modern Uygur, the deaf participant did not change their attention allocation pattern when perceiving the two languages. Conclusions Normal-hearing and deaf adults adopt different audiovisual speech perception strategies: Normal-hearing adults mainly look at the eyes, and deaf adults mainly look at the mouth. Additionally, language and task can also modulate the speech perception strategy.

Focused Multisensory Anatomy Observation and Drawing for Enhancing Social Learning and Three-Dimensional Spatial Understanding

The concept that multisensory observation and drawing can be effective for enhancing anatomy learning is supported by pedagogic research and theory, and theories of drawing. A haptico-visual observation and drawing (HVOD) process has been previously introduced to support understanding of the three-dimensional (3D) spatial form of anatomical structures. The HVOD process involves exploration of 3D anatomy with the combined use of touch and sight, and the simultaneous act of making graphite marks on paper which correspond to the anatomy under observation. Findings from a previous study suggest that HVOD can increase perceptual understanding of anatomy through memorization and recall of the 3D form of observed structures. Here, additional pedagogic and cognitive underpinnings are presented to further demonstrate how and why HVOD can be effective for anatomy learning. Delivery of a HVOD workshop is described as a detailed guide for instructors, and themes arising from a phenomenological study of educator experiences of the HVOD process are presented. Findings indicate that HVOD can provide an engaging approach for the spatial exploration of anatomy within a supportive social learning environment, but also requires modification for effective curricular integration. Consequently, based on the most effective research-informed, theoretical, and logistical elements of art-based approaches in anatomy learning, including the framework provided by the observe-reflect-draw-edit-repeat (ORDER) method, an optimized "ORDER Touch" observation and drawing process has been developed. This is with the aim of providing a widely accessible resource for supporting social learning and 3D spatial understanding of anatomy, in addition to improving specific anatomical knowledge.

Auditory Enhancement of Illusory Contour Perception

Illusory contours (ICs) are borders that are perceived in the absence of contrast gradients. Until recently, IC processes were considered exclusively visual in nature and presumed to be unaffected by information from other senses. Electrophysiological data in humans indicates that sounds can enhance IC processes. Despite cross-modal enhancement being observed at the neurophysiological level, to date there is no evidence of direct amplification of behavioural performance in IC processing by sounds. We addressed this knowledge gap. Healthy adults ( n = 15) discriminated instances when inducers were arranged to form an IC from instances when no IC was formed (NC). Inducers were low-constrast and masked, and there was continuous background acoustic noise throughout a block of trials. On half of the trials, i.e., independently of IC vs NC, a 1000-Hz tone was presented synchronously with the inducer stimuli. Sound presence improved the accuracy of indicating when an IC was presented, but had no impact on performance with NC stimuli (significant IC presence/absence × Sound presence/absence interaction). There was no evidence that this was due to general alerting or to a speed-accuracy trade-off (no main effect of sound presence on accuracy rates and no comparable significant interaction on reaction times). Moreover, sound presence increased sensitivity and reduced bias on the IC vs NC discrimination task. These results demonstrate that multisensory processes augment mid-level visual functions, exemplified by IC processes. Aside from the impact on neurobiological and computational models of vision, our findings may prove clinically beneficial for low-vision or sight-restored patients.

Plasticity of Temporal Binding in Children with Autism Spectrum Disorder:A Single Case Experimental Design Perceptual Training Study

BACKGROUND

Many children with autism spectrum disorder (ASD) demonstrate atypical responses to multisensory stimuli. These disruptions, which are frequently seen in response to audiovisual speech, may produce cascading effects on the broader development of children with ASD. Perceptual training has been shown to enhance multisensory speech perception in typically developed adults. This study was the first to examine the effects of perceptual training on audiovisual speech perception in children with ASD.

METHOD

A multiple baseline across participants design was utilized with four 7- to 13-year-old children with ASD. The dependent variable, which was probed outside the training task each day using a simultaneity judgment task in baseline, intervention, and maintenance conditions, was audiovisual temporal binding window (TBW), an index of multisensory temporal acuity. During perceptual training, participants completed the same simultaneity judgment task with feedback on their accuracy after each trial in easy-, medium-, and hard-difficulty blocks.

RESULTS

A functional relation between the multisensory perceptual training program and TBW size was not observed. Of the three participants who were entered into training, one participant demonstrated a strong effect, characterized by a fairly immediate change in TBW trend. The two remaining participants demonstrated a less clear response (i.e., longer latency to effect, lack of functional independence). The first participant to enter the training condition demonstrated some maintenance of a narrower TBW post-training.

CONCLUSIONS

Results indicate TBWs in children with ASD may be malleable, but additional research is needed and may entail further adaptation to the multisensory perceptual training paradigm.

Generative Memory for Lifelong Learning

Lifelong learning is a crucial issue in advanced artificial intelligence. It requires the learning system to learn and accumulate knowledge from sequential tasks. The learning system needs to deal with increasingly more domains and tasks. We consider that the key to an effective and efficient lifelong learning system is the ability to memorize and recall the learned knowledge using neural networks. Following this idea, we propose Generative Memory (GM) as a novel memory module, and the resulting lifelong learning system is referred to as the GM Net (GMNet). To make the GMNet feasible, we propose a novel learning mechanism, referred to as P -invariant learning method. It replaces the memory of the real data by a memory of the data distribution, which makes it possible for the learning system to accurately and continuously accumulate the learned experiences. We demonstrate that GMNet achieves the state-of-the-art performance on lifelong learning tasks.

Selective attention to sound features mediates cross-modal activation of visual cortices

Contemporary schemas of brain organization now include multisensory processes both in low-level cortices as well as at early stages of stimulus processing. Evidence has also accumulated showing that unisensory stimulus processing can result in cross-modal effects. For example, task-irrelevant and lateralised sounds can activate visual cortices; a phenomenon referred to as the auditory-evoked contralateral occipital positivity (ACOP). Some claim this is an example of automatic attentional capture in visual cortices. Other results, however, indicate that context may play a determinant role. Here, we investigated whether selective attention to spatial features of sounds is a determining factor in eliciting the ACOP. We recorded high-density auditory evoked potentials (AEPs) while participants selectively attended and discriminated sounds according to four possible stimulus attributes: location, pitch, speaker identity or syllable. Sound acoustics were held constant, and their location was always equiprobable (50% left, 50% right). The only manipulation was to which sound dimension participants attended. We analysed the AEP data from healthy participants within an electrical neuroimaging framework. The presence of sound-elicited activations of visual cortices depended on the to-be-discriminated, goal-based dimension. The ACOP was elicited only when participants were required to discriminate sound location, but not when they attended to any of the non-spatial features. These results provide a further indication that the ACOP is not automatic. Moreover, our findings showcase the interplay between task-relevance and spatial (un)predictability in determining the presence of the cross-modal activation of visual cortices.

A N400 ERP Study in letter recognition after passive tactile stimulation training in blind children and sighted controls

BACKGROUND

We previously demonstrated that using a sensory substitution device (SSD) for one week, tactile stimulation results in faster activation of lateral occipital complex in blind children than in seeing controls.

OBJECTIVE

We used long-term haptic tactile stimulation training with an SSD to test if it results in stable cross-modal reassignment of visual pathways after six months, to provide high level processing of tactile semantic content.

METHODS

We enrolled 12 blind and 12 sighted children. The SSD transforms images to a stimulation matrix in contact with the dominant hand. Subjects underwent twice-daily training sessions, 5 days/week for six months. Children were asked to describe line orientation, name letters, and read words. ERP sessions were performed at baseline and 6 months to analyze the N400 ERP component and reaction times (RT). N400 sources were estimated with Low Resolution Electromagnetic Tomography (LORETA). SPM8 was used to make population-level inferences.

RESULTS

We found no group differences in RTs, accuracy of identifications, N400 latencies or distributions with the line task at 1 week or at 6 months. RTs on the letter recognition task were also similar. After 6 months, behavioral training increased accurate letter identification in both seeing and blind children (Chi 2 = 11906.934, p = 0.000), but the increase was larger in blind children (Chi 2 = 8.272, p = 0.004). Behavioral training shifted peak N400 amplitude to left occipital and bilateral parietal cortices in blind children, but to left precentral and postcentral and bilateral occipital cortices in sighted controls.

CONCLUSIONS

Blind children learn to recognize SSD-delivered letters better than seeing controls and had greater N400 amplitude in the occipital region. To the best of our knowledge, our results provide the first published example of standard letter recognition (not Braille) by children with blindness using a tactile delivery system.

Mental Rotation of Digitally-Rendered Haptic Objects by the Visually-Impaired

In the event of visual impairment or blindness, information from other intact senses can be used as substitutes to retrain (and in extremis replace) visual functions. Abilities including reading, mental representation of objects and spatial navigation can be performed using tactile information. Current technologies can convey a restricted library of stimuli, either because they depend on real objects or renderings with low resolution layouts. Digital haptic technologies can overcome such limitations. The applicability of this technology was previously demonstrated in sighted participants. Here, we reasoned that visually-impaired and blind participants can create mental representations of letters presented haptically in normal and mirror-reversed form without the use of any visual information, and mentally manipulate such representations. Visually-impaired and blind volunteers were blindfolded and trained on the haptic tablet with two letters (either L and P or F and G). During testing, they haptically explored on any trial one of the four letters presented at 0°, 90°, 180°, or 270° rotation from upright and indicated if the letter was either in a normal or mirror-reversed form. Rotation angle impacted performance; greater deviation from 0° resulted in greater impairment for trained and untrained normal letters, consistent with mental rotation of these haptically-rendered objects. Performance was also generally less accurate with mirror-reversed stimuli, which was not affected by rotation angle. Our findings demonstrate, for the first time, the suitability of a digital haptic technology in the blind and visually-impaired. Classic devices remain limited in their accessibility and in the flexibility of their applications. We show that mental representations can be generated and manipulated using digital haptic technology. This technology may thus offer an innovative solution to the mitigation of impairments in the visually-impaired, and to the training of skills dependent on mental representations and their spatial manipulation.

Double Flash Illusions: Current Findings and Future Directions

Twenty years ago, the first report on the sound-induced double flash illusion, a visual illusion induced by sound, was published. In this paradigm, participants are presented with different numbers of auditory and visual stimuli. In case of an incongruent number of auditory and visual stimuli, the influence of auditory information on visual perception can lead to the perception of the illusion. Thus, combining two auditory stimuli with one visual stimulus can induce the perception of two visual stimuli, the so-called fission illusion. Alternatively, combining one auditory stimulus with two visual stimuli can induce the perception of one visual stimulus, the so-called fusion illusion. Overall, current research shows that the illusion is a reliable indicator of multisensory integration. It has also been replicated using different stimulus combinations, such as visual and tactile stimuli. Importantly, the robustness of the illusion allows the widespread use for assessing multisensory integration across different groups of healthy participants and clinical populations and in various task setting. This review will give an overview of the experimental evidence supporting the illusion, the current state of research concerning the influence of cognitive processes on the illusion, the neural mechanisms underlying the illusion, and future research directions. Moreover, an exemplary experimental setup will be described with different options to examine perception, alongside code to test and replicate the illusion online or in the laboratory.

Audiovisual temporal integration: Cognitive processing, neural mechanisms, developmental trajectory and potential interventions

To integrate auditory and visual signals into a unified percept, the paired stimuli must co-occur within a limited time window known as the Temporal Binding Window (TBW). The width of the TBW, a proxy of audiovisual temporal integration ability, has been found to be correlated with higher-order cognitive and social functions. A comprehensive review of studies investigating audiovisual TBW reveals several findings: (1) a wide range of top-down processes and bottom-up features can modulate the width of the TBW, facilitating adaptation to the changing and multisensory external environment; (2) a large-scale brain network works in coordination to ensure successful detection of audiovisual (a)synchrony; (3) developmentally, audiovisual TBW follows a U-shaped pattern across the lifespan, with a protracted developmental course into late adolescence and rebounding in size again in late life; (4) an enlarged TBW is characteristic of a number of neurodevelopmental disorders; and (5) the TBW is highly flexible via perceptual and musical training. Interventions targeting the TBW may be able to improve multisensory function and ameliorate social communicative symptoms in clinical populations.

Multisensory Gains in Simple Detection Predict Global Cognition in Schoolchildren

The capacity to integrate information from different senses is central for coherent perception across the lifespan from infancy onwards. Later in life, multisensory processes are related to cognitive functions, such as speech or social communication. During learning, multisensory processes can in fact enhance subsequent recognition memory for unisensory objects. These benefits can even be predicted; adults' recognition memory performance is shaped by earlier responses in the same task to multisensory - but not unisensory - information. Everyday environments where learning occurs, such as classrooms, are inherently multisensory in nature. Multisensory processes may therefore scaffold healthy cognitive development. Here, we provide the first evidence of a predictive relationship between multisensory benefits in simple detection and higher-level cognition that is present already in schoolchildren. Multiple regression analyses indicated that the extent to which a child (N = 68; aged 4.5-15years) exhibited multisensory benefits on a simple detection task not only predicted benefits on a continuous recognition task involving naturalistic objects (p = 0.009), even when controlling for age, but also the same relative multisensory benefit also predicted working memory scores (p = 0.023) and fluid intelligence scores (p = 0.033) as measured using age-standardised test batteries. By contrast, gains in unisensory detection did not show significant prediction of any of the above global cognition measures. Our findings show that low-level multisensory processes predict higher-order memory and cognition already during childhood, even if still subject to ongoing maturation. These results call for revision of traditional models of cognitive development (and likely also education) to account for the role of multisensory processing, while also opening exciting opportunities to facilitate early learning through multisensory programs. More generally, these data suggest that a simple detection task could provide direct insights into the integrity of global cognition in schoolchildren and could be further developed as a readily-implemented and cost-effective screening tool for neurodevelopmental disorders, particularly in cases when standard neuropsychological tests are infeasible or unavailable.

Multisensory Integration as a Window into Orderly and Disrupted Cognition and Communication

During our everyday lives, we are confronted with a vast amount of information from several sensory modalities. This multisensory information needs to be appropriately integrated for us to effectively engage with and learn from our world. Research carried out over the last half century has provided new insights into the way such multisensory processing improves human performance and perception; the neurophysiological foundations of multisensory function; the time course for its development; how multisensory abilities differ in clinical populations; and, most recently, the links between multisensory processing and cognitive abilities. This review summarizes the extant literature on multisensory function in typical and atypical circumstances, discusses the implications of the work carried out to date for theory and research, and points toward next steps for advancing the field.

Audiovisual temporal integration and rapid temporal recalibration in adolescents and adults: Age-related changes and its correlation with autistic traits

Temporal structure is a key factor in determining the relatedness of multisensory stimuli. Stimuli that are close in time are more likely to be integrated into a unified perceptual representation. To investigate the age-related developmental differences in audiovisual temporal integration and rapid temporal recalibration, we administered simultaneity judgment (SJ) tasks to a group of adolescents (11-14 years) and young adults (18-28 years). No age-related changes were found in the width of the temporal binding window within which participants are highly likely to combine multisensory stimuli. The main distinction between adolescents and adults was audiovisual temporal recalibration. Although participants of both age groups could rapidly recalibrate based on the previous trial for speech stimuli (i.e., syllable utterances), only adults but not adolescents showed short-term recalibration for simple and non-speech stimuli. In both adolescents and adults, no significant correlation was found between audiovisual temporal integration ability and autistic or schizotypal traits. These findings provide new information on the developmental trajectory of basic multisensory function and may have implications for neurodevelopmental disorders (e.g., autism) with altered audiovisual temporal integration. Autism Res 2020, 13: 615-626. © 2019 International Society for Autism Research, Wiley Periodicals, Inc. LAY SUMMARY: Utilizing temporal cues to integrate and separate audiovisual information is a fundamental ability underlying higher order social communicative functions. This study examines the developmental changes of the ability to detect audiovisual asynchrony and rapidly adjust sensory decisions based on previous sensory input. In healthy adolescents and young adults, the correlation between autistic traits and audiovisual integration ability failed to reach a significant level. Therefore, more research is needed to examine whether impairment in basic sensory functions is correlated with broader autism phenotype in nonclinical populations. These results may help us understand altered multisensory integration in people with autism.

Sharing sounds: The development of auditory joint engagement during early parent-child interaction

Joint engagement-the sharing of events during social interactions-is an important context for early learning. To date, sharing topics that are only heard has not been systematically documented. To describe the development of auditory joint engagement, 48 child-parent dyads were observed 5 times from 12 to 30 months during seminaturalistic play. Reactions to 4 types of sounds-overheard speech about the child, instrumental music, animal calls, and mechanical noises-were observed before and as parents scaffolded shared listening and after the sound ceased. Before parents reacted, even 12-month-old infants readily alerted and oriented to the sounds; over time they increasingly tried to share new sounds with their parents. When parents then joined in sharing a sound, periods of auditory joint engagement often ensued, increasing from two thirds of 12-month observations to almost ceiling level at the 18- through 30-month observations. Overall, the developmental course and structure of auditory joint engagement and joint engagement with multimodal objects and events are remarkably similar. Symbol-infused auditory joint engagement occurred rarely at first but increased steadily. Children's labeling of the sound and parents' language scaffolding also increased linearly while child pointing toward it rose until 18 months and then declined. Future studies should address variations in the development of auditory joint engagement, whether autism spectrum disorder affects how toddlers share sounds, and the role auditory joint engagement may play in gestural and language development. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Perception it is: Processing level in multisensory selection

When repeatedly exposed to simultaneously presented stimuli, associations between these stimuli are nearly always established, both within as well as between sensory modalities. Such associations guide our subsequent actions and may also play a role in multisensory selection. Thus, crossmodal associations (i.e., associations between stimuli from different modalities) learned in a multisensory interference task might affect subsequent information processing. The aim of this study was to investigate the processing level of multisensory stimuli in multisensory selection by means of crossmodal aftereffects. Either feature or response associations were induced in a multisensory flanker task while the amount of interference in a subsequent crossmodal flanker task was measured. The results of Experiment 1 revealed the existence of crossmodal interference after multisensory selection. Experiments 2 and 3 then went on to demonstrate the dependence of this effect on the perceptual associations between features themselves, rather than on the associations between feature and response. Establishing response associations did not lead to a subsequent crossmodal interference effect (Experiment 2), while stimulus feature associations without response associations (obtained by changing the response effectors) did (Experiment 3). Taken together, this pattern of results suggests that associations in multisensory selection, and the interference of (crossmodal) distractors, predominantly work at the perceptual, rather than at the response, level.

Alteration in binocular fusion modifies audiovisual integration in children

Background: In the field of multisensory integration, vision is generally thought to dominate audiovisual interactions, at least in spatial tasks, but the role of binocular fusion in audiovisual integration has not yet been studied.

Methods: Using the Maddox test, a classical ophthalmological test used to subjectively detect a latent unilateral eye deviation, we checked whether an alteration in binocular vision in young patients would be able to change audiovisual integration. The study was performed on a group of ten children (five males and five females aged 11.3±1.6 years) with normal binocular vision, and revealed a visual phenomenon consisting of stochastic disappearanceof part of a visual scene caused by auditory stimulation.

Results: Indeed, during the Maddox test, brief sounds induced transient visual scotomas (VSs) in the visual field of the eye in front of where the Maddox rod was placed. We found a significant correlation between the modification of binocular vision and VS occurrence. No significant difference was detected in the percentage or location of VS occurrence between the right and left eye using the Maddox rod test orbetween sound frequencies.

Conclusion: The results indicate a specific role of the oculomotor system in audiovisual integration in children. This convenient protocol may also have significant interest for clinical investigations of developmental pathologies where relationships between vision and hearing are specifically affected.

Visual temporal integration windows are adult-like in 5- to 7-year-old children

The visual system must organize dynamic input into useful percepts across time, balancing between stability and sensitivity to change. The temporal integration window (TIW) has been hypothesized to underlie this balance: If two or more stimuli fall within the same TIW, they are integrated into a single percept; those that fall in different windows are segmented (Arnett & Di Lollo, 1979; Wutz, Muschter, van Koningsbruggen, Weisz, & Melcher, 2016). Visual TIWs have been studied in adults, showing average windows of 65 ms (Wutz et al., 2016); however, it is unclear how windows develop through early childhood. Here we measured TIWs in 5- to 7-year-old children and adults, using a variant of the missing dot task (Di Lollo, 1980; Wutz et al. 2016), in which integration and segmentation thresholds were measured within the same participant, using the same stimuli. Participants saw a sequence of two displays separated by an interstimulus interval (ISI) that determined the visibility of a visual search target. Longer ISIs increased the likelihood of detecting a segmentation target (but decreased detection for the integration target) although shorter ISIs increased the likelihood of detecting the integration target (but decreased detection of the segmentation target). We could then estimate the TIW by measuring the point at which these two functions intersect. Children's TIWs (M = 68 ms) were comparable to adults' (M = 73 ms) with no appreciable age trend within our sample, indicating that TIWs reach adult levels by approximately 5 years of age.

Stability and flexibility in multisensory sampling: insights from perceptual illusions

Neural, oscillatory, and computational counterparts of multisensory processing remain a crucial challenge for neuroscientists. Converging evidence underlines a certain efficiency in balancing stability and flexibility of sensory sampling, supporting the general idea that multiple parallel and hierarchically organized processing stages in the brain contribute to our understanding of the (sensory/perceptual) world. Intriguingly, how temporal dynamics impact and modulate multisensory processes in our brain can be investigated benefiting from studies on perceptual illusions.

Are We Ready for Real-world Neuroscience?

Real-world environments are typically dynamic, complex, and multisensory in nature and require the support of top-down attention and memory mechanisms for us to be able to drive a car, make a shopping list, or pour a cup of coffee. Fundamental principles of perception and functional brain organization have been established by research utilizing well-controlled but simplified paradigms with basic stimuli. The last 30 years ushered a revolution in computational power, brain mapping, and signal processing techniques. Drawing on those theoretical and methodological advances, over the years, research has departed more and more from traditional, rigorous, and well-understood paradigms to directly investigate cognitive functions and their underlying brain mechanisms in real-world environments. These investigations typically address the role of one or, more recently, multiple attributes of real-world environments. Fundamental assumptions about perception, attention, or brain functional organization have been challenged-by studies adapting the traditional paradigms to emulate, for example, the multisensory nature or varying relevance of stimulation or dynamically changing task demands. Here, we present the state of the field within the emerging heterogeneous domain of real-world neuroscience. To be precise, the aim of this Special Focus is to bring together a variety of the emerging "real-world neuroscientific" approaches. These approaches differ in their principal aims, assumptions, or even definitions of "real-world neuroscience" research. Here, we showcase the commonalities and distinctive features of the different "real-world neuroscience" approaches. To do so, four early-career researchers and the speakers of the Cognitive Neuroscience Society 2017 Meeting symposium under the same title answer questions pertaining to the added value of such approaches in bringing us closer to accurate models of functional brain organization and cognitive functions.

Brain and Cognitive Mechanisms of Top–Down Attentional Control in a Multisensory World: Benefits of Electrical Neuroimaging

In real-world environments, information is typically multisensory, and objects are a primary unit of information processing. Object recognition and action necessitate attentional selection of task-relevant from among task-irrelevant objects. However, the brain and cognitive mechanisms governing these processes remain not well understood. Here, we demonstrate that attentional selection of visual objects is controlled by integrated top–down audiovisual object representations (“attentional templates”) while revealing a new brain mechanism through which they can operate. In multistimulus (visual) arrays, attentional selection of objects in humans and animal models is traditionally quantified via “the N2pc component”: spatially selective enhancements of neural processing of objects within ventral visual cortices at approximately 150–300 msec poststimulus. In our adaptation of Folk et al.'s [Folk, C. L., Remington, R. W., & Johnston, J. C. Involuntary covert orienting is contingent on attentional control settings. Journal of Experimental Psychology: Human Perception and Performance, 18, 1030–1044, 1992] spatial cueing paradigm, visual cues elicited weaker behavioral attention capture and an attenuated N2pc during audiovisual versus visual search. To provide direct evidence for the brain, and so, cognitive, mechanisms underlying top–down control in multisensory search, we analyzed global features of the electrical field at the scalp across our N2pcs. In the N2pc time window (170–270 msec), color cues elicited brain responses differing in strength and their topography. This latter finding is indicative of changes in active brain sources. Thus, in multisensory environments, attentional selection is controlled via integrated top–down object representations, and so not only by separate sensory-specific top–down feature templates (as suggested by traditional N2pc analyses). We discuss how the electrical neuroimaging approach can aid research on top–down attentional control in naturalistic, multisensory settings and on other neurocognitive functions in the growing area of real-world neuroscience.