Are Supramodality and Cross-Modal Plasticity the Yin and Yang of Brain Development? From Blindness to Rehabilitation

Perceiving depth beyond sight: Evaluating intrinsic and learned cues via a proof of concept sensory substitution method in the visually impaired and sighted

This study explores spatial perception of depth by employing a novel proof of concept sensory substitution algorithm. The algorithm taps into existing cognitive scaffolds such as language and cross modal correspondences by naming objects in the scene while representing their elevation and depth by manipulation of the auditory properties for each axis. While the representation of verticality utilized a previously tested correspondence with pitch, the representation of depth employed an ecologically inspired manipulation, based on the loss of gain and filtration of higher frequency sounds over distance. The study, involving 40 participants, seven of which were blind (5) or visually impaired (2), investigates the intrinsicness of an ecologically inspired mapping of auditory cues for depth by comparing it to an interchanged condition where the mappings of the two axes are swapped. All participants successfully learned to use the algorithm following a very brief period of training, with the blind and visually impaired participants showing similar levels of success for learning to use the algorithm as did their sighted counterparts. A significant difference was found at baseline between the two conditions, indicating the intuitiveness of the original ecologically inspired mapping. Despite this, participants were able to achieve similar success rates following the training in both conditions. The findings indicate that both intrinsic and learned cues come into play with respect to depth perception. Moreover, they suggest that by employing perceptual learning, novel sensory mappings can be trained in adulthood. Regarding the blind and visually impaired, the results also support the convergence view, which claims that with training, their spatial abilities can converge with those of the sighted. Finally, we discuss how the algorithm can open new avenues for accessibility technologies, virtual reality, and other practical applications.

Longitudinal stability of individual brain plasticity patterns in blindness

The primary visual cortex (V1) in blindness is engaged in a wide spectrum of tasks and sensory modalities, including audition, touch, language, and memory. This widespread involvement raises questions regarding the constancy of its role and whether it might exhibit flexibility in its function over time, connecting to diverse network functions specific to task demands. This would suggest that reorganized V1 assumes a role like multiple-demand system regions. Alternatively, varying patterns of plasticity in blind V1 may be attributed to individual factors, with different blind individuals recruiting V1 preferentially for different functions. In support of this, we recently showed that V1 functional connectivity (FC) varies greatly across blind individuals. But do these represent stable individual patterns of plasticity, or are they driven more by instantaneous changes, like a multiple-demand system now inhabiting V1? Here, we tested whether individual FC patterns from the V1 of blind individuals are stable over time. We show that over two years, FC from the V1 is unique and highly stable in a small sample of repeatedly sampled congenitally blind individuals. Further, using multivoxel pattern analysis, we demonstrate that the unique reorganization patterns of these individuals allow decoding of participant identity. Together with recent evidence for substantial individual differences in V1 connectivity, this indicates that there may be a consistent role for V1 in blindness, which may differ for each individual. Further, it suggests that the variability in visual reorganization in blindness across individuals could be used to seek stable neuromarkers for sight rehabilitation and assistive approaches.

Hearing temperatures: employing machine learning for elucidating the cross-modal perception of thermal properties through audition

People can use their sense of hearing for discerning thermal properties, though they are for the most part unaware that they can do so. While people unequivocally claim that they cannot perceive the temperature of pouring water through the auditory properties of hearing it being poured, our research further strengthens the understanding that they can. This multimodal ability is implicitly acquired in humans, likely through perceptual learning over the lifetime of exposure to the differences in the physical attributes of pouring water. In this study, we explore people's perception of this intriguing cross modal correspondence, and investigate the psychophysical foundations of this complex ecological mapping by employing machine learning. Our results show that not only can the auditory properties of pouring water be classified by humans in practice, the physical characteristics underlying this phenomenon can also be classified by a pre-trained deep neural network.

Localizing 3D motion through the fingertips: Following in the footsteps of elephants

Each sense serves a different specific function in spatial perception, and they all form a joint multisensory spatial representation. For instance, hearing enables localization in the entire 3D external space, while touch traditionally only allows localization of objects on the body (i.e., within the peripersonal space alone). We use an in-house touch-motion algorithm (TMA) to evaluate individuals' capability to understand externalized 3D information through touch, a skill that was not acquired during an individual's development or in evolution. Four experiments demonstrate quick learning and high accuracy in localization of motion using vibrotactile inputs on fingertips and successful audio-tactile integration in background noise. Subjective responses in some participants imply spatial experiences through visualization and perception of tactile "moving" sources beyond reach. We discuss our findings with respect to developing new skills in an adult brain, including combining a newly acquired "sense" with an existing one and computation-based brain organization.

Changes in primary visual and auditory cortex of blind and sighted adults following 10 weeks of click-based echolocation training

Recent work suggests that the adult human brain is very adaptable when it comes to sensory processing. In this context, it has also been suggested that structural "blueprints" may fundamentally constrain neuroplastic change, e.g. in response to sensory deprivation. Here, we trained 12 blind participants and 14 sighted participants in echolocation over a 10-week period, and used MRI in a pre-post design to measure functional and structural brain changes. We found that blind participants and sighted participants together showed a training-induced increase in activation in left and right V1 in response to echoes, a finding difficult to reconcile with the view that sensory cortex is strictly organized by modality. Further, blind participants and sighted participants showed a training induced increase in activation in right A1 in response to sounds per se (i.e. not echo-specific), and this was accompanied by an increase in gray matter density in right A1 in blind participants and in adjacent acoustic areas in sighted participants. The similarity in functional results between sighted participants and blind participants is consistent with the idea that reorganization may be governed by similar principles in the two groups, yet our structural analyses also showed differences between the groups suggesting that a more nuanced view may be required.

Longitudinal stability of individual brain plasticity patterns in blindness

The primary visual cortex (V1) in individuals born blind is engaged in a wide spectrum of tasks and sensory modalities, including audition, touch, language, and memory. This widespread involvement raises questions regarding the constancy of its role and whether it might exhibit flexibility in its function over time, connecting to diverse network functions in response to task-specific demands. This would suggest that reorganized V1 takes on a role similar to cognitive multiple-demand system regions. Alternatively, it is possible that the varying patterns of plasticity observed in the blind V1 can be attributed to individual factors, whereby different blind individuals recruit V1 for different functions, highlighting the immense idiosyncrasy of plasticity. In support of this second account, we have recently shown that V1 functional connectivity varies greatly across blind individuals. But do these represent stable individual patterns of plasticity or merely instantaneous changes, for a multiple-demand system now inhabiting V1? Here we tested if individual connectivity patterns from the visual cortex of blind individuals are stable over time. We show that over two years, fMRI functional connectivity from the primary visual cortex is unique and highly stable in a small sample of repeatedly sampled congenitally blind individuals. Further, using multivoxel pattern analysis, we demonstrate that the unique reorganization patterns of these individuals allow decoding of participant identity. Together with recent evidence for substantial individual differences in visual cortex connectivity, this indicates there may be a consistent role for the visual cortex in blindness, which may differ for each individual. Further, it suggests that the variability in visual reorganization in blindness across individuals could be used to seek stable neuromarkers for sight rehabilitation and assistive approaches.

Brain structural changes in blindness: a systematic review and an anatomical likelihood estimation (ALE) meta-analysis

In recent decades, numerous structural brain imaging studies investigated purported morphometric changes in early (EB) and late onset blindness (LB). The results of these studies have not yielded very consistent results, neither with respect to the type, nor to the anatomical locations of the brain morphometric alterations. To better characterize the effects of blindness on brain morphometry, we performed a systematic review and an Anatomical-Likelihood-Estimation (ALE) coordinate-based-meta-analysis of 65 eligible studies on brain structural changes in EB and LB, including 890 EB, 466 LB and 1257 sighted controls. Results revealed atrophic changes throughout the whole extent of the retino-geniculo-striate system in both EB and LB, whereas changes in areas beyond the occipital lobe occurred in EB only. We discuss the nature of some of the contradictory findings with respect to the used brain imaging methodologies and characteristics of the blind populations such as the onset, duration and cause of blindness. Future studies should aim for much larger sample sizes, eventually by merging data from different brain imaging centers using the same imaging sequences, opt for multimodal structural brain imaging, and go beyond a purely structural approach by combining functional with structural connectivity network analyses.

Loss of action-related function and connectivity in the blind extrastriate body area

The Extrastriate Body Area (EBA) participates in the visual perception and motor actions of body parts. We recently showed that EBA’s perceptual function develops independently of visual experience, responding to stimuli with body-part information in a supramodal fashion. However, it is still unclear if the EBA similarly maintains its action-related function. Here, we used fMRI to study motor-evoked responses and connectivity patterns in the congenitally blind brain. We found that, unlike the case of perception, EBA does not develop an action-related response without visual experience. In addition, we show that congenital blindness alters EBA’s connectivity profile in a counterintuitive way—functional connectivity with sensorimotor cortices dramatically decreases, whereas connectivity with perception-related visual occipital cortices remains high. To the best of our knowledge, we show for the first time that action-related functions and connectivity in the visual cortex could be contingent on visuomotor experience. We further discuss the role of the EBA within the context of visuomotor control and predictive coding theory.

Visual Plasticity in Adulthood: Perspectives from Hebbian and Homeostatic Plasticity

The visual system retains profound plastic potential in adulthood. In the current review, we summarize the evidence of preserved plasticity in the adult visual system during visual perceptual learning as well as both monocular and binocular visual deprivation. In each condition, we discuss how such evidence reflects two major cellular mechanisms of plasticity: Hebbian and homeostatic processes. We focus on how these two mechanisms work together to shape plasticity in the visual system. In addition, we discuss how these two mechanisms could be further revealed in future studies investigating cross-modal plasticity in the visual system.

Testing geometry and 3D perception in children following vision restoring cataract-removal surgery

As neuroscience and rehabilitative techniques advance, age-old questions concerning the visual experience of those who gain sight after blindness, once thought to be philosophical alone, take center stage and become the target for scientific inquiries. In this study, we employ a battery of visual perception tasks to study the unique experience of a small group of children who have undergone vision-restoring cataract removal surgery as part of the Himalayan Cataract Project. We tested their abilities to perceive in three dimensions (3D) using a binocular rivalry task and the Brock string task, perceive visual illusions, use cross-modal mappings between touch and vision, and spatially group based on geometric cues. Some of the children in this study gained a sense of sight for the first time in their lives, having been born with bilateral congenital cataracts, while others suffered late-onset blindness in one eye alone. This study simultaneously supports yet raises further questions concerning Hubel and Wiesel's critical periods theory and provides additional insight into Molyneux's problem, the ability to correlate vision with touch quickly. We suggest that our findings present a relatively unexplored intermediate stage of 3D vision development. Importantly, we spotlight some essential geometrical perception visual abilities that strengthen the idea that spontaneous geometry intuitions arise independently from visual experience (and education), thus replicating and extending previous studies. We incorporate a new model, not previously explored, of testing children with congenital cataract removal surgeries who perform the task via vision. In contrast, previous work has explored these abilities in the congenitally blind via touch. Taken together, our findings provide insight into the development of what is commonly known as the visual system in the visually deprived and highlight the need to further empirically explore an amodal, task-based interpretation of specializations in the development and structure of the brain. Moreover, we propose a novel objective method, based on a simple binocular rivalry task and the Brock string task, for determining congenital (early) vs. late blindness where medical history and records are partial or lacking (e.g., as is often the case in cataract removal cases).

Shape detection beyond the visual field using a visual-to-auditory sensory augmentation device

Current advancements in both technology and science allow us to manipulate our sensory modalities in new and unexpected ways. In the present study, we explore the potential of expanding what we perceive through our natural senses by utilizing a visual-to-auditory sensory substitution device (SSD), the EyeMusic, an algorithm that converts images to sound. The EyeMusic was initially developed to allow blind individuals to create a spatial representation of information arriving from a video feed at a slow sampling rate. In this study, we aimed to use the EyeMusic for the blind areas of sighted individuals. We use it in this initial proof-of-concept study to test the ability of sighted subjects to combine visual information with surrounding auditory sonification representing visual information. Participants in this study were tasked with recognizing and adequately placing the stimuli, using sound to represent the areas outside the standard human visual field. As such, the participants were asked to report shapes' identities as well as their spatial orientation (front/right/back/left), requiring combined visual (90° frontal) and auditory input (the remaining 270°) for the successful performance of the task (content in both vision and audition was presented in a sweeping clockwise motion around the participant). We found that participants were successful at a highly above chance level after a brief 1-h-long session of online training and one on-site training session of an average of 20 min. They could even draw a 2D representation of this image in some cases. Participants could also generalize, recognizing new shapes they were not explicitly trained on. Our findings provide an initial proof of concept indicating that sensory augmentation devices and techniques can potentially be used in combination with natural sensory information in order to expand the natural fields of sensory perception.

Neural substrates of spatial processing and navigation in blindness: An activation likelihood estimation meta-analysis

Even though vision is considered the best suited sensory modality to acquire spatial information, blind individuals can form spatial representations to navigate and orient themselves efficiently in space. Consequently, many studies support the amodality hypothesis of spatial representations since sensory modalities other than vision contribute to the formation of spatial representations, independently of visual experience and imagery. However, given the high variability in abilities and deficits observed in blind populations, a clear consensus about the neural representations of space has yet to be established. To this end, we performed a meta-analysis of the literature on the neural correlates of spatial processing and navigation via sensory modalities other than vision, like touch and audition, in individuals with early and late onset blindness. An activation likelihood estimation (ALE) analysis of the neuroimaging literature revealed that early blind individuals and sighted controls activate the same neural networks in the processing of non-visual spatial information and navigation, including the posterior parietal cortex, frontal eye fields, insula, and the hippocampal complex. Furthermore, blind individuals also recruit primary and associative occipital areas involved in visuo-spatial processing via cross-modal plasticity mechanisms. The scarcity of studies involving late blind individuals did not allow us to establish a clear consensus about the neural substrates of spatial representations in this specific population. In conclusion, the results of our analysis on neuroimaging studies involving early blind individuals support the amodality hypothesis of spatial representations.

Multisensory representations of space and time in sensory cortices

Clear evidence demonstrated a supramodal organization of sensory cortices with multisensory processing occurring even at early stages of information encoding. Within this context, early recruitment of sensory areas is necessary for the development of fine domain-specific (i.e., spatial or temporal) skills regardless of the sensory modality involved, with auditory areas playing a crucial role in temporal processing and visual areas in spatial processing. Given the domain-specificity and the multisensory nature of sensory areas, in this study, we hypothesized that preferential domains of representation (i.e., space and time) of visual and auditory cortices are also evident in the early processing of multisensory information. Thus, we measured the event-related potential (ERP) responses of 16 participants while performing multisensory spatial and temporal bisection tasks. Audiovisual stimuli occurred at three different spatial positions and time lags and participants had to evaluate whether the second stimulus was spatially (spatial bisection task) or temporally (temporal bisection task) farther from the first or third audiovisual stimulus. As predicted, the second audiovisual stimulus of both spatial and temporal bisection tasks elicited an early ERP response (time window 50-90 ms) in visual and auditory regions. However, this early ERP component was more substantial in the occipital areas during the spatial bisection task, and in the temporal regions during the temporal bisection task. Overall, these results confirmed the domain specificity of visual and auditory cortices and revealed that this aspect selectively modulates also the cortical activity in response to multisensory stimuli.

A case study in phenomenology of visual experience with retinal prosthesis versus visual-to-auditory sensory substitution

The phenomenology of the blind has provided an age-old, unparalleled means of exploring the enigmatic link between the brain and mind. This paper delves into the unique phenomenological experience of a man who became blind in adulthood. He subsequently underwent both an Argus II retinal prosthesis implant and training, and extensive training on the EyeMusic visual to auditory sensory substitution device (SSD), thereby becoming the first reported case to date of dual proficiency with both devices. He offers a firsthand account into what he considers the great potential of combining sensory substitution devices with visual prostheses as part of a complete visual restoration protocol. While the Argus II retinal prosthesis alone provided him with immediate visual percepts by way of electrically stimulated phosphenes elicited by the device, the EyeMusic SSD requires extensive training from the onset. Yet following the extensive training program with the EyeMusic sensory substitution device, our subject reports that the sensory substitution device allowed him to experience a richer, more complex perceptual experience, that felt more "second nature" to him, while the Argus II prosthesis (which also requires training) did not allow him to achieve the same levels of automaticity and transparency. Following long-term use of the EyeMusic SSD, our subject reported that visual percepts representing mainly, but not limited to, colors portrayed by the EyeMusic SSD are elicited in association with auditory stimuli, indicating the acquisition of a high level of automaticity. Finally, the case study indicates an additive benefit to the combination of both devices on the user's subjective phenomenological visual experience.

Altered Spontaneous Brain Activity and Network Property in Patients With Congenital Monocular Blindness

Individuals with congenital monocular blindness may have specific brain changes since the brain is prenatally deprived of half the normal visual input. To explore characteristic brain functional changes of congenital monocular blindness, we analyzed resting-state functional MRI (rs-fMRI) data of 16 patients with unilateral congenital microphthalmia and 16 healthy subjects with normal vision to compare intergroup differences of amplitude of low frequency fluctuations (ALFFs), functional connectivity (FC), and network topolgoical properties. Compared with controls, patients with microphthalmia exhibited significantly lower ALFF values in the left inferior occipital and temporal gyri, superior temporal gyrus, inferior parietal lobe and post-central gyrus, whereas higher ALFF in the right middle and inferior temporal gyri, middle and superior frontal gyri, left superior frontal, and temporal gyri, such as angular gyrus. Meanwhile, FC between left medial superior frontal gyrus and angular gyrus, FC between left superior temporal gyrus and inferior parietal lobe and post-central gyrus decreased in the patients with congenital microphthalmia. In addition, a graph theory-analysis revealed increased regional network metrics (degree centrality and nodal efficiency) in the middle and inferior temporal gyri and middle and superior frontal gyri, while decreased values in the inferior occipital and temporal gyri, inferior parietal lobule, post-central gyrus, and angular gyrus. Taken together, patients with congenital microphthalmia had widespread abnormal activities within neural networks involving the vision and language and language-related regions played dominant roles in their brain networks. These findings may provide clues for functional reorganization of vision and language networks induced by the congenital monocular blindness.

Training-induced plasticity enables visualizing sounds with a visual-to-auditory conversion device

Sensory substitution devices aim at restoring visual functions by converting visual information into auditory or tactile stimuli. Although these devices show promise in the range of behavioral abilities they allow, the processes underlying their use remain underspecified. In particular, while an initial debate focused on the visual versus auditory or tactile nature of sensory substitution, since over a decade, the idea that it reflects a mixture of both has emerged. In order to investigate behaviorally the extent to which visual and auditory processes are involved, participants completed a Stroop-like crossmodal interference paradigm before and after being trained with a conversion device which translates visual images into sounds. In addition, participants' auditory abilities and their phenomenologies were measured. Our study revealed that, after training, when asked to identify sounds, processes shared with vision were involved, as participants’ performance in sound identification was influenced by the simultaneously presented visual distractors. In addition, participants’ performance during training and their associated phenomenology depended on their auditory abilities, revealing that processing finds its roots in the input sensory modality. Our results pave the way for improving the design and learning of these devices by taking into account inter-individual differences in auditory and visual perceptual strategies.

Brain-Machine Interfaces to Assist the Blind

The loss or absence of vision is probably one of the most incapacitating events that can befall a human being. The importance of vision for humans is also reflected in brain anatomy as approximately one third of the human brain is devoted to vision. It is therefore unsurprising that throughout history many attempts have been undertaken to develop devices aiming at substituting for a missing visual capacity. In this review, we present two concepts that have been prevalent over the last two decades. The first concept is sensory substitution, which refers to the use of another sensory modality to perform a task that is normally primarily sub-served by the lost sense. The second concept is cross-modal plasticity, which occurs when loss of input in one sensory modality leads to reorganization in brain representation of other sensory modalities. Both phenomena are training-dependent. We also briefly describe the history of blindness from ancient times to modernity, and then proceed to address the means that have been used to help blind individuals, with an emphasis on modern technologies, invasive (various type of surgical implants) and non-invasive devices. With the advent of brain imaging, it has become possible to peer into the neural substrates of sensory substitution and highlight the magnitude of the plastic processes that lead to a rewired brain. Finally, we will address the important question of the value and practicality of the available technologies and future directions.

Social cognition in the blind brain: A coordinate-based meta-analysis

Social cognition skills are typically acquired on the basis of visual information (e.g., the observation of gaze, facial expressions, gestures). In light of this, a critical issue is whether and how the lack of visual experience affects neurocognitive mechanisms underlying social skills. This issue has been largely neglected in the literature on blindness, despite difficulties in social interactions may be particular salient in the life of blind individuals (especially children). Here we provide a meta-analysis of neuroimaging studies reporting brain activations associated to the representation of self and others' in early blind individuals and in sighted controls. Our results indicate that early blindness does not critically impact on the development of the "social brain," with social tasks performed on the basis of auditory or tactile information driving consistent activations in nodes of the action observation network, typically active during actual observation of others in sighted individuals. Interestingly though, activations along this network appeared more left-lateralized in the blind than in sighted participants. These results may have important implications for the development of specific training programs to improve social skills in blind children and young adults.

Brain plasticity dynamics during tactile Braille learning in sighted subjects: Multi-contrast MRI approach

A growing body of empirical evidence supports the notion of diverse neurobiological processes underlying learning-induced plasticity changes in the human brain. There are still open questions about how brain plasticity depends on cognitive task complexity, how it supports interactions between brain systems and with what temporal and spatial trajectory. We investigated brain and behavioural changes in sighted adults during 8-months training of tactile Braille reading whilst monitoring brain structure and function at 5 different time points. We adopted a novel multivariate approach that includes behavioural data and specific MRI protocols sensitive to tissue properties to assess local functional and structural and myelin changes over time. Our results show that while the reading network, located in the ventral occipitotemporal cortex, rapidly adapts to tactile input, sensory areas show changes in grey matter volume and intra-cortical myelin at different times. This approach has allowed us to examine and describe neuroplastic mechanisms underlying complex cognitive systems and their (sensory) inputs and (motor) outputs differentially, at a mesoscopic level.

Evidence for an effector-independent action system from people born without hands

What are the principles of brain organization? In the motor domain, separate pathways were found for reaching and grasping actions performed by the hand. To what extent is this organization specific to the hand or based on abstract action types, regardless of which body part performs them? We tested people born without hands who perform actions with their feet. Activity in frontoparietal association motor areas showed preference for an action type (reaching or grasping), regardless of whether it was performed by the foot in people born without hands or by the hand in typically-developed controls. These findings provide evidence that some association areas are organized based on abstract functions of action types, independent of specific sensorimotor experience and parameters of specific body parts.

Many parts of the visuomotor system guide daily hand actions, like reaching for and grasping objects. Do these regions depend exclusively on the hand as a specific body part whose movement they guide, or are they organized for the reaching task per se, for any body part used as an effector? To address this question, we conducted a neuroimaging study with people born without upper limbs—individuals with dysplasia—who use the feet to act, as they and typically developed controls performed reaching and grasping actions with their dominant effector. Individuals with dysplasia have no prior experience acting with hands, allowing us to control for hand motor imagery when acting with another effector (i.e., foot). Primary sensorimotor cortices showed selectivity for the hand in controls and foot in individuals with dysplasia. Importantly, we found a preference based on action type (reaching/grasping) regardless of the effector used in the association sensorimotor cortex, in the left intraparietal sulcus and dorsal premotor cortex, as well as in the basal ganglia and anterior cerebellum. These areas also showed differential response patterns between action types for both groups. Intermediate areas along a posterior–anterior gradient in the left dorsal premotor cortex gradually transitioned from selectivity based on the body part to selectivity based on the action type. These findings indicate that some visuomotor association areas are organized based on abstract action functions independent of specific sensorimotor parameters, paralleling sensory feature-independence in visual and auditory cortices in people born blind and deaf. Together, they suggest association cortices across action and perception may support specific computations, abstracted from low-level sensorimotor elements.

Distinctive Interaction Between Cognitive Networks and the Visual Cortex in Early Blind Individuals

In early blind individuals, brain activation by a variety of nonperceptual cognitive tasks extends to the visual cortex, while in the sighted it is restricted to supramodal association areas. We hypothesized that such activation results from the integration of different sectors of the visual cortex into typical task-dependent networks. We tested this hypothesis with fMRI in blind and sighted subjects using tasks assessing speech comprehension, incidental long-term memory and both verbal and nonverbal executive control, in addition to collecting resting-state data. All tasks activated the visual cortex in blind relative to sighted subjects, which enabled its segmentation according to task sensitivity. We then assessed the unique brain-scale functional connectivity of the segmented areas during resting state. Language-related seeds were preferentially connected to frontal and temporal language areas; the seed derived from the executive task was connected to the right dorsal frontoparietal executive network; and the memory-related seed was uniquely connected to mesial frontoparietal areas involved in episodic memory retrieval. Thus, using a broad set of language, executive, and memory tasks in the same subjects, combined with resting state connectivity, we demonstrate the selective integration of different patches of the visual cortex into brain-scale networks with distinct localization, lateralization, and functional roles.

Available Sensory Input Determines Motor Performance and Strategy in Early Blind and Sighted Short-Tailed Opossums

The early loss of vision results in a reorganized neocortex, affecting areas of the brain that process both the spared and lost senses, and leads to heightened abilities on discrimination tasks involving the spared senses. Here, we used performance measures and machine learning algorithms that quantify behavioral strategy to determine if and how early vision loss alters adaptive sensorimotor behavior. We tested opossums on a motor task involving somatosensation and found that early blind animals had increased limb placement accuracy compared with sighted controls, while showing similarities in crossing strategy. However, increased reliance on tactile inputs in early blind animals resulted in greater deficits in limb placement and behavioral flexibility when the whiskers were trimmed. These data show that compensatory cross-modal plasticity extends beyond sensory discrimination tasks to motor tasks involving the spared senses and highlights the importance of whiskers in guiding forelimb control.

•

Early blind opossums outperform sighted controls during ladder rung walking

•

Whisker trimming causes forelimb accuracy deficits in blind and sighted opossums

•

Whisker trimming, but not the loss of vision, impacts stereotypical movements

•

Both groups adopt conservative approaches to ladder crossing after whisker trimming

Navigation aid for blind persons by visual-to-auditory sensory substitution: A pilot study

PURPOSE

In this study, we investigate to what degree augmented reality technology can be used to create and evaluate a visual-to-auditory sensory substitution device to improve the performance of blind persons in navigation and recognition tasks.

METHODS

A sensory substitution algorithm that translates 3D visual information into audio feedback was designed. This algorithm was integrated in an augmented reality based mobile phone application. Using the mobile device as sensory substitution device, a study with blind participants (n = 7) was performed. The participants navigated through pseudo-randomized obstacle courses using either the sensory substitution device, a white cane or a combination of both. In a second task, virtual 3D objects and structures had to be identified by the participants using the same sensory substitution device.

RESULTS

The realized application for mobile devices enabled participants to complete the navigation and object recognition tasks in an experimental environment already within the first trials without previous training. This demonstrates the general feasibility and low entry barrier of the designed sensory substitution algorithm. In direct comparison to the white cane, within the study duration of ten hours the sensory substitution device did not offer a statistically significant improvement in navigation.

Posterior parietal cortex contributions to cross-modal brain plasticity upon sensory loss

Sensory loss causes compensatory behavior, like echolocation upon vision loss or improved visual motion detection upon deafness. This is enabled by recruitment of the deprived cortical area by the intact senses. Such cross-modal plasticity can however hamper rehabilitation via sensory substitution devices. To steer rehabilitation towards the desired outcome for the patient, having control over the cross-modal take-over is essential. Evidence accumulates to support a role for the posterior parietal cortex (PPC) in multimodal plasticity. This area shows increased activity after sensory loss, keeping similar functions but driven by other senses. Patient-specific factors like stress, social situation, age and attention, have a significant influence on the PPC and on cross-modal plasticity. We propose that understanding the response of the PPC to sensory loss and context is extremely important for determining the best possible implant-based therapies, and that mouse research holds potential to help unraveling the underlying anatomical, cellular and neuromodulatory mechanisms.

Spatial Competence and Brain Plasticity in Congenital Blindness via Sensory Substitution Devices

In congenital blindness (CB), tactile, and auditory information can be reinterpreted by the brain to compensate for visual information through mechanisms of brain plasticity triggered by training. Visual deprivation does not cause a cognitive spatial deficit since blind people are able to acquire spatial knowledge about the environment. However, this spatial competence takes longer to achieve but is eventually reached through training-induced plasticity. Congenitally blind individuals can further improve their spatial skills with the extensive use of sensory substitution devices (SSDs), either visual-to-tactile or visual-to-auditory. Using a combination of functional and anatomical neuroimaging techniques, our recent work has demonstrated the impact of spatial training with both visual to tactile and visual to auditory SSDs on brain plasticity, cortical processing, and the achievement of certain forms of spatial competence. The comparison of performances between CB and sighted people using several different sensory substitution devices in perceptual and sensory-motor tasks uncovered the striking ability of the brain to rewire itself during perceptual learning and to interpret novel sensory information even during adulthood. We discuss here the implications of these findings for helping blind people in navigation tasks and to increase their accessibility to both real and virtual environments.

Reduced perceptual narrowing in synesthesia

Synesthesia is a neurologic trait in which specific inducers, such as sounds, automatically elicit additional idiosyncratic percepts, such as color (thus "colored hearing"). One explanation for this trait-and the one tested here-is that synesthesia results from unusually weak pruning of cortical synaptic hyperconnectivity during early perceptual development. We tested the prediction from this hypothesis that synesthetes would be superior at making discriminations from nonnative categories that are normally weakened by experience-dependent pruning during a critical period early in development-namely, discrimination among nonnative phonemes (Hindi retroflex /d̪a/ and dental /ɖa/), among chimpanzee faces, and among inverted human faces. Like the superiority of 6-mo-old infants over older infants, the synesthetic groups were significantly better than control groups at making all the nonnative discriminations across five samples and three testing sites. The consistent superiority of the synesthetic groups in making discriminations that are normally eliminated during infancy suggests that residual cortical connectivity in synesthesia supports changes in perception that extend beyond the specific synesthetic percepts, consistent with the incomplete pruning hypothesis.

Spatial metric in blindness: behavioural and cortical processing

Visual modality dominates spatial perception and, in lack of vision, space representation might be altered. Here we review our work showing that blind individuals have a strong deficit when performing spatial bisection tasks (Gori et al., 2014). We also describe the neural correlates associated with this deficit, as blind individuals do not show the same ERP response mimicking the visual C1 reported in sighted people during spatial bisection (Campus et al., 2019). Interestingly, the deficit is not always evident in late blind individuals, and it is dependent on blindness duration. We report that the deficit disappears when one presents coherent temporal and spatial cues to blind people. This suggests that they may use time information to infer spatial maps (Gori et al., 2018). Finally, we propose a model to explain why blind individuals are impaired in this task, speculating that a lack of vision drives the construction of a multi-sensory cortical network that codes space based on temporal, rather than spatial, coordinates.

A conceptual framework for plasticity in the developing brain

In this chapter, we highlight the various definitions of early brain plasticity commonly used in the scientific literature. We then present a conceptual framework of early brain plasticity that focuses on plasticity at the level of the synapse (synaptic plasticity) and the level of the network (connectivity). The proposed framework is organized around three main domains through which current theories and principles of early brain plasticity can be integrated: (1) the mechanisms of plasticity and constraints at the synaptic level and network connectivity, (2) the importance of temporal considerations related to the development of the immature brain, and (3) the functions early brain plasticity serve. We then apply this framework to discuss some clinical disorders caused by and/or associated with impaired plasticity mechanisms. We propose that a careful examination of the relationship between mechanisms, constraints, and functions of early brain plasticity in health and disease may provide an integrative understanding of the current theories and principles generated by experimental and observational studies.

A Novel Approach to Sensorimotor Skill Acquisition Utilizing Sensory Substitution: A Driving Simulation Study

The aim of this study is to demonstrate the potential of sensory substitution/augmentation (SS/A) techniques for driver assistance systems in a simulated driving environment. Using a group-comparison design, we examined lane-keeping skill acquisition in a driving simulator that can provide information regarding vehicle lateral position by changing the binaural balance of auditory white noise delivered to the driver. Consequently, lane-keeping accuracy was significantly degraded when the lower visual scene (proximal part of the road) was occluded, suggesting it conveyed critical visual information necessary for lane keeping. After 40 minutes of training with auditory cueing of vehicle lateral position, lane-keeping accuracy returned to the baseline (normal driving) level. This indicates that auditory cueing can compensate for the loss of visual information. Taken together, our data suggest that auditory cueing of vehicle lateral position is sufficient for lane-keeping skill acquisition and that SS/A techniques can potentially be used for the development of driver assistance systems, particularly for situations where immediate time-sensitive actions are required in response to rapidly changing sensory information. Although this study is the first to apply SS/A techniques to driver assistance, further studies are however required to establish the generalizability of the findings to real-world settings.

A thalamocortical pathway for fast rerouting of tactile information to occipital cortex in congenital blindness

In congenitally blind individuals, the occipital cortex responds to various nonvisual inputs. Some animal studies raise the possibility that a subcortical pathway allows fast re-routing of tactile information to the occipital cortex, but this has not been shown in humans. Here we show using magnetoencephalography (MEG) that tactile stimulation produces occipital cortex activations, starting as early as 35 ms in congenitally blind individuals, but not in blindfolded sighted controls. Given our measured thalamic response latencies of 20 ms and a mean estimated lateral geniculate nucleus to primary visual cortex transfer time of 15 ms, we claim that this early occipital response is mediated by a direct thalamo-cortical pathway. We also observed stronger directed connectivity in the alpha band range from posterior thalamus to occipital cortex in congenitally blind participants. Our results strongly suggest the contribution of a fast thalamo-cortical pathway in the cross-modal activation of the occipital cortex in congenitally blind humans.

In congenitally blind people, tactile stimuli can activate the occipital (visual) cortex. Here, the authors show using magnetoencephalography (MEG) that occipital activation can occur within 35 ms following tactile stimulation, suggesting the existence of a fast thalamocortical pathway for touch in congenitally blind humans.

Retinotopic-like maps of spatial sound in primary 'visual' cortex of blind human echolocators

The functional specializations of cortical sensory areas were traditionally viewed as being tied to specific modalities. A radically different emerging view is that the brain is organized by task rather than sensory modality, but it has not yet been shown that this applies to primary sensory cortices. Here, we report such evidence by showing that primary 'visual' cortex can be adapted to map spatial locations of sound in blind humans who regularly perceive space through sound echoes. Specifically, we objectively quantify the similarity between measured stimulus maps for sound eccentricity and predicted stimulus maps for visual eccentricity in primary 'visual' cortex (using a probabilistic atlas based on cortical anatomy) to find that stimulus maps for sound in expert echolocators are directly comparable to those for vision in sighted people. Furthermore, the degree of this similarity is positively related with echolocation ability. We also rule out explanations based on top-down modulation of brain activity-e.g. through imagery. This result is clear evidence that task-specific organization can extend even to primary sensory cortices, and in this way is pivotal in our reinterpretation of the functional organization of the human brain.

Peripheral sounds elicit stronger activity in contralateral occipital cortex in blind than sighted individuals

Previous research has shown that peripheral, task-irrelevant sounds elicit activity in contralateral visual cortex of sighted people, as revealed by a sustained positive deflection in the event-related potential (ERP) over the occipital scalp contralateral to the sound’s location. This Auditory-evoked Contralateral Occipital Positivity (ACOP) appears between 200–450 ms after sound onset, and is present even when the task is entirely auditory and no visual stimuli are presented at all. Here, we investigate whether this cross-modal activation of contralateral visual cortex is influenced by visual experience. To this end, ERPs were recorded in 12 sighted and 12 blind subjects during a unimodal auditory task. Participants listened to a stream of sounds and pressed a button every time they heard a central target tone, while ignoring the peripheral noise bursts. It was found that task-irrelevant noise bursts elicited a larger ACOP in blind compared to sighted participants, indicating for the first time that peripheral sounds can enhance neural activity in visual cortex in a spatially lateralized manner even in visually deprived individuals. Overall, these results suggest that the cross-modal activation of contralateral visual cortex triggered by peripheral sounds does not require any visual input to develop, and is rather enhanced by visual deprivation.

Visual imagery and visual perception induce similar changes in occipital slow waves of sleep

Previous studies have shown that regional slow-wave activity (SWA) during non-rapid eye movement (NREM) sleep is modulated by prior experience and learning. Although this effect has been convincingly demonstrated for the sensorimotor domain, attempts to extend these findings to the visual system have provided mixed results. In this study we asked whether depriving subjects of external visual stimuli during daytime would lead to regional changes in slow waves during sleep and whether the degree of "internal visual stimulation" (spontaneous imagery) would influence such changes. In two 8-h sessions spaced 1 wk apart, 12 healthy volunteers either were blindfolded while listening to audiobooks or watched movies (control condition), after which their sleep was recorded with high-density EEG. We found that during NREM sleep, the number of small, local slow waves in the occipital cortex decreased after listening with blindfolding relative to movie watching in a way that depended on the degree of visual imagery subjects reported during blindfolding: subjects with low visual imagery showed a significant reduction of occipital sleep slow waves, whereas those who reported a high degree of visual imagery did not. We also found a positive relationship between the reliance on visual imagery during blindfolding and audiobook listening and the degree of correlation in sleep SWA between visual areas and language-related areas. These preliminary results demonstrate that short-term alterations in visual experience may trigger slow-wave changes in cortical visual areas. Furthermore, they suggest that plasticity-related EEG changes during sleep may reflect externally induced ("bottom up") visual experiences, as well as internally generated ("top down") processes. NEW & NOTEWORTHY Previous work has shown that slow-wave activity, a marker of sleep depth, is linked to neural plasticity in the sensorimotor cortex. We show that after short-term visual deprivation, subjects who reported little visual imagery had a reduced incidence of occipital slow waves. This effect was absent in subjects who reported strong spontaneous visual imagery. These findings suggest that visual imagery may "substitute" for visual perception and induce similar changes in non-rapid eye movement slow waves.

Why would Parkinson's disease lead to sudden changes in creativity, motivation, or style with visual art?: A review of case evidence and new neurobiological, contextual, and genetic hypotheses

Parkinson's disease (PD) is a devastating diagnosis with, however, potential for an extremely intriguing aesthetic component. Despite motor and cognitive deficits, an emerging collection of studies report a burst of visual artistic output and alterations in produced art in a subgroup of patients. This provides a unique window into the neurophysiological bases for why and how we might create and enjoy visual art, as well as into general brain function and the nature of PD or other neurodegenerative diseases. However, there has not been a comprehensive organization of literature on this topic. Nor has there been an attempt to connect case evidence and knowledge on PD with present understanding of visual art making in psychology and neuroaesthetics in order to propose hypotheses for documented artistic changes. Here, we collect the current research on this topic, tie this to PD symptoms and neurobiology, and provide new theories focusing on dopaminergic neuron damage, over-stimulation from dopamine agonist therapy, and context or genetic factors revealing the neurobiological basis of the visual artistic brain.

Can Simulated Green Exercise Improve Recovery From Acute Mental Stress?

This exploratory study enhances previous research into green exercise and addresses a gap in the research by exploring the contribution of individual and combined senses in the recovery of mood and stress after a psychological stressor, whilst rigorously controlling exercise intensity. The hypotheses were: (i) recovery of mood and stress from a state of psychological stress would be greater following simulated green exercise compared to rest, (ii) green exercise would facilitate better recovery than exercise alone, (iii) these effects would remain 10 min following intervention, and (iv) visual stimuli alone would enhance recovery from a state of psychological stress compared to sound. Fifty participants were randomly assigned to one of five groups: REST, exercise, exercise with nature sounds, exercise with nature visual and exercise with nature sound and visual. An initial visit to obtain predicted peak power output values and to familiarize participants with the equipment being used was followed by a second visit, where participants experienced one test condition. Baseline measures of heart rate, blood pressure, total mood disturbance (TMD), and perceived stress were taken, before participants completed a stressor based on the Trier Social Stress test. Measures of heart rate and blood pressure were recorded in the last 30 s of the stressor to assess efficacy of the stressor. Immediately post stressor, measures of mood and perceived stress were taken followed by the intervention assigned (one of five described above). Measures of mood and perceived stress were taken again immediately post intervention and 10 min post intervention. Results showed that green exercise improved mood and stress scores more than exercise alone or REST. For both TMD and perceived stress, improvements in all simulated nature conditions were significantly improved compared to REST or exercise alone immediately post intervention. There were no significant changes 10 min post intervention in either mood or perceived stress compared to immediately post intervention values in any of the groups. This study suggests that environmental exercise settings including nature sounds, visual or both combined should be considered as important in the use of exercise as a therapeutic activity or recovery from acute psychological stress.

The Effect of Blindness on Long-Term Episodic Memory for Odors and Sounds

We recently showed that compared with sighted, early blind individuals have better episodic memory for environmental sounds, but not odors, after a short retention interval (∼ 8 – 9 min). Few studies have investigated potential effects of blindness on memory across long time frames, such as months or years. Consequently, it was unclear whether compensatory effects may vary as a function of retention interval. In this study, we followed-up participants (N = 57 out of 60) approximately 1 year after the initial testing and retested episodic recognition for environmental sounds and odors, and identification ability. In contrast to our previous findings, the early blind participants (n = 14) performed at a similar level as the late blind (n = 13) and sighted (n = 30) participants for sound recognition. Moreover, the groups had similar recognition performance of odors and identification ability of odors and sounds. These findings suggest that episodic odor memory is unaffected by blindness after both short and long retention intervals. However, the effect of blindness on episodic memory for sounds may vary as a function of retention interval, such that early blind individuals have an advantage over sighted across short but not long time frames. We speculate that the finding of a differential effect of blindness on auditory episodic memory across retention intervals may be related to different memory strategies at initial and follow-up assessments. In conclusion, this study suggests that blindness does not influence auditory or olfactory episodic memory as assessed after a long retention interval.

The spatial representation of number, time, and serial order following sensory deprivation: A systematic review

The spatial representation of numerical and temporal information is thought to be rooted in our multisensory experiences. Accordingly, we may expect visual or auditory deprivation to affect the way we represent numerical magnitude and time spatially. Here, we systematically review recent findings on how blind and deaf individuals represent abstract concepts such as magnitude and time (e.g., past/future, serial order of events) in a spatial format. Interestingly, available evidence suggests that sensory deprivation does not prevent the spatial "re-mapping" of abstract information, but differences compared to normally sighted and hearing individuals may emerge depending on the specific dimension considered (i.e., numerical magnitude, time as past/future, serial order). Herein we discuss how the study of sensory deprived populations may shed light on the specific, and possibly distinct, mechanisms subserving the spatial representation of these concepts. Furthermore, we pinpoint unresolved issues that need to be addressed by future studies to grasp a full understanding of the spatial representation of abstract information associated with visual and auditory deprivation.

Peripersonal space representation develops independently from visual experience

Our daily-life actions are typically driven by vision. When acting upon an object, we need to represent its visual features (e.g. shape, orientation, etc.) and to map them into our own peripersonal space. But what happens with people who have never had any visual experience? How can they map object features into their own peripersonal space? Do they do it differently from sighted agents? To tackle these questions, we carried out a series of behavioral experiments in sighted and congenitally blind subjects. We took advantage of a spatial alignment effect paradigm, which typically refers to a decrease of reaction times when subjects perform an action (e.g., a reach-to-grasp pantomime) congruent with that afforded by a presented object. To systematically examine peripersonal space mapping, we presented visual or auditory affording objects both within and outside subjects’ reach. The results showed that sighted and congenitally blind subjects did not differ in mapping objects into their own peripersonal space. Strikingly, this mapping occurred also when objects were presented outside subjects’ reach, but within the peripersonal space of another agent. This suggests that (the lack of) visual experience does not significantly affect the development of both one’s own and others’ peripersonal space representation.

Task Selectivity as a Comprehensive Principle for Brain Organization

How do the anatomically consistent functional selectivities of the brain emerge? A new study by Bola and colleagues reveals task selectivity in auditory rhythm-selective areas in congenitally deaf adults perceiving visual rhythm sequences. Here, we contextualize this result with accumulating evidence from animal and human studies supporting sensory-independent task specializations as a comprehensive principle shaping brain (re)organization.