Neural correlates of natural human echolocation in early and late blind echolocation experts

Longitudinal stability of individual brain plasticity patterns in blindness

The primary visual cortex (V1) in blindness is engaged in a wide spectrum of tasks and sensory modalities, including audition, touch, language, and memory. This widespread involvement raises questions regarding the constancy of its role and whether it might exhibit flexibility in its function over time, connecting to diverse network functions specific to task demands. This would suggest that reorganized V1 assumes a role like multiple-demand system regions. Alternatively, varying patterns of plasticity in blind V1 may be attributed to individual factors, with different blind individuals recruiting V1 preferentially for different functions. In support of this, we recently showed that V1 functional connectivity (FC) varies greatly across blind individuals. But do these represent stable individual patterns of plasticity, or are they driven more by instantaneous changes, like a multiple-demand system now inhabiting V1? Here, we tested whether individual FC patterns from the V1 of blind individuals are stable over time. We show that over two years, FC from the V1 is unique and highly stable in a small sample of repeatedly sampled congenitally blind individuals. Further, using multivoxel pattern analysis, we demonstrate that the unique reorganization patterns of these individuals allow decoding of participant identity. Together with recent evidence for substantial individual differences in V1 connectivity, this indicates that there may be a consistent role for V1 in blindness, which may differ for each individual. Further, it suggests that the variability in visual reorganization in blindness across individuals could be used to seek stable neuromarkers for sight rehabilitation and assistive approaches.

Changes in primary visual and auditory cortex of blind and sighted adults following 10 weeks of click-based echolocation training

Recent work suggests that the adult human brain is very adaptable when it comes to sensory processing. In this context, it has also been suggested that structural "blueprints" may fundamentally constrain neuroplastic change, e.g. in response to sensory deprivation. Here, we trained 12 blind participants and 14 sighted participants in echolocation over a 10-week period, and used MRI in a pre-post design to measure functional and structural brain changes. We found that blind participants and sighted participants together showed a training-induced increase in activation in left and right V1 in response to echoes, a finding difficult to reconcile with the view that sensory cortex is strictly organized by modality. Further, blind participants and sighted participants showed a training induced increase in activation in right A1 in response to sounds per se (i.e. not echo-specific), and this was accompanied by an increase in gray matter density in right A1 in blind participants and in adjacent acoustic areas in sighted participants. The similarity in functional results between sighted participants and blind participants is consistent with the idea that reorganization may be governed by similar principles in the two groups, yet our structural analyses also showed differences between the groups suggesting that a more nuanced view may be required.

Preliminary investigation of SEZUAL device for basic material identification and simple spatial navigation for blind and visually impaired people

PURPOSE

we present a preliminary set of experimental studies that demonstrates device-aided echolocation enabling in blind and visually impaired individuals. The proposed device emits a click-like sound into the surrounding space and returning sound is perceived by participants to infer the surrounding environment.

MATERIALS AND METHODS

two sets of experiments were set up to evaluate the echolocation abilities of nine blind participants. The first setup was designed to identify four material types based on the sound reflection properties of materials, such as glass, metal, wood, and ceramics. The second setup was navigation through a basic maze with the device.

RESULTS

experimental data demonstrate that the use of the proposed device enables active echolocation abilities in blind participants, particularly for material identification and spatial mobility.

CONCLUSION

the proposed device can potentially be used to rehabilitate disabled blind and visually impaired individuals in terms of spatial mobility and orientation.

Developing cortex is functionally pluripotent: Evidence from blindness

How rigidly does innate architecture constrain function of developing cortex? What is the contribution of early experience? We review insights into these questions from visual cortex function in people born blind. In blindness, occipital cortices are active during auditory and tactile tasks. What 'cross-modal' plasticity tells us about cortical flexibility is debated. On the one hand, visual networks of blind people respond to higher cognitive information, such as sentence grammar, suggesting drastic repurposing. On the other, in line with 'metamodal' accounts, sighted and blind populations show shared domain preferences in ventral occipito-temporal cortex (vOTC), suggesting visual areas switch input modality but perform the same or similar perceptual functions (e.g., face recognition) in blindness. Here we bring these disparate literatures together, reviewing and synthesizing evidence that speaks to whether visual cortices have similar or different functions in blind and sighted people. Together, the evidence suggests that in blindness, visual cortices are incorporated into higher-cognitive (e.g., fronto-parietal) networks, which are a major source long-range input to the visual system. We propose the connectivity-constrained experience-dependent account. Functional development is constrained by innate anatomical connectivity, experience and behavioral needs. Infant cortex is pluripotent, the same anatomical constraints develop into different functional outcomes.

Representational maps in the brain: concepts, approaches, and applications

Neural systems have evolved to process sensory stimuli in a way that allows for efficient and adaptive behavior in a complex environment. Recent technological advances enable us to investigate sensory processing in animal models by simultaneously recording the activity of large populations of neurons with single-cell resolution, yielding high-dimensional datasets. In this review, we discuss concepts and approaches for assessing the population-level representation of sensory stimuli in the form of a representational map. In such a map, not only are the identities of stimuli distinctly represented, but their relational similarity is also mapped onto the space of neuronal activity. We highlight example studies in which the structure of representational maps in the brain are estimated from recordings in humans as well as animals and compare their methodological approaches. Finally, we integrate these aspects and provide an outlook for how the concept of representational maps could be applied to various fields in basic and clinical neuroscience.

Object recognition via echoes: quantifying the crossmodal transfer of three-dimensional shape information between echolocation, vision, and haptics

Active echolocation allows blind individuals to explore their surroundings via self-generated sounds, similarly to dolphins and other echolocating animals. Echolocators emit sounds, such as finger snaps or mouth clicks, and parse the returning echoes for information about their surroundings, including the location, size, and material composition of objects. Because a crucial function of perceiving objects is to enable effective interaction with them, it is important to understand the degree to which three-dimensional shape information extracted from object echoes is useful in the context of other modalities such as haptics or vision. Here, we investigated the resolution of crossmodal transfer of object-level information between acoustic echoes and other senses. First, in a delayed match-to-sample task, blind expert echolocators and sighted control participants inspected common (everyday) and novel target objects using echolocation, then distinguished the target object from a distractor using only haptic information. For blind participants, discrimination accuracy was overall above chance and similar for both common and novel objects, whereas as a group, sighted participants performed above chance for the common, but not novel objects, suggesting that some coarse object information (a) is available to both expert blind and novice sighted echolocators, (b) transfers from auditory to haptic modalities, and (c) may be facilitated by prior object familiarity and/or material differences, particularly for novice echolocators. Next, to estimate an equivalent resolution in visual terms, we briefly presented blurred images of the novel stimuli to sighted participants (N = 22), who then performed the same haptic discrimination task. We found that visuo-haptic discrimination performance approximately matched echo-haptic discrimination for a Gaussian blur kernel σ of ~2.5°. In this way, by matching visual and echo-based contributions to object discrimination, we can estimate the quality of echoacoustic information that transfers to other sensory modalities, predict theoretical bounds on perception, and inform the design of assistive techniques and technology available for blind individuals.

Longitudinal stability of individual brain plasticity patterns in blindness

The primary visual cortex (V1) in individuals born blind is engaged in a wide spectrum of tasks and sensory modalities, including audition, touch, language, and memory. This widespread involvement raises questions regarding the constancy of its role and whether it might exhibit flexibility in its function over time, connecting to diverse network functions in response to task-specific demands. This would suggest that reorganized V1 takes on a role similar to cognitive multiple-demand system regions. Alternatively, it is possible that the varying patterns of plasticity observed in the blind V1 can be attributed to individual factors, whereby different blind individuals recruit V1 for different functions, highlighting the immense idiosyncrasy of plasticity. In support of this second account, we have recently shown that V1 functional connectivity varies greatly across blind individuals. But do these represent stable individual patterns of plasticity or merely instantaneous changes, for a multiple-demand system now inhabiting V1? Here we tested if individual connectivity patterns from the visual cortex of blind individuals are stable over time. We show that over two years, fMRI functional connectivity from the primary visual cortex is unique and highly stable in a small sample of repeatedly sampled congenitally blind individuals. Further, using multivoxel pattern analysis, we demonstrate that the unique reorganization patterns of these individuals allow decoding of participant identity. Together with recent evidence for substantial individual differences in visual cortex connectivity, this indicates there may be a consistent role for the visual cortex in blindness, which may differ for each individual. Further, it suggests that the variability in visual reorganization in blindness across individuals could be used to seek stable neuromarkers for sight rehabilitation and assistive approaches.

Linguistic Relativity in Cross-Cultural Context: Converging Evidence From Neuroanthropology

Kemmerer's discussion of links between semantic typology and embodied cognition is welcome, especially his survey of available evidence. Focusing on mechanisms of embodied enculturation, however, we must understand that language is just one part of developmental assemblies that shape cognition, alongside other cultural elements such as sensory learning, behavior patterns, social interactions, and emotional experience. We believe that a source of this problem is an obsolete definition of "culture" as shared mental information that is inconsistent with models of embodied cognition and yet pervasive in human and cognitive sciences. We point to microethnographies of cognitive ecologies as a tractable remedy.

The effects of abnormal visual experience on neurodevelopmental disorders

Normal visual development is supported by intrinsic neurobiological mechanisms and by appropriate stimulation from the environment, both of which facilitate the maturation of visual functions. However, an offset of this balance can give rise to visual disorders. Therefore, understanding the factors that support normal vision during development and in the mature brain is important, as vision guides movement, enables social interaction, and allows children to recognize and understand their environment. In this paper, we review fundamental mechanisms that support the maturation of visual functions and discuss and draw links between the perceptual and neurobiological impairments in autism spectrum disorder (ASD) and schizophrenia. We aim to explore how this is evident in the case of ASD, and how perceptual and neurobiological deficits further degrade social ability. Furthermore, we describe the altered perceptual experience of those with schizophrenia and evaluate theories of the underlying neural deficits that alter perception.

Binaural Listening with Head Rotation Helps Persons with Blindness Perceive Narrow Obstacles

Orientation and mobility (O&M) are important abilities that people with visual impairments use in their independent performance of daily activities. In orientation, people with total blindness pinpoint nonsounding objects and sounding objects. The ability to perceive nonsounding objects is called obstacle sense, wherein people with blindness recognize the various characteristics of an obstacle using acoustic cues. Although body movement and listening style may enhance the sensing of obstacles, experimental studies on this topic are lacking. Elucidating their contributions to obstacle sense may lead to the further systematization of techniques of O&M training. This study sheds light on the contribution of head rotation and binaural hearing to obstacle sense among people with blindness. We conducted an experiment on the perceived presence and distance of nonsounding obstacles, which varied width and distance, for participants with blindness under the conditions of binaural or monaural hearing, with or without head rotation. The results indicated that head rotation and binaural listening can enhance the localization of nonsounding obstacles. Further, when people with blindness are unable to perform head rotation or use binaural hearing, their judgment can become biased in favor of the presence of an obstacle due to risk avoidance.

EchoRead Programme: Learning echolocation skills through self-paced professional development during the COVID-19 pandemic

Echolocation is used by people with low vision or blindness to support their navigation. Internationally, Orientation and Mobility (O&M) Specialists have learned echolocation skills and how to teach them to clients, through formal workshops with a subject matter expert. However, COVID-19 has limited access to these in-person professional development opportunities. This study investigated whether an O&M professional could learn echolocation skills in a self-paced programme with only the support of a lay assistant. We developed the EchoRead Programme to equip an individual O&M Specialist to learn basic echolocation skills in 4 hours. This auto-ethnographical perspective describes how the draft programme was trialled by one trainee O&M Specialist in her home and local neighbourhood. She developed sufficient skills to complete most of the seated, standing, and walking tasks in the programme, but needed more support developing tongue-clicking and recognising driveways when shorelining fences. She found it was important to use learning environments that were graduated in physical and audio complexity. The EchoRead Programme was then trialled and revised by an experienced O&M Specialist, beginning at home, then exploring a range of venues available within a 5 km radius – the roaming range allowed during COVID lockdown. The resulting EchoRead Programme can equip O&M professionals to be self-directed in learning early echolocation skills, using online and locally available resources. This programme could be especially useful for vision professionals and their clients, who have limited access to in-person learning opportunities with colleagues or peers because of geographical isolation, low resources, or a global pandemic.

Objective Evaluation of Obstacle Perception Using Spontaneous Body Movements of Blind People Evoked by Movements of Acoustic Virtual Wall

Obstacle perception using sound is the ability to detect silent objects, such as walls and poles. It is very important for blind people to recognize their environment using acoustic information through their auditory sense when walking or conducting various daily activities. In this paper, to develop an objective method for evaluating the degree of obstacle perception acquisition in the education and rehabilitation of the blind, the authors measured the spontaneous body movements evoked by the approach of an acoustic virtual wall. Ten blind persons who have experienced obstacle perception in their daily life, and seven sighted persons with no such experience participated in the experiment. The reciprocal (approach and receding) movements of the virtual wall were presented using simulated reflected sound, and the spontaneous body movements of the subjects were measured. As the results indicate, eight of the ten blind participants showed large maximum values for the correlation function between the wall and their body movements, whereas six of the seven sighted participants showed small maximum values. These results indicate that body movements can be used for an objective evaluation of obstacle perception. In particular, it was determined that the maximum value of the correlation function is the most appropriate for such an evaluation, because it does not depend on the subject’s physique.

Neural substrates of spatial processing and navigation in blindness: An activation likelihood estimation meta-analysis

Even though vision is considered the best suited sensory modality to acquire spatial information, blind individuals can form spatial representations to navigate and orient themselves efficiently in space. Consequently, many studies support the amodality hypothesis of spatial representations since sensory modalities other than vision contribute to the formation of spatial representations, independently of visual experience and imagery. However, given the high variability in abilities and deficits observed in blind populations, a clear consensus about the neural representations of space has yet to be established. To this end, we performed a meta-analysis of the literature on the neural correlates of spatial processing and navigation via sensory modalities other than vision, like touch and audition, in individuals with early and late onset blindness. An activation likelihood estimation (ALE) analysis of the neuroimaging literature revealed that early blind individuals and sighted controls activate the same neural networks in the processing of non-visual spatial information and navigation, including the posterior parietal cortex, frontal eye fields, insula, and the hippocampal complex. Furthermore, blind individuals also recruit primary and associative occipital areas involved in visuo-spatial processing via cross-modal plasticity mechanisms. The scarcity of studies involving late blind individuals did not allow us to establish a clear consensus about the neural substrates of spatial representations in this specific population. In conclusion, the results of our analysis on neuroimaging studies involving early blind individuals support the amodality hypothesis of spatial representations.

Echovis – A collection of human echolocation tests performed by blind and sighted individuals: A pilot study

The article presents research on the echolocation skills of blind and sighted individuals gathered for the purpose of development of an echolocation training app. The goal of the research was to determine the influence of the environment type, reverberation, clicking patterns, and the average differences between the two tested groups. Ten blind and 10 sighted subjects were tested in various echolocation tasks – stationary, moving, indoor and outdoor, using mechanical clickers, and artificially generated clicks. Ten blind children also took part in the static indoor tests. The tests were repeated using binaural recordings and spatially rendered virtual audio. The following parameters and dependencies between them were analyzed: correctness of the obstacle localization, certainty of answer, type of environment and clicker sound, and frequency and number of clicks. It was found that the number of clicks influenced the correctness and certainty of the answer in case of obstacle direction determination, but not the distance. Better results were obtained in outdoor environments and in an empty room, what implies that reverberation has a positive influence on echolocation. The expected success rates in tested echolocation tasks provided a comparison of the echolocation abilities of blind and sighted subjects and set a benchmark for future tests.

Human Echolocators Have Better Localization Off Axis

Here, we report novel empirical results from a psychophysical experiment in which we tested the echolocation abilities of nine blind adult human experts in click-based echolocation. We found that they had better acuity in localizing a target and used lower intensity emissions (i.e., mouth clicks) when a target was placed 45° off to the side compared with when it was placed at 0° (straight ahead). We provide a possible explanation of the behavioral result in terms of binaural-intensity signals, which appear to change more rapidly around 45°. The finding that echolocators have better echo-localization off axis is surprising, because for human source localization (i.e., regular spatial hearing), it is well known that performance is best when targets are straight ahead (0°) and decreases as targets move farther to the side. This may suggest that human echolocation and source hearing rely on different acoustic cues and that human spatial hearing has more facets than previously thought.

Spatial acuity of the bottlenose dolphin (Tursiops truncatus) biosonar system with a bat and human comparison

Horizontal angular resolution was measured in two bottlenose dolphins using a two-alternative forced-choice, biosonar target discrimination paradigm. The task required a stationary dolphin positioned in a hoop to discriminate two physical targets at a range of 4 m. The angle separating the targets was manipulated to estimate an angular discrimination threshold of 1.5°. In a second experiment, a similar two-target biosonar discrimination task was conducted with one free-swimming dolphin, to test whether its emission beam was a critical factor in discriminating the targets. The spatial separation between two targets was manipulated to measure a discrimination threshold of 6.7 cm. There was a relationship between differences in acoustic signals received at each target and the dolphin's performance. The results of the angular resolution experiment were in good agreement with measures of the minimum audible angle of both dolphins and humans and remarkably similar to measures of angular difference discrimination in echolocating dolphins, bats, and humans. The results suggest that horizontal auditory spatial acuity may be a common feature of the mammalian auditory system rather than a specialized feature exclusive to echolocating auditory predators.

Effects of Congenital Blindness on Ultrasonic Vocalizations and Social Behaviors in the ZRDBA Mouse

Mice produce ultrasonic vocalizations (USVs) at different ages and social contexts, including maternal-pup separation, social play in juveniles, social interactions, and mating in adults. The USVs' recording can be used as an index of sensory detection, internal state, and social motivation. While sensory deprivation may alter USVs' emission and some social behaviors in deaf and anosmic rodents, little is known about the effects of visual deprivation in rodents. This longitudinal study aimed to assess acoustic communication and social behaviors using a mouse model of congenital blindness. Anophthalmic and sighted mice were assayed to a series of behavioral tests at three different ages, namely, the maternal isolation-induced pup USV test and the home odor discrimination and preference test on postnatal day (PND) 7, the juvenile social test on PND 30-35, and the female urine-induced USVs and scent-marking behavior at 2-3 months. Our results evidenced that (1) at PND 7, USVs' total number between both groups was similar, all mice vocalized less during the second isolation period than the first period, and both phenotypes showed similar discrimination and preference, favoring exploration of the home bedding odor; (2) at PND 30-35, anophthalmic mice engaged less in social behaviors in the juvenile play test than sighted ones, but the number of total USVs produced is not affected; and (3) at adulthood, when exposed to a female urine spot, anophthalmic male mice displayed faster responses in terms of USVs' emission and sniffing behavior, associated with a longer time spent exploring the female urinary odor. Interestingly, acoustic behavior in the pups and adults was correlated in sighted mice only. Together, our study reveals that congenital visual deprivation had no effect on the number of USVs emitted in the pups and juveniles, but affected the USVs' emission in the adult male and impacted the social behavior in juvenile and adult mice.

A Participatory Design Approach to Creating Echolocation-Enabled Virtual Environments

As virtual environments—in the form of videogames and augmented and virtual reality experiences—become more popular, it is important to ensure that they are accessible to all. Previous research has identified echolocation as a useful interaction approach to enable people with visual impairment to access virtual environments. In this paper, we further investigate the usefulness of echolocation to explore virtual environments. We follow a participatory design approach that comprised a focus group session coupled with two fast prototyping and evaluation iterations. During the focus group session, expert echolocators produced a series of seven design recommendations, of which we implemented and trialed four. Our trials revealed that the use of ambient sounds, the ability to place landmarks, directional control, and the ability to use pre-recorded mouth-clicks produced by expert echolocators improved the overall experience of our participants by facilitating the detection of openings and obstacles. The recommendations presented and evaluated in this paper may help to develop virtual environments that support a broader range of users while recognising the value of the lived experience of people with disability as a source of knowledge.

Superior verbal but not nonverbal memory in congenital blindness

Previous studies suggest that people who are congenitally blind outperform sighted people on some memory tasks. Whether blindness-associated memory advantages are specific to verbal materials or are also observed with nonverbal sounds has not been determined. Congenitally blind individuals (n = 20) and age and education matched blindfolded sighted controls (n = 22) performed a series of auditory memory tasks. These included: verbal forward and backward letter spans, a complex letter span with intervening equations, as well as two matched recognition tasks: one with verbal stimuli (i.e., letters) and one with nonverbal complex meaningless sounds. Replicating previously observed findings, blind participants outperformed sighted people on forward and backward letter span tasks. Blind participants also recalled more letters on the complex letter span task despite the interference of intervening equations. Critically, the same blind participants showed larger advantages on the verbal as compared to the nonverbal recognition task. These results suggest that blindness selectively enhances memory for verbal material. Possible explanations for blindness-related verbal memory advantages include blindness-induced memory practice and 'visual' cortex recruitment for verbal processing.

Naturalistic Audio-Movies reveal common spatial organization across "visual" cortices of different blind individuals

Occipital cortices of different sighted people contain analogous maps of visual information (e.g. foveal vs. peripheral). In congenital blindness, "visual" cortices respond to nonvisual stimuli. Do visual cortices of different blind people represent common informational maps? We leverage naturalistic stimuli and inter-subject pattern similarity analysis to address this question. Blindfolded sighted (n = 22) and congenitally blind (n = 22) participants listened to 6 sound clips (5-7 min each): 3 auditory excerpts from movies; a naturalistic spoken narrative; and matched degraded auditory stimuli (Backwards Speech, scrambled sentences), during functional magnetic resonance imaging scanning. We compared the spatial activity patterns evoked by each unique 10-s segment of the different auditory excerpts across blind and sighted people. Segments of meaningful naturalistic stimuli produced distinctive activity patterns in frontotemporal networks that were shared across blind and across sighted individuals. In the blind group only, segment-specific, cross-subject patterns emerged in visual cortex, but only for meaningful naturalistic stimuli and not Backwards Speech. Spatial patterns of activity within visual cortices are sensitive to time-varying information in meaningful naturalistic auditory stimuli in a broadly similar manner across blind individuals.

Urban Parks as Perceived by City Residents with Mobility Difficulties: A Qualitative Study with In-Depth Interviews

Urban green spaces make an invaluable contribution to the health and well-being of all city residents. Therefore, urban park quality and accessibility are crucial factors in stimulating physical and mental health benefits. This study aimed to assess the quality of urban parks and their accessibility as reported by people with mobility difficulties (seniors, blind and partially sighted people). Four key features of a place (accessibility and linkages, comfort and image, uses and activities and sociability) were considered in an in-depth-interviews (IDI) and “walk-and-talk” interviews. Study results indicate a problem of accessibility of urban parks for people with mobility difficulties (uneven gravel surfaces). However, non-physical aspects of park visits (social activities, cultural events, place branding) were reported as essential factors in explaining the motivation for park visits. Despite individual preferences, experience or reported difficulties, all respondents’ attitudes towards park trips were positive. Therefore, we assume that accessibility is more than just physical comfort. Cultural and social activities play an important role in motivating people with a disability to visit a park.

A natural history of vertebrate vision loss: Insight from mammalian vision for human visual function

Research on the origin of vision and vision loss in naturally "blind" animal species can reveal the tasks that vision fulfills and the brain's role in visual experience. Models that incorporate evolutionary history, natural variation in visual ability, and experimental manipulations can help disentangle visual ability at a superficial level from behaviors linked to vision but not solely reliant upon it, and could assist the translation of ophthalmological research in animal models to human treatments. To unravel the similarities between blind individuals and blind species, we review concepts of 'blindness' and its behavioral correlates across a range of species. We explore the ancestral emergence of vision in vertebrates, and the loss of vision in blind species with reference to an evolution-based classification scheme. We applied phylogenetic comparative methods to a mammalian tree to explore the evolution of visual acuity using ancestral state estimations. Future research into the natural history of vision loss could help elucidate the function of vision and inspire innovations in how to address vision loss in humans.

Ecological validity in exercise neuroscience research: A systematic investigation

The contribution of cortical processes to adaptive motor behaviour is of great interest in the field of exercise neuroscience. Next to established criteria of objectivity, reliability and validity, ecological validity refers to the concerns of whether measurements and behaviour in research settings are representative of the real world. Because exercise neuroscience investigations using mobile electroencephalography are oftentimes conducted in laboratory settings under controlled environments, methodological approaches may interfere with the idea of ecological validity. This review utilizes an original ecological validity tool to assess the degree of ecological validity in current exercise neuroscience research. A systematic literature search was conducted to identify articles investigating cortical dynamics during goal-directed sports movement. To assess ecological validity, five elements (environment, stimulus, response, body and mind) were assessed on a continuum of artificiality-naturality and simplicity-complexity. Forty-seven studies were included in the present review. Results indicate lowest average ratings for the element of environment. The elements stimulus, body and mind had mediocre ratings, and the element of response had the highest overall ratings. In terms of the type of sport, studies that assessed closed-skill indoor sports had the highest ratings, whereas closed-skill outdoor sports had the lowest overall rating. Our findings identify specific elements that are lacking in ecological validity and areas of improvement in current exercise neuroscience literature. Future studies may potentially increase ecological validity by moving from reductionist, artificial environments towards complex, natural environments and incorporating real-world sport elements such as adaptive responses and competition.

Phenomenal Holism and Cognitive Phenomenology

The cognitive phenomenology debate centers on two questions. (1) What is an apt characterization of the phenomenology of conscious thought? And (2), what role does this phenomenology play? I argue that the answers to the former question bear significantly on the answers to the latter question. In particular, I show that conservatism about cognitive phenomenology is not compatible with the view that phenomenology explains the constitution of conscious thought. I proceed as follows: To begin with, I analyze the phenomenology of our sensory experiences and argue for a weak phenomenal holism (WPH) about sensory phenomenology. Next, I explore how WPH can be integrated into the competing accounts of cognitive phenomenology. I argue that, given WPH, conservatism turns out to reduce phenomenal character to a merely concomitant phenomenon that has no explanatory power when it comes to the constitution of conscious thoughts. In contrast, liberalism is explanatorily more powerful in this respect. Finally, I propose a new version of liberalism that explains how phenomenology constitutes conscious thoughts and fits best with WPH.

Early auditory cortical processing predicts auditory speech in noise identification and lipreading

Individuals typically exhibit better cross-sensory perception following unisensory loss, demonstrating improved perception of information available from the remaining senses and increased cross-sensory use of neural resources. Even individuals with no sensory loss will exhibit such changes in cross-sensory processing following temporary sensory deprivation, suggesting that the brain's capacity for recruiting cross-sensory sources to compensate for degraded unisensory input is a general characteristic of the perceptual process. Many studies have investigated how auditory and visual neural structures respond to within- and cross-sensory input. However, little attention has been given to how general auditory and visual neural processing relates to within and cross-sensory perception. The current investigation examines the extent to which individual differences in general auditory neural processing accounts for variability in auditory, visual, and audiovisual speech perception in a sample of young healthy adults. Auditory neural processing was assessed using a simple click stimulus. We found that individuals with a smaller P1 peak amplitude in their auditory-evoked potential (AEP) had more difficulty identifying speech sounds in difficult listening conditions, but were better lipreaders. The results suggest that individual differences in the auditory neural processing of healthy adults can account for variability in the perception of information available from the auditory and visual modalities, similar to the cross-sensory perceptual compensation observed in individuals with sensory loss.

Merging familiar and new senses to perceive and act in space

Our experience of the world seems to unfold seamlessly in a unitary 3D space. For this to be possible, the brain has to merge many disparate cognitive representations and sensory inputs. How does it do so? I discuss work on two key combination problems: coordinating multiple frames of reference (e.g. egocentric and allocentric), and coordinating multiple sensory signals (e.g. visual and proprioceptive). I focus on two populations whose spatial processing we can observe at a crucial stage of being configured and optimised: children, whose spatial abilities are still developing significantly, and naïve adults learning new spatial skills, such as sensing distance using auditory cues. The work uses a model-based approach to compare participants’ behaviour with the predictions of alternative information processing models. This lets us see when and how—during development, and with experience—the perceptual-cognitive computations underpinning our experiences in space change. I discuss progress on understanding the limits of effective spatial computation for perception and action, and how lessons from the developing spatial cognitive system can inform approaches to augmenting human abilities with new sensory signals provided by technology.

Neuroplastic Reorganization Induced by Sensory Augmentation for Self-Localization During Locomotion

Sensory skills can be augmented through training and technological support. This process is underpinned by neural plasticity in the brain. We previously demonstrated that auditory-based sensory augmentation can be used to assist self-localization during locomotion. However, the neural mechanisms underlying this phenomenon remain unclear. Here, by using functional magnetic resonance imaging, we aimed to identify the neuroplastic reorganization induced by sensory augmentation training for self-localization during locomotion. We compared activation in response to auditory cues for self-localization before, the day after, and 1 month after 8 days of sensory augmentation training in a simulated driving environment. Self-localization accuracy improved after sensory augmentation training, compared with the control (normal driving) condition; importantly, sensory augmentation training resulted in auditory responses not only in temporal auditory areas but also in higher-order somatosensory areas extending to the supramarginal gyrus and the parietal operculum. This sensory reorganization had disappeared by 1 month after the end of the training. These results suggest that the use of auditory cues for self-localization during locomotion relies on multimodality in higher-order somatosensory areas, despite substantial evidence that information for self-localization during driving is estimated from visual cues on the proximal part of the road. Our findings imply that the involvement of higher-order somatosensory, rather than visual, areas is crucial for acquiring augmented sensory skills for self-localization during locomotion.

3D Imaging from Multipath Temporal Echoes

Echo location is a broad approach to imaging and sensing that includes both manmade RADAR, LIDAR, SONAR, and also animal navigation. However, full 3D information based on echo location requires some form of scanning of the scene in order to provide the spatial location of the echo origin-points. Without this spatial information, imaging objects in three-dimensional (3D) is a very challenging task as the inverse retrieval problem is strongly ill-posed. Here, we show that the temporal information encoded in the return echoes that are reflected multiple times within a scene is sufficient to faithfully render an image in 3D. Numerical modeling and an information theoretic perspective prove the concept and provide insight into the role of the multipath information. We experimentally demonstrate the concept by using both radio frequency and acoustic waves for imaging individuals moving in a closed environment.

Dance Is More Than Meets the Eye-How Can Dance Performance Be Made Accessible for a Non-sighted Audience?

Dance is regarded as visual art form by common arts and science perspectives. Definitions of dance as means of communication agree that its message is conveyed by the dancer/choreographer via the human body for the observer, leaving no doubt that dance is performed to be watched. Brain activation elicited by the visual perception of dance has also become a topic of interest in cognitive neuroscience, with regards to action observation in the context of learning, expertise and aesthetics. The view that the aesthetic experience of dance is primarily a visual one is still shared by many artists and cultural institutions, yet there is growing interest in making dance performances accessible for individuals with visual impairment / blindness. Means of supporting the non-visual experience of dance include verbal (audio description), auditive (choreographed body sounds, movement sonification), and haptic (touch tour) techniques, applied for different purposes by artists and researchers, with three main objectives: to strengthen the cultural participation of a non-sighted audience in the cultural and aesthetic experience of dance; to expand the scope of dance as an artistic research laboratory toward novel ways of perceiving what dance can convey; and to inspire new lines of (neuro-cognitive) research beyond watching dance. Reviewing literature from different disciplines and drawing on the personal experience of an inclusive performance of Simon Mayer's "Sons of Sissy," we argue that a non-exclusively visual approach can be enriching and promising for all three perspectives and conclude by proposing hypotheses for multidisciplinary lines of research.

Echolocation as a Means for People with Visual Impairment (PVI) to Acquire Spatial Knowledge of Virtual Space

In virtual environments, spatial information is communicated visually. This prevents people with visual impairment (PVI) from accessing such spaces. In this article, we investigate whether echolocation could be used as a tool to convey spatial information by answering the following research questions: What features of virtual space can be perceived by PVI through the use of echolocation? How does active echolocation support PVI in acquiring spatial knowledge of a virtual space? And what are PVI’s opinions regarding the use of echolocation to acquire landmark and survey knowledge of virtual space? To answer these questions, we conducted a two-part within-subjects experiment with 12 people who were blind or had a visual impairment and found that size and materials of rooms and 90-degree turns were detectable through echolocation, participants preferred using echoes derived from footsteps rather than from artificial sound pulses, and echolocation supported the acquisition of mental maps of a virtual space. Ultimately, we propose that appropriately designed echolocation in virtual environments improves understanding of spatial information and access to digital games for PVI.

Early visual cortex response for sound in expert blind echolocators, but not in early blind non-echolocators

Echolocation is a perceptual and navigational skill that can be acquired by some individuals. Regarding blind people, this skill can help them "see" the environment around them via a new form of auditory information based on echoes. Expert human echolocators benefit from using this technique not only in controlled environments but also in their everyday lives. In the current study, we investigate the effect of echolocation on blind people's auditory spatial abilities at the cortical level. In an auditory spatial bisection task, we tested people who are early blinds and early blind expert echolocators, along with sighted people. Our results showed that there is similar early activation (50-90 ms) in the posterior area of the scalp for both early blind expert echolocators and sighted participants, but not in the early blind group. This activation was related to sound stimulation, and it is contralateral to the position of the sound in space. These findings indicate that echolocation is a good substitute for the visual modality that enables the development of auditory spatial representations when vision is not available.

Efficient Multi-Object Detection and Smart Navigation Using Artificial Intelligence for Visually Impaired People

Visually impaired people face numerous difficulties in their daily life, and technological interventions may assist them to meet these challenges. This paper proposes an artificial intelligence-based fully automatic assistive technology to recognize different objects, and auditory inputs are provided to the user in real time, which gives better understanding to the visually impaired person about their surroundings. A deep-learning model is trained with multiple images of objects that are highly relevant to the visually impaired person. Training images are augmented and manually annotated to bring more robustness to the trained model. In addition to computer vision-based techniques for object recognition, a distance-measuring sensor is integrated to make the device more comprehensive by recognizing obstacles while navigating from one place to another. The auditory information that is conveyed to the user after scene segmentation and obstacle identification is optimized to obtain more information in less time for faster processing of video frames. The average accuracy of this proposed method is 95.19% and 99.69% for object detection and recognition, respectively. The time complexity is low, allowing a user to perceive the surrounding scene in real time.

Spatial Competence and Brain Plasticity in Congenital Blindness via Sensory Substitution Devices

In congenital blindness (CB), tactile, and auditory information can be reinterpreted by the brain to compensate for visual information through mechanisms of brain plasticity triggered by training. Visual deprivation does not cause a cognitive spatial deficit since blind people are able to acquire spatial knowledge about the environment. However, this spatial competence takes longer to achieve but is eventually reached through training-induced plasticity. Congenitally blind individuals can further improve their spatial skills with the extensive use of sensory substitution devices (SSDs), either visual-to-tactile or visual-to-auditory. Using a combination of functional and anatomical neuroimaging techniques, our recent work has demonstrated the impact of spatial training with both visual to tactile and visual to auditory SSDs on brain plasticity, cortical processing, and the achievement of certain forms of spatial competence. The comparison of performances between CB and sighted people using several different sensory substitution devices in perceptual and sensory-motor tasks uncovered the striking ability of the brain to rewire itself during perceptual learning and to interpret novel sensory information even during adulthood. We discuss here the implications of these findings for helping blind people in navigation tasks and to increase their accessibility to both real and virtual environments.

Binaural summation of amplitude modulation involves weak interaural suppression

The brain combines sounds from the two ears, but what is the algorithm used to achieve this summation of signals? Here we combine psychophysical amplitude modulation discrimination and steady-state electroencephalography (EEG) data to investigate the architecture of binaural combination for amplitude-modulated tones. Discrimination thresholds followed a ‘dipper’ shaped function of pedestal modulation depth, and were consistently lower for binaural than monaural presentation of modulated tones. The EEG responses were greater for binaural than monaural presentation of modulated tones, and when a masker was presented to one ear, it produced only weak suppression of the response to a signal presented to the other ear. Both data sets were well-fit by a computational model originally derived for visual signal combination, but with suppression between the two channels (ears) being much weaker than in binocular vision. We suggest that the distinct ecological constraints on vision and hearing can explain this difference, if it is assumed that the brain avoids over-representing sensory signals originating from a single object. These findings position our understanding of binaural summation in a broader context of work on sensory signal combination in the brain, and delineate the similarities and differences between vision and hearing.

Retinotopic-like maps of spatial sound in primary 'visual' cortex of blind human echolocators

The functional specializations of cortical sensory areas were traditionally viewed as being tied to specific modalities. A radically different emerging view is that the brain is organized by task rather than sensory modality, but it has not yet been shown that this applies to primary sensory cortices. Here, we report such evidence by showing that primary 'visual' cortex can be adapted to map spatial locations of sound in blind humans who regularly perceive space through sound echoes. Specifically, we objectively quantify the similarity between measured stimulus maps for sound eccentricity and predicted stimulus maps for visual eccentricity in primary 'visual' cortex (using a probabilistic atlas based on cortical anatomy) to find that stimulus maps for sound in expert echolocators are directly comparable to those for vision in sighted people. Furthermore, the degree of this similarity is positively related with echolocation ability. We also rule out explanations based on top-down modulation of brain activity-e.g. through imagery. This result is clear evidence that task-specific organization can extend even to primary sensory cortices, and in this way is pivotal in our reinterpretation of the functional organization of the human brain.

Texture Classification Using Spectral Entropy of Acoustic Signal Generated by a Human Echolocator

Human echolocation is a biological process wherein the human emits a punctuated acoustic signal, and the ear analyzes the echo in order to perceive the surroundings. The peculiar acoustic signal is normally produced by clicking inside the mouth. This paper utilized this unique acoustic signal from a human echolocator as a source of transmitted signal in a synthetic human echolocation technique. Thus, the aim of the paper was to extract information from the echo signal and develop a classification scheme to identify signals reflected from different textures at various distance. The scheme was based on spectral entropy extracted from Mel-scale filtering output in the Mel-frequency cepstrum coefficient of a reflected echo signal. The classification process involved data mining, features extraction, clustering, and classifier validation. The reflected echo signals were obtained via an experimental setup resembling a human echolocation scenario, configured for synthetic data collection. Unlike in typical speech signals, extracted entropy from the formant characteristics was likely not visible for the human mouth-click signals. Instead, multiple peak spectral features derived from the synthesis signal of the mouth-click were assumed as the entropy obtained from the Mel-scale filtering output. To realize the classification process, K-means clustering and K-nearest neighbor processes were employed. Moreover, the impacts of sound propagation toward the extracted spectral entropy used in the classification outcome were also investigated. The outcomes of the classifier performance herein indicated that spectral entropy is essential for human echolocation.

Naturalistic Audio-Movies and Narrative Synchronize "Visual" Cortices across Congenitally Blind But Not Sighted Individuals

How does developmental experience, as opposed to intrinsic physiology, shape cortical function? Naturalistic stimuli were used to elicit neural synchrony in individuals blind from birth (n = 18) and those who grew up with sight (n = 18). Blind and blindfolded sighted participants passively listened to three audio-movie clips, an auditory narrative, a sentence shuffled version of the narrative (maintaining language but lacking a plotline), and a version of the narrative backward (lacking both language and plot). For both groups, early auditory cortices were synchronized to a similar degree across stimulus types, whereas higher-cognitive temporoparietal and prefrontal areas were more synchronized by meaningful, temporally extended stimuli (i.e., audio movies and narrative). "Visual" cortices were more synchronized across blind than sighted individuals, but only for audio-movies and narrative. In the blind group, visual cortex synchrony was low for backward speech and intermediate for sentence shuffle. Meaningful auditory stimuli synchronize visual cortices of people born blind.SIGNIFICANCE STATEMENT Naturalistic stimuli engage cognitive processing at many levels. Here, we harnessed this richness to investigate the effect of experience on cortical function. We find that listening to naturalistic audio movies and narrative drives synchronized activity across "visual" cortices of blind, more so than sighted, individuals. Visual cortex synchronization varies with meaningfulness and cognitive complexity. Higher synchrony is observed for temporally extended meaningful stimuli (e.g., movies/narrative), intermediate for shuffled sentences, lowest for time varying complex noise. By contrast, auditory cortex was synchronized equally by meaningful and meaningless stimuli. In congenitally blind individuals most of visual cortex is engaged by meaningful naturalistic stimuli.

Peripheral sounds elicit stronger activity in contralateral occipital cortex in blind than sighted individuals

Previous research has shown that peripheral, task-irrelevant sounds elicit activity in contralateral visual cortex of sighted people, as revealed by a sustained positive deflection in the event-related potential (ERP) over the occipital scalp contralateral to the sound’s location. This Auditory-evoked Contralateral Occipital Positivity (ACOP) appears between 200–450 ms after sound onset, and is present even when the task is entirely auditory and no visual stimuli are presented at all. Here, we investigate whether this cross-modal activation of contralateral visual cortex is influenced by visual experience. To this end, ERPs were recorded in 12 sighted and 12 blind subjects during a unimodal auditory task. Participants listened to a stream of sounds and pressed a button every time they heard a central target tone, while ignoring the peripheral noise bursts. It was found that task-irrelevant noise bursts elicited a larger ACOP in blind compared to sighted participants, indicating for the first time that peripheral sounds can enhance neural activity in visual cortex in a spatially lateralized manner even in visually deprived individuals. Overall, these results suggest that the cross-modal activation of contralateral visual cortex triggered by peripheral sounds does not require any visual input to develop, and is rather enhanced by visual deprivation.

Visual hallucinations associated with multimodal hallucinations, suicide attempts and morbidity of illness in psychotic disorders

BACKGROUND

Visual hallucinations (VH) are a common, but understudied symptom of psychosis, experienced by individuals across diagnostic categories of psychotic and neuropsychiatric conditions. There are limited data on VH and associated clinical phenotypes in adult idiopathic psychotic disorders, which are needed to elucidate their relevance to psychotic illness paradigms.

METHOD

In this cross-sectional study, we examined clinical risk factors for VH in a well-characterized sample of 766 patients with adult psychotic disorders across diagnostic categories of schizophrenia (n = 227), schizoaffective disorder (n = 210), and bipolar I disorder (n = 329). The Structured Clinical Interview for DSM-IV-TR was used for diagnosis and symptom measurements.

RESULTS

The prevalence of VH was 26.1% (200/766). Multivariate logistic regression showed that VH were independently associated with the presence of hallucinations in other modalities, including auditory, tactile, olfactory, and gustatory hallucinations. History of a suicide attempt and catatonic behavior were also associated with VH. In addition, specific delusions were associated with VH, in particular, delusions of control, and religious, erotomanic and jealousy delusions. Diagnosis, negative symptoms, and family history of psychosis were not independent predictors of VH.

CONCLUSIONS

Results showed the clinical and disease relevance of VH as they were associated with severe morbidity of illness, including suicide attempts and catatonic behavior. Findings also suggest a phenotype associated with hallucinations in other modalities and specific types of delusions. Based on our findings, VH may be a significant factor in assessing for suicidality and illness severity, warranting clinical attention and further study of underlying mechanisms.

Bat-inspired signal design for target discrimination in human echolocation

Echolocating bats exhibit sophisticated sonar behaviors using ultrasounds with actively adjusted acoustic characteristics (e.g., frequency and time-frequency structure) depending on the situation. In this study, the utility of ultrasound in human echolocation was examined. By listening to ultrasonic echoes with a shifted pitch to be audible, the participants (i.e., sighted echolocation novices) could discriminate the three-dimensional (3D) roundness of edge contours. This finding suggests that sounds with suitable wavelengths (i.e., ultrasounds) can provide useful information about 3D shapes. In addition, the shape, texture, and material discrimination experiments were conducted using ultrasonic echoes binaurally measured with a 1/7 scaled miniature dummy head. The acoustic and statistical analyses showed that intensity and timbre cues were useful for shape and texture discriminations, respectively. Furthermore, in the discrimination of objects with various features (e.g., acrylic board and artificial grass), the perceptual distances between objects were more dispersed when frequency-modulated sweep signals were used than when a constant-frequency signal was used. These suggest that suitable signal design, i.e., echolocation sounds employed by bats, allowed echolocation novices to discriminate the 3D shape and texture. This top-down approach using human subjects may be able to efficiently help interpret the sensory perception, "seeing by sound," in bat biosonar.

How body motion influences echolocation while walking

This study investigated the influence of body motion on an echolocation task. We asked a group of blindfolded novice sighted participants to walk along a corridor, made with plastic sound-reflecting panels. By self-generating mouth clicks, the participants attempted to understand some spatial properties of the corridor, i.e. a left turn, a right turn or a dead end. They were asked to explore the corridor and stop whenever they were confident about the corridor shape. Their body motion was captured by a camera system and coded. Most participants were able to accomplish the task with the percentage of correct guesses above the chance level. We found a mutual interaction between some kinematic variables that can lead to optimal echolocation skills. These variables are head motion, accounting for spatial exploration, the motion stop-point of the person and the amount of correct guesses about the spatial structure. The results confirmed that sighted people are able to use self-generated echoes to navigate in a complex environment. The inter-individual variability and the quality of echolocation tasks seems to depend on how and how much the space is explored.

LIDAR Assist Spatial Sensing for the Visually Impaired and Performance Analysis

Echolocation enables people with impaired or no vision to comprehend the surrounding spatial information through the reflected sound. However, this technique often requires substantial training, and the accuracy of echolocation is subject to various conditions. Furthermore, the individuals who practice this sensing method must simultaneously generate the sound and process the received audio information. This paper proposes and evaluates a proof-of-concept light detection and ranging (LIDAR) assist spatial sensing (LASS) system, which intends to overcome these restrictions by obtaining the spatial information of the user's surroundings through a LIDAR sensor and translating the spatial information into the stereo sound of various pitches. The stereo sound of relative pitch represents the information regarding objects' angular orientation and horizontal distance, respectively, thus granting visually impaired users an enhanced spatial perception of his or her surrounding areas and potential obstacles. This paper is divided into two phases: Phase I is to engineer the hardware and software of the LASS system and Phase II focuses on the system efficacy study. The study, approved by the Penn State Institutional Review Board, included 18 student volunteers, who were recruited through the Penn State Department of Psychology Subject Pool. This paper demonstrates that the blindfolded individuals equipped with the LASS system are able to quantitatively identify the surrounding obstacles, differentiate their relative distance, and distinguish the angular location of multiple objects with minimal training.

Why Does the Cortex Reorganize after Sensory Loss?

A growing body of evidence demonstrates that the brain can reorganize dramatically following sensory loss. Although the existence of such neuroplastic crossmodal changes is not in doubt, the functional significance of these changes remains unclear. The dominant belief is that reorganization is compensatory. However, results thus far do not unequivocally indicate that sensory deprivation results in markedly enhanced abilities in other senses. Here, we consider alternative reasons besides sensory compensation that might drive the brain to reorganize after sensory loss. One such possibility is that the cortex reorganizes not to confer functional benefits, but to avoid undesirable physiological consequences of sensory deafferentation. Empirical assessment of the validity of this and other possibilities defines a rich program for future research.

Human Echolocation for Target Detection Is More Accurate With Emissions Containing Higher Spectral Frequencies, and This Is Explained by Echo Intensity

Humans can learn to use acoustic echoes to detect and classify objects. Echolocators typically use tongue clicks to induce these echoes, and there is some evidence that higher spectral frequency content of an echolocator’s tongue click is associated with better echolocation performance. This may be explained by the intensity of the echoes. The current study tested experimentally (a) if emissions with higher spectral frequencies lead to better performance for target detection, and (b) if this is mediated by echo intensity. Participants listened to sound recordings that contained an emission and sometimes an echo from an object. The peak spectral frequency of the emission was varied between 3.5 and 4.5 kHz. Participants judged whether they heard the object in these recordings and did the same under conditions in which the intensity of the echoes had been digitally equated. Participants performed better using emissions with higher spectral frequencies, but this advantage was eliminated when the intensity of the echoes was equated. These results demonstrate that emissions with higher spectral frequencies can benefit echolocation performance in conditions where they lead to an increase in echo intensity. The findings suggest that people who train to echolocate should be instructed to make emissions (e.g. mouth clicks) with higher spectral frequency content.