1
|
Kim H, Kim JS, Chung CK. Visual Mental Imagery and Neural Dynamics of Sensory Substitution in the Blindfolded Subjects. Neuroimage 2024; 295:120621. [PMID: 38797383 DOI: 10.1016/j.neuroimage.2024.120621] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/28/2023] [Revised: 04/16/2024] [Accepted: 04/18/2024] [Indexed: 05/29/2024] Open
Abstract
Although one can recognize the environment by soundscape substituting vision to auditory signal, whether subjects could perceive the soundscape as visual or visual-like sensation has been questioned. In this study, we investigated hierarchical process to elucidate the recruitment mechanism of visual areas by soundscape stimuli in blindfolded subjects. Twenty-two healthy subjects were repeatedly trained to recognize soundscape stimuli converted by visual shape information of letters. An effective connectivity method called dynamic causal modeling (DCM) was employed to reveal how the brain was hierarchically organized to recognize soundscape stimuli. The visual mental imagery model generated cortical source signals of five regions of interest better than auditory bottom-up, cross-modal perception, and mixed models. Spectral couplings between brain areas in the visual mental imagery model were analyzed. While within-frequency coupling is apparent in bottom-up processing where sensory information is transmitted, cross-frequency coupling is prominent in top-down processing, corresponding to the expectation and interpretation of information. Sensory substitution in the brain of blindfolded subjects derived visual mental imagery by combining bottom-up and top-down processing.
Collapse
Affiliation(s)
- HongJune Kim
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea
| | - June Sic Kim
- Clinical Research Institute, Konkuk University Medical Center Seoul, Republic of Korea; Research Institute of Biomedical Science & Technology, Konkuk University, Seoul, Republic of Korea.
| | - Chun Kee Chung
- Dept. of Brain and Cognitive Sciences, Seoul National University, Seoul, Republic of Korea; Interdisciplinary Program in Neuroscience, Seoul National University, Seoul, Republic of Korea; Dept. of Neurosurgery, Seoul National University Hospital, Seoul, Republic of Korea; Neuroscience Research Institute, Seoul National University Medical Research Center, Seoul, Republic of Korea
| |
Collapse
|
2
|
Qiu Y, Li H, Liao J, Chen K, Wu X, Liu B, Huang R. Forming cognitive maps for abstract spaces: the roles of the human hippocampus and orbitofrontal cortex. Commun Biol 2024; 7:517. [PMID: 38693344 PMCID: PMC11063219 DOI: 10.1038/s42003-024-06214-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 04/18/2024] [Indexed: 05/03/2024] Open
Abstract
How does the human brain construct cognitive maps for decision-making and inference? Here, we conduct an fMRI study on a navigation task in multidimensional abstract spaces. Using a deep neural network model, we assess learning levels and categorized paths into exploration and exploitation stages. Univariate analyses show higher activation in the bilateral hippocampus and lateral prefrontal cortex during exploration, positively associated with learning level and response accuracy. Conversely, the bilateral orbitofrontal cortex (OFC) and retrosplenial cortex show higher activation during exploitation, negatively associated with learning level and response accuracy. Representational similarity analysis show that the hippocampus, entorhinal cortex, and OFC more accurately represent destinations in exploitation than exploration stages. These findings highlight the collaboration between the medial temporal lobe and prefrontal cortex in learning abstract space structures. The hippocampus may be involved in spatial memory formation and representation, while the OFC integrates sensory information for decision-making in multidimensional abstract spaces.
Collapse
Affiliation(s)
- Yidan Qiu
- School of Psychology; Center for the Study of Applied Psychology; Key Laboratory of Mental Health and Cognitive Science of Guangdong Province; Key Laboratory of Brain, Cognition and Education Sciences, Ministry of Education; South China Normal University, Guangzhou, 510631, China
| | - Huakang Li
- School of Computer Science and Engineering, South China University of Technology, Guangzhou, 510006, China
| | - Jiajun Liao
- School of Psychology; Center for the Study of Applied Psychology; Key Laboratory of Mental Health and Cognitive Science of Guangdong Province; Key Laboratory of Brain, Cognition and Education Sciences, Ministry of Education; South China Normal University, Guangzhou, 510631, China
| | - Kemeng Chen
- School of Psychology; Center for the Study of Applied Psychology; Key Laboratory of Mental Health and Cognitive Science of Guangdong Province; Key Laboratory of Brain, Cognition and Education Sciences, Ministry of Education; South China Normal University, Guangzhou, 510631, China
| | - Xiaoyan Wu
- School of Psychology; Center for the Study of Applied Psychology; Key Laboratory of Mental Health and Cognitive Science of Guangdong Province; Key Laboratory of Brain, Cognition and Education Sciences, Ministry of Education; South China Normal University, Guangzhou, 510631, China
| | - Bingyi Liu
- School of Psychology; Center for the Study of Applied Psychology; Key Laboratory of Mental Health and Cognitive Science of Guangdong Province; Key Laboratory of Brain, Cognition and Education Sciences, Ministry of Education; South China Normal University, Guangzhou, 510631, China
| | - Ruiwang Huang
- School of Psychology; Center for the Study of Applied Psychology; Key Laboratory of Mental Health and Cognitive Science of Guangdong Province; Key Laboratory of Brain, Cognition and Education Sciences, Ministry of Education; South China Normal University, Guangzhou, 510631, China.
| |
Collapse
|
3
|
Chow JK, Palmeri TJ, Gauthier I. Distinct but related abilities for visual and haptic object recognition. Psychon Bull Rev 2024:10.3758/s13423-024-02471-x. [PMID: 38381302 DOI: 10.3758/s13423-024-02471-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/29/2024] [Indexed: 02/22/2024]
Abstract
People vary in their ability to recognize objects visually. Individual differences for matching and recognizing objects visually is supported by a domain-general ability capturing common variance across different tasks (e.g., Richler et al., Psychological Review, 126, 226-251, 2019). Behavioral (e.g., Cooke et al., Neuropsychologia, 45, 484-495, 2007) and neural evidence (e.g., Amedi, Cerebral Cortex, 12, 1202-1212, 2002) suggest overlapping mechanisms in the processing of visual and haptic information in the service of object recognition, but it is unclear whether such group-average results generalize to individual differences. Psychometrically validated measures are required, which have been lacking in the haptic modality. We investigate whether object recognition ability is specific to vision or extends to haptics using psychometric measures we have developed. We use multiple visual and haptic tests with different objects and different formats to measure domain-general visual and haptic abilities and to test for relations across them. We measured object recognition abilities using two visual tests and four haptic tests (two each for two kinds of haptic exploration) in 97 participants. Partial correlation and confirmatory factor analyses converge to support the existence of a domain-general haptic object recognition ability that is moderately correlated with domain-general visual object recognition ability. Visual and haptic abilities share about 25% of their variance, supporting the existence of a multisensory domain-general ability while leaving a substantial amount of residual variance for modality-specific abilities. These results extend our understanding of the structure of object recognition abilities; while there are mechanisms that may generalize across categories, tasks, and modalities, there are still other mechanisms that are distinct between modalities.
Collapse
Affiliation(s)
- Jason K Chow
- Department of Psychology, Vanderbilt University, 111 21st Avenue South, Nashville, TN, 37240, USA.
| | - Thomas J Palmeri
- Department of Psychology, Vanderbilt University, 111 21st Avenue South, Nashville, TN, 37240, USA
| | - Isabel Gauthier
- Department of Psychology, Vanderbilt University, 111 21st Avenue South, Nashville, TN, 37240, USA
| |
Collapse
|
4
|
Tivadar RI, Franceschiello B, Minier A, Murray MM. Learning and navigating digitally rendered haptic spatial layouts. NPJ SCIENCE OF LEARNING 2023; 8:61. [PMID: 38102127 PMCID: PMC10724186 DOI: 10.1038/s41539-023-00208-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Accepted: 11/28/2023] [Indexed: 12/17/2023]
Abstract
Learning spatial layouts and navigating through them rely not simply on sight but rather on multisensory processes, including touch. Digital haptics based on ultrasounds are effective for creating and manipulating mental images of individual objects in sighted and visually impaired participants. Here, we tested if this extends to scenes and navigation within them. Using only tactile stimuli conveyed via ultrasonic feedback on a digital touchscreen (i.e., a digital interactive map), 25 sighted, blindfolded participants first learned the basic layout of an apartment based on digital haptics only and then one of two trajectories through it. While still blindfolded, participants successfully reconstructed the haptically learned 2D spaces and navigated these spaces. Digital haptics were thus an effective means to learn and translate, on the one hand, 2D images into 3D reconstructions of layouts and, on the other hand, navigate actions within real spaces. Digital haptics based on ultrasounds represent an alternative learning tool for complex scenes as well as for successful navigation in previously unfamiliar layouts, which can likely be further applied in the rehabilitation of spatial functions and mitigation of visual impairments.
Collapse
Affiliation(s)
- Ruxandra I Tivadar
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
- Department of Ophthalmology, Fondation Asile des Aveugles, Lausanne, Switzerland.
- Centre for Integrative and Complementary Medicine, Department of Anesthesiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
- Cognitive Computational Neuroscience Group, Institute for Computer Science, University of Bern, Bern, Switzerland.
- The Sense Innovation and Research Center, Lausanne and Sion, Switzerland.
| | - Benedetta Franceschiello
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- The Sense Innovation and Research Center, Lausanne and Sion, Switzerland
- Institute of Systems Engineering, School of Engineering, University of Applied Sciences Western Switzerland (HES-SO Valais), Sion, Switzerland
| | - Astrid Minier
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland
- Department of Ophthalmology, Fondation Asile des Aveugles, Lausanne, Switzerland
| | - Micah M Murray
- The Department of Radiology, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland.
- Department of Ophthalmology, Fondation Asile des Aveugles, Lausanne, Switzerland.
- The Sense Innovation and Research Center, Lausanne and Sion, Switzerland.
| |
Collapse
|
5
|
Newell FN, McKenna E, Seveso MA, Devine I, Alahmad F, Hirst RJ, O'Dowd A. Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions. Philos Trans R Soc Lond B Biol Sci 2023; 378:20220342. [PMID: 37545304 PMCID: PMC10404931 DOI: 10.1098/rstb.2022.0342] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/10/2023] [Accepted: 06/29/2023] [Indexed: 08/08/2023] Open
Abstract
Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.
Collapse
Affiliation(s)
- F. N. Newell
- School of Psychology and Institute of Neuroscience, Trinity College Dublin, College Green, Dublin D02 PN40, Ireland
| | - E. McKenna
- School of Psychology and Institute of Neuroscience, Trinity College Dublin, College Green, Dublin D02 PN40, Ireland
| | - M. A. Seveso
- School of Psychology and Institute of Neuroscience, Trinity College Dublin, College Green, Dublin D02 PN40, Ireland
| | - I. Devine
- School of Psychology and Institute of Neuroscience, Trinity College Dublin, College Green, Dublin D02 PN40, Ireland
| | - F. Alahmad
- School of Psychology and Institute of Neuroscience, Trinity College Dublin, College Green, Dublin D02 PN40, Ireland
| | - R. J. Hirst
- School of Psychology and Institute of Neuroscience, Trinity College Dublin, College Green, Dublin D02 PN40, Ireland
| | - A. O'Dowd
- School of Psychology and Institute of Neuroscience, Trinity College Dublin, College Green, Dublin D02 PN40, Ireland
| |
Collapse
|
6
|
Korczyk M, Zimmermann M, Bola Ł, Szwed M. Superior visual rhythm discrimination in expert musicians is most likely not related to cross-modal recruitment of the auditory cortex. Front Psychol 2022; 13:1036669. [PMID: 36337485 PMCID: PMC9632485 DOI: 10.3389/fpsyg.2022.1036669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2022] [Accepted: 10/06/2022] [Indexed: 11/25/2022] Open
Abstract
Training can influence behavioral performance and lead to brain reorganization. In particular, training in one modality, for example, auditory, can improve performance in another modality, for example, visual. Previous research suggests that one of the mechanisms behind this phenomenon could be the cross-modal recruitment of the sensory areas, for example, the auditory cortex. Studying expert musicians offers a chance to explore this process. Rhythm is an aspect of music that can be presented in various modalities. We designed an fMRI experiment in which professional pianists and non-musicians discriminated between two sequences of rhythms presented auditorily (series of sounds) or visually (series of flashes). Behavioral results showed that musicians performed in both visual and auditory rhythmic tasks better than non-musicians. We found no significant between-group differences in fMRI activations within the auditory cortex. However, we observed that musicians had increased activation in the right Inferior Parietal Lobe when compared to non-musicians. We conclude that the musicians’ superior visual rhythm discrimination is not related to cross-modal recruitment of the auditory cortex; instead, it could be related to activation in higher-level, multimodal areas in the cortex.
Collapse
Affiliation(s)
| | | | - Łukasz Bola
- Intitute of Psychology, Jagiellonian University, Kraków, Poland
- Institute of Psychology, Polish Academy of Sciences, Warszawa, Poland
| | - Marcin Szwed
- Intitute of Psychology, Jagiellonian University, Kraków, Poland
- *Correspondence: Marcin Szwed,
| |
Collapse
|
7
|
Grey Matter Hypertrophy and Atrophy in Early-Blind Adolescents: A Surface-Based Morphometric Study. DISEASE MARKERS 2022; 2022:8550714. [PMID: 35557871 PMCID: PMC9090530 DOI: 10.1155/2022/8550714] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 08/05/2021] [Accepted: 04/09/2022] [Indexed: 11/25/2022]
Abstract
Objective This study is aimed at exploring the regional changes in brain cortical morphology (thickness, volume, and surface area) in the early-blind adolescents (EBAs) by using the surface-based morphometric (SBM) method. Methods High-resolution structural T1-weighted images (T1WI) of 23 early-blind adolescents (EBAs) and 21 age- and gender-matched normal-sighted controls (NSCs) were acquired. Structural indices, including cortical thickness (CT), cortical volume (CV), and surface area (SA), were analyzed by using FreeSurfer software, and the correlations between structural indices and the blindness duration were computed by Pearson correlation analysis. Results Compared to controls, EBAs had significantly reduced CV and SA mainly in the primary visual cortex (V1) and decreased CV in the left vision-related cortices (r-MFC). There were no regions that EBAs had a significantly larger CV or SA than NSCs. EBAs had significantly increased CT in the V1 and strongly involved the visual cortex (right lateral occipital gyrus, LOG.R) and the left superior temporal gyrus (STG.L), while it had decreased CT in the left superior parietal lobule (SPL.L) and the right lingual gyrus (LING.R). Additionally, no correlation was found between cortical morphometric measures and clinical variables in the EBA group. Conclusions SBM is a useful method for detecting human brain structural abnormalities in blindness. The results showed that these structural abnormalities in the visual cortex and visual-related areas outside the occipital cortex in the EBAs not only may be influenced by neurodevelopment, degeneration, plasticity, and so on but also involved the interaction of these factors after the early visual deprivation.
Collapse
|
8
|
Kim HS, Kim KB, Lee JH, Jung JJ, Kim YJ, Kim SP, Choi MH, Yi JH, Chung SC. Mid-Air Tactile Sensations Evoked by Laser-Induced Plasma: A Neurophysiological Study. Front Neurosci 2021; 15:733423. [PMID: 34658771 PMCID: PMC8517193 DOI: 10.3389/fnins.2021.733423] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2021] [Accepted: 09/06/2021] [Indexed: 11/22/2022] Open
Abstract
This study demonstrates the feasibility of a mid-air means of haptic stimulation at a long distance using the plasma effect induced by laser. We hypothesize that the stress wave generated by laser-induced plasma in the air can propagate through the air to reach the nearby human skin and evoke tactile sensation. To validate this hypothesis, we investigated somatosensory responses in the human brain to laser plasma stimuli by analyzing electroencephalography (EEG) in 14 participants. Three types of stimuli were provided to the index finger: a plasma stimulus induced from the laser, a mechanical stimulus transferred through Styrofoam stick, and a sham stimulus providing only the sound of the plasma and mechanical stimuli at the same time. The event-related desynchronization/synchronization (ERD/S) of sensorimotor rhythms (SMRs) in EEG was analyzed. Every participant verbally reported that they could feel a soft tap on the finger in response to the laser stimulus, but not to the sham stimulus. The spectrogram of EEG evoked by laser stimulation was similar to that evoked by mechanical stimulation; alpha ERD and beta ERS were present over the sensorimotor area in response to laser as well as mechanical stimuli. A decoding analysis revealed that classification error increased when discriminating ERD/S patterns between laser and mechanical stimuli, compared to the case of discriminating between laser and sham, or mechanical and sham stimuli. Our neurophysiological results confirm that tactile sensation can be evoked by the plasma effect induced by laser in the air, which may provide a mid-air haptic stimulation method.
Collapse
Affiliation(s)
- Hyung-Sik Kim
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| | - Kyu Beom Kim
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| | - Je-Hyeop Lee
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| | - Jin-Ju Jung
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| | - Ye-Jin Kim
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| | - Sung-Phil Kim
- Department of Biomedical Engineering, Ulsan National Institute of Science and Technology, Ulsan, South Korea
| | - Mi-Hyun Choi
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| | - Jeong-Han Yi
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| | - Soon-Cheol Chung
- Department of Biomedical Engineering, BK21 Plus Research Institute of Biomedical Engineering, School of ICT Convergence Engineering, College of Science and Technology, Konkuk University, Chungju-si, South Korea
| |
Collapse
|
9
|
Lowe MX, Mohsenzadeh Y, Lahner B, Charest I, Oliva A, Teng S. Cochlea to categories: The spatiotemporal dynamics of semantic auditory representations. Cogn Neuropsychol 2021; 38:468-489. [PMID: 35729704 PMCID: PMC10589059 DOI: 10.1080/02643294.2022.2085085] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/30/2021] [Revised: 03/31/2022] [Accepted: 05/25/2022] [Indexed: 10/17/2022]
Abstract
How does the auditory system categorize natural sounds? Here we apply multimodal neuroimaging to illustrate the progression from acoustic to semantically dominated representations. Combining magnetoencephalographic (MEG) and functional magnetic resonance imaging (fMRI) scans of observers listening to naturalistic sounds, we found superior temporal responses beginning ∼55 ms post-stimulus onset, spreading to extratemporal cortices by ∼100 ms. Early regions were distinguished less by onset/peak latency than by functional properties and overall temporal response profiles. Early acoustically-dominated representations trended systematically toward category dominance over time (after ∼200 ms) and space (beyond primary cortex). Semantic category representation was spatially specific: Vocalizations were preferentially distinguished in frontotemporal voice-selective regions and the fusiform; scenes and objects were distinguished in parahippocampal and medial place areas. Our results are consistent with real-world events coded via an extended auditory processing hierarchy, in which acoustic representations rapidly enter multiple streams specialized by category, including areas typically considered visual cortex.
Collapse
Affiliation(s)
- Matthew X. Lowe
- Computer Science and Artificial Intelligence Lab (CSAIL), MIT, Cambridge, MA
- Unlimited Sciences, Colorado Springs, CO
| | - Yalda Mohsenzadeh
- Computer Science and Artificial Intelligence Lab (CSAIL), MIT, Cambridge, MA
- The Brain and Mind Institute, The University of Western Ontario, London, ON, Canada
- Department of Computer Science, The University of Western Ontario, London, ON, Canada
| | - Benjamin Lahner
- Computer Science and Artificial Intelligence Lab (CSAIL), MIT, Cambridge, MA
| | - Ian Charest
- Département de Psychologie, Université de Montréal, Montréal, Québec, Canada
- Center for Human Brain Health, University of Birmingham, UK
| | - Aude Oliva
- Computer Science and Artificial Intelligence Lab (CSAIL), MIT, Cambridge, MA
| | - Santani Teng
- Computer Science and Artificial Intelligence Lab (CSAIL), MIT, Cambridge, MA
- Smith-Kettlewell Eye Research Institute (SKERI), San Francisco, CA
| |
Collapse
|
10
|
Pesnot Lerousseau J, Arnold G, Auvray M. Training-induced plasticity enables visualizing sounds with a visual-to-auditory conversion device. Sci Rep 2021; 11:14762. [PMID: 34285265 PMCID: PMC8292401 DOI: 10.1038/s41598-021-94133-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2021] [Accepted: 06/28/2021] [Indexed: 12/04/2022] Open
Abstract
Sensory substitution devices aim at restoring visual functions by converting visual information into auditory or tactile stimuli. Although these devices show promise in the range of behavioral abilities they allow, the processes underlying their use remain underspecified. In particular, while an initial debate focused on the visual versus auditory or tactile nature of sensory substitution, since over a decade, the idea that it reflects a mixture of both has emerged. In order to investigate behaviorally the extent to which visual and auditory processes are involved, participants completed a Stroop-like crossmodal interference paradigm before and after being trained with a conversion device which translates visual images into sounds. In addition, participants' auditory abilities and their phenomenologies were measured. Our study revealed that, after training, when asked to identify sounds, processes shared with vision were involved, as participants’ performance in sound identification was influenced by the simultaneously presented visual distractors. In addition, participants’ performance during training and their associated phenomenology depended on their auditory abilities, revealing that processing finds its roots in the input sensory modality. Our results pave the way for improving the design and learning of these devices by taking into account inter-individual differences in auditory and visual perceptual strategies.
Collapse
Affiliation(s)
| | | | - Malika Auvray
- Sorbonne Université, CNRS UMR 7222, Institut des Systèmes Intelligents et de Robotique (ISIR), 75005, Paris, France.
| |
Collapse
|
11
|
Graven T, Desebrock C. Touching and hearing the shapes: How auditory angular and curved sounds influence proficiency in recognising tactile angle and curve shapes when experienced and inexperienced in using haptic touch. BRITISH JOURNAL OF VISUAL IMPAIRMENT 2021. [DOI: 10.1177/02646196211003114] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
This study investigated whether adding auditory angular and curved sounds to tactile angle and curve shapes – one unspecified sound to one unspecified shape – positively influences the accuracy and exploration time in recognising tactile angles and curves when experienced and inexperienced in using haptic touch. A within-participant experiment was conducted, with two groups of participants: experienced and inexperienced in using haptic touch, and with two conditions: congruous (e.g., angle shape and angular sound) and incongruous (e.g., angle shape and curved sound) tactile and auditory shape information. Adding congruous auditory angular and curved sounds to tactile angle and curve shapes positively influences the accuracy in recognising tactile angles and curves both when experienced and inexperienced in using haptic touch, and the exploration time on correct recognitions when experienced. People integrate tactile and auditory (angle; curve) shape information and this improves their proficiency in recognising tactile angles and curves.
Collapse
|
12
|
Heimler B, Amedi A. Are critical periods reversible in the adult brain? Insights on cortical specializations based on sensory deprivation studies. Neurosci Biobehav Rev 2020; 116:494-507. [DOI: 10.1016/j.neubiorev.2020.06.034] [Citation(s) in RCA: 17] [Impact Index Per Article: 4.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2019] [Revised: 06/07/2020] [Accepted: 06/25/2020] [Indexed: 02/06/2023]
|
13
|
Jicol C, Lloyd-Esenkaya T, Proulx MJ, Lange-Smith S, Scheller M, O'Neill E, Petrini K. Efficiency of Sensory Substitution Devices Alone and in Combination With Self-Motion for Spatial Navigation in Sighted and Visually Impaired. Front Psychol 2020; 11:1443. [PMID: 32754082 PMCID: PMC7381305 DOI: 10.3389/fpsyg.2020.01443] [Citation(s) in RCA: 18] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/13/2020] [Accepted: 05/29/2020] [Indexed: 11/13/2022] Open
Abstract
Human adults can optimally combine vision with self-motion to facilitate navigation. In the absence of visual input (e.g., dark environments and visual impairments), sensory substitution devices (SSDs), such as The vOICe or BrainPort, which translate visual information into auditory or tactile information, could be used to increase navigation precision when integrated together or with self-motion. In Experiment 1, we compared and assessed together The vOICe and BrainPort in aerial maps task performed by a group of sighted participants. In Experiment 2, we examined whether sighted individuals and a group of visually impaired (VI) individuals could benefit from using The vOICe, with and without self-motion, to accurately navigate a three-dimensional (3D) environment. In both studies, 3D motion tracking data were used to determine the level of precision with which participants performed two different tasks (an egocentric and an allocentric task) and three different conditions (two unisensory conditions and one multisensory condition). In Experiment 1, we found no benefit of using the devices together. In Experiment 2, the sighted performance during The vOICe was almost as good as that for self-motion despite a short training period, although we found no benefit (reduction in variability) of using The vOICe and self-motion in combination compared to the two in isolation. In contrast, the group of VI participants did benefit from combining The vOICe and self-motion despite the low number of trials. Finally, while both groups became more accurate in their use of The vOICe with increased trials, only the VI group showed an increased level of accuracy in the combined condition. Our findings highlight how exploiting non-visual multisensory integration to develop new assistive technologies could be key to help blind and VI persons, especially due to their difficulty in attaining allocentric information.
Collapse
Affiliation(s)
- Crescent Jicol
- Department of Psychology, University of Bath, Bath, United Kingdom
| | | | - Michael J Proulx
- Department of Psychology, University of Bath, Bath, United Kingdom
| | - Simon Lange-Smith
- School of Sport and Exercise Sciences, Liverpool John Moores University, Liverpool, United Kingdom
| | - Meike Scheller
- Department of Psychology, University of Bath, Bath, United Kingdom
| | - Eamonn O'Neill
- Department of Computer Science, University of Bath, Bath, United Kingdom
| | - Karin Petrini
- Department of Psychology, University of Bath, Bath, United Kingdom
| |
Collapse
|
14
|
Zhou W, Pang W, Zhang L, Xu H, Li P, Shu H. Altered connectivity of the visual word form area in the low-vision population: A resting-state fMRI study. Neuropsychologia 2020; 137:107302. [DOI: 10.1016/j.neuropsychologia.2019.107302] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2018] [Revised: 08/22/2019] [Accepted: 12/06/2019] [Indexed: 01/26/2023]
|
15
|
Auvray M. Multisensory and spatial processes in sensory substitution. Restor Neurol Neurosci 2019; 37:609-619. [DOI: 10.3233/rnn-190950] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Affiliation(s)
- Malika Auvray
- Institut des Systèmes Intelligents et de Robotique, CNRS UMR 7222, Sorbonne Université, Paris, France
| |
Collapse
|
16
|
Zhang C, Lee TMC, Fu Y, Ren C, Chan CCH, Tao Q. Properties of cross-modal occipital responses in early blindness: An ALE meta-analysis. NEUROIMAGE-CLINICAL 2019; 24:102041. [PMID: 31677587 PMCID: PMC6838549 DOI: 10.1016/j.nicl.2019.102041] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/12/2019] [Revised: 09/20/2019] [Accepted: 10/17/2019] [Indexed: 11/10/2022]
Abstract
ALE meta-analysis reveals distributed brain networks for object and spatial functions in individuals with early blindness. ALE contrast analysis reveals specific activations in the left cuneus and lingual gyrus for language function, suggesting a reverse hierarchical organization of the visual cortex for early blind individuals. The findings contribute to visual rehabilitation in blind individuals by revealing the function-dependent and sensory-independent networks during nonvisual processing.
Cross-modal occipital responses appear to be essential for nonvisual processing in individuals with early blindness. However, it is not clear whether the recruitment of occipital regions depends on functional domain or sensory modality. The current study utilized a coordinate-based meta-analysis to identify the distinct brain regions involved in the functional domains of object, spatial/motion, and language processing and the common brain regions involved in both auditory and tactile modalities in individuals with early blindness. Following the PRISMA guidelines, a total of 55 studies were included in the meta-analysis. The specific analyses revealed the brain regions that are consistently recruited for each function, such as the dorsal fronto-parietal network for spatial function and ventral occipito-temporal network for object function. This is consistent with the literature, suggesting that the two visual streams are preserved in early blind individuals. The contrast analyses found specific activations in the left cuneus and lingual gyrus for language function. This finding is novel and suggests a reverse hierarchical organization of the visual cortex for early blind individuals. The conjunction analyses found common activations in the right middle temporal gyrus, right precuneus and a left parieto-occipital region. Clinically, this work contributes to visual rehabilitation in early blind individuals by revealing the function-dependent and sensory-independent networks during nonvisual processing.
Collapse
Affiliation(s)
- Caiyun Zhang
- Psychology Department, School of Medicine, Jinan University, Guangzhou 510632, China
| | - Tatia M C Lee
- Laboratory of Neuropsychology, The University of Hong Kong, Hong Kong, CHINA; Laboratory of Cognitive Affective Neuroscience, The University of Hong Kong, Hong Kong, CHINA; The Affiliated Brain Hospital of Guangzhou Medical University, Guangzhou, China
| | - Yunwei Fu
- Guangdong-Hongkong-Macau Institute of CNS Regeneration, Ministry of Education CNS Regeneration Collaborative Joint Laboratory, Jinan University, Guangzhou, 510632, China
| | - Chaoran Ren
- Guangdong-Hongkong-Macau Institute of CNS Regeneration, Ministry of Education CNS Regeneration Collaborative Joint Laboratory, Jinan University, Guangzhou, 510632, China; Guangdong key Laboratory of Brain Function and Diseases, Jinan University, Guangzhou, 510632, China; Co-innovation Center of Neuroregeneration, Nantong University, Nantong, 226001, China; Center for Brain Science and Brain-Inspired Intelligence, Guangdong-Hong Kong-Macao Greater Bay Area, Guangzhou, China
| | - Chetwyn C H Chan
- Applied Cognitive Neuroscience Laboratory, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, CHINA.
| | - Qian Tao
- Psychology Department, School of Medicine, Jinan University, Guangzhou 510632, China; Center for Brain Science and Brain-Inspired Intelligence, Guangdong-Hong Kong-Macao Greater Bay Area, Guangzhou, China.
| |
Collapse
|
17
|
Wolak T, Cieśla K, Pluta A, Włodarczyk E, Biswal B, Skarżyński H. Altered Functional Connectivity in Patients With Sloping Sensorineural Hearing Loss. Front Hum Neurosci 2019; 13:284. [PMID: 31507391 PMCID: PMC6713935 DOI: 10.3389/fnhum.2019.00284] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2019] [Accepted: 08/02/2019] [Indexed: 12/12/2022] Open
Abstract
Background Sensory deprivation, such as hearing loss, has been demonstrated to change the intrinsic functional connectivity (FC) of the brain, as measured with resting-state functional magnetic resonance imaging (rs-fMRI). Patients with sloping sensorineural hearing loss (SNHL) are a unique population among the hearing impaired, as they have all been exposed to some auditory input throughout their lifespan and all use spoken language. Materials and Methods Twenty patients with SNHL and 21 control subjects participated in a rs-fMRI study. Whole-brain seed-driven FC maps were obtained, with audiological scores of patients, including hearing loss severity and speech performance, used as covariates. Results Most profound differences in FC were found between patients with prelingual (before language development, PRE) vs. postlingual onset (after language development, POST) of SNHL. An early onset was related to enhancement in long-range network connections, including the default-mode network, the dorsal-attention network and the fronto-parietal network, as well as in local sensory networks, the visual and the sensorimotor. A number of multisensory brain regions in frontal and parietal cortices, as well as the cerebellum, were also more internally connected. We interpret these effects as top-down mechanisms serving optimization of multisensory experience in SNHL with a prelingual onset. At the same time, POST patients showed enhanced FC between the salience network and multisensory parietal areas, as well as with the hippocampus, when they were compared to those with PRE hearing loss. Signal in several cortex regions subserving visual processing was also more intra-correlated in POST vs. PRE patients. This outcome might point to more attention resources directed to multisensory as well as memory experience. Finally, audiological scores correlated with FC in several sensory and high-order brain regions in all patients. Conclusion The results show that a sloping hearing loss is related to altered resting-state brain organization. Effects were shown in attention and cognitive control networks, as well as visual and sensorimotor regions. Specifically, we found that even in a partial hearing deficit (affecting only some of the hearing frequency ranges), the age at the onset affects the brain function differently, pointing to the role of sensitive periods in brain development.
Collapse
Affiliation(s)
- Tomasz Wolak
- Institute of Physiology and Pathology of Hearing, Bioimaging Research Center, World Hearing Center, Warsaw, Poland
| | - Katarzyna Cieśla
- Institute of Physiology and Pathology of Hearing, Bioimaging Research Center, World Hearing Center, Warsaw, Poland
| | - Agnieszka Pluta
- Institute of Physiology and Pathology of Hearing, Bioimaging Research Center, World Hearing Center, Warsaw, Poland.,Faculty of Psychology, University of Warsaw, Warsaw, Poland
| | - Elżbieta Włodarczyk
- Institute of Physiology and Pathology of Hearing, Bioimaging Research Center, World Hearing Center, Warsaw, Poland
| | - Bharat Biswal
- Department of Biomedical Engineering and Department of Radiology, New Jersey Medical School, NJIT, Newark, NJ, United States
| | - Henryk Skarżyński
- Institute of Physiology and Pathology of Hearing, Bioimaging Research Center, World Hearing Center, Warsaw, Poland
| |
Collapse
|
18
|
Bola Ł, Matuszewski J, Szczepanik M, Droździel D, Sliwinska MW, Paplińska M, Jednoróg K, Szwed M, Marchewka A. Functional hierarchy for tactile processing in the visual cortex of sighted adults. Neuroimage 2019; 202:116084. [PMID: 31400530 DOI: 10.1016/j.neuroimage.2019.116084] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/28/2019] [Revised: 07/07/2019] [Accepted: 08/06/2019] [Indexed: 12/15/2022] Open
Abstract
Perception via different sensory modalities was traditionally believed to be supported by largely separate brain systems. However, a growing number of studies demonstrate that the visual cortices of typical, sighted adults are involved in tactile and auditory perceptual processing. Here, we investigated the spatiotemporal dynamics of the visual cortex's involvement in a complex tactile task: Braille letter recognition. Sighted subjects underwent Braille training and then participated in a transcranial magnetic stimulation (TMS) study in which they tactually identified single Braille letters. During this task, TMS was applied to their left early visual cortex, visual word form area (VWFA), and left early somatosensory cortex at five time windows from 20 to 520 ms following the Braille letter presentation's onset. The subjects' response accuracy decreased when TMS was applied to the early visual cortex at the 120-220 ms time window and when TMS was applied to the VWFA at the 320-420 ms time window. Stimulation of the early somatosensory cortex did not have a time-specific effect on the accuracy of the subjects' Braille letter recognition, but rather caused a general slowdown during this task. Our results indicate that the involvement of sighted people's visual cortices in tactile perception respects the canonical visual hierarchy-the early tactile processing stages involve the early visual cortex, whereas more advanced tactile computations involve high-level visual areas. Our findings are compatible with the metamodal account of brain organization and suggest that the whole visual cortex may potentially support spatial perception in a task-specific, sensory-independent manner.
Collapse
Affiliation(s)
- Łukasz Bola
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology, Polish Academy of Sciences, 3 Pasteura Street, 02-093, Warsaw, Poland; Institute of Psychology, Jagiellonian University, 6 Ingardena Street, 30-060, Krakow, Poland.
| | - Jacek Matuszewski
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology, Polish Academy of Sciences, 3 Pasteura Street, 02-093, Warsaw, Poland
| | - Michał Szczepanik
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology, Polish Academy of Sciences, 3 Pasteura Street, 02-093, Warsaw, Poland
| | - Dawid Droździel
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology, Polish Academy of Sciences, 3 Pasteura Street, 02-093, Warsaw, Poland
| | | | - Małgorzata Paplińska
- The Maria Grzegorzewska University, 40 Szczęśliwicka Street, 02-353, Warsaw, Poland
| | - Katarzyna Jednoróg
- Laboratory of Language Neurobiology, Nencki Institute of Experimental Biology, Polish Academy of Sciences, 3 Pasteura Street, 02-093, Warsaw, Poland
| | - Marcin Szwed
- Institute of Psychology, Jagiellonian University, 6 Ingardena Street, 30-060, Krakow, Poland.
| | - Artur Marchewka
- Laboratory of Brain Imaging, Nencki Institute of Experimental Biology, Polish Academy of Sciences, 3 Pasteura Street, 02-093, Warsaw, Poland.
| |
Collapse
|
19
|
Jan JE, Heaven RKB, Matsuba C, Langley MB, Roman-Lantzy C, Anthony TL. Windows into the Visual Brain: New Discoveries about the Visual System, Its Functions, and Implications for Practitioners. JOURNAL OF VISUAL IMPAIRMENT & BLINDNESS 2019. [DOI: 10.1177/0145482x1310700402] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
Introduction In recent years, major progress has been made in understanding the human visual system because of new investigative techniques. These developments often contradict older concepts about visual function. Methods A detailed literature search and interprofessional discussions. Results Recent innovative neurological tests are described that are able to show much more accurately the visual pathways, the process of vision, and the close relationships among sensory modalities. These tests also reveal the remarkable neuroplasticity of the human brain and disorders of connectivity that frequently involve visual function. Discussion How these recent neurological advances may benefit service providers is discussed. Implications for practitioners It is important that from time to time new neurological and ophthalmic developments are summarized for professionals who are involved in the clinical management of individuals with visual disorders and how the newly acquired knowledge affects the diagnosis and intervention strategies. Visual rehabilitation must be based on up-to-date science, which continually changes and grows with research.
Collapse
Affiliation(s)
- James E. Jan
- Pediatric Neurology, Department of Neurophysiology, University of British Columbia, BC Children Hospital, 4480 Oak Street, Vancouver, BC, V6H3V4, Canada
| | - Roberta K. B. Heaven
- Department of Psychiatry, University of British Columbia, and team leader, Visual Impairment Program, BC Children's Hospital and Sunny Hill Hospital for Children, 3644 Slocan Street, Vancouver, BC, V5M, 3E8, Canada
| | - Carey Matsuba
- Visual Impairment Program, Department of Pediatrics, University of British Columbia, and pediatric consultant, Visual Impairment Program, BC Children's Hospital, 4480 Oak Street, Vancouver, BC, V6H 3V4, Canada
| | - M. Beth Langley
- Pre-kindergarten Assessment Team, Pinellas County Schools, 301 Fourth Street SW, Largo, FL 33770
| | - Christine Roman-Lantzy
- Western Pennsylvania Hospital, 4800 Friendship Avenue, Pittsburgh, PA 15224; CVI project leader, American Printing House for the Blind; and special assistant to the superintendent, Western Pennsylvania School for the Blind
| | - Tanni L. Anthony
- Exceptional Student Services Unit, Colorado Department of Education, 1560 Broadway, Suite 1175, Denver, CO 80202
| |
Collapse
|
20
|
Pundik S, Scoco A, Skelly M, McCabe JP, Daly JJ. Greater Cortical Thickness Is Associated With Enhanced Sensory Function After Arm Rehabilitation in Chronic Stroke. Neurorehabil Neural Repair 2018; 32:590-601. [DOI: 10.1177/1545968318778810] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Objective. Somatosensory function is critical to normal motor control. After stroke, dysfunction of the sensory systems prevents normal motor function and degrades quality of life. Structural neuroplasticity underpinnings of sensory recovery after stroke are not fully understood. The objective of this study was to identify changes in bilateral cortical thickness (CT) that may drive recovery of sensory acuity. Methods. Chronic stroke survivors (n = 20) were treated with 12 weeks of rehabilitation. Measures were sensory acuity (monofilament), Fugl-Meyer upper limb and CT change. Permutation-based general linear regression modeling identified cortical regions in which change in CT was associated with change in sensory acuity. Results. For the ipsilesional hemisphere in response to treatment, CT increase was significantly associated with sensory improvement in the area encompassing the occipital pole, lateral occipital cortex (inferior and superior divisions), intracalcarine cortex, cuneal cortex, precuneus cortex, inferior temporal gyrus, occipital fusiform gyrus, supracalcarine cortex, and temporal occipital fusiform cortex. For the contralesional hemisphere, increased CT was associated with improved sensory acuity within the posterior parietal cortex that included supramarginal and angular gyri. Following upper limb therapy, monofilament test score changed from 45.0 ± 13.3 to 42.6 ± 12.9 mm ( P = .063) and Fugl-Meyer score changed from 22.1 ± 7.8 to 32.3 ± 10.1 ( P < .001). Conclusions. Rehabilitation in the chronic stage after stroke produced structural brain changes that were strongly associated with enhanced sensory acuity. Improved sensory perception was associated with increased CT in bilateral high-order association sensory cortices reflecting the complex nature of sensory function and recovery in response to rehabilitation.
Collapse
Affiliation(s)
- Svetlana Pundik
- Case Western Reserve University, Cleveland, OH, USA
- Cleveland VA Medical Center, Cleveland, OH, USA
| | - Aleka Scoco
- Case Western Reserve University, Cleveland, OH, USA
| | | | | | - Janis J. Daly
- University of Florida, Gainesville, FL, USA
- Gainesville VA Medical Center, Gainesville, FL, USA
| |
Collapse
|
21
|
Graulty C, Papaioannou O, Bauer P, Pitts MA, Canseco-Gonzalez E. Hearing Shapes: Event-related Potentials Reveal the Time Course of Auditory-Visual Sensory Substitution. J Cogn Neurosci 2017; 30:498-513. [PMID: 29211649 DOI: 10.1162/jocn_a_01210] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
In auditory-visual sensory substitution, visual information (e.g., shape) can be extracted through strictly auditory input (e.g., soundscapes). Previous studies have shown that image-to-sound conversions that follow simple rules [such as the Meijer algorithm; Meijer, P. B. L. An experimental system for auditory image representation. Transactions on Biomedical Engineering, 39, 111-121, 1992] are highly intuitive and rapidly learned by both blind and sighted individuals. A number of recent fMRI studies have begun to explore the neuroplastic changes that result from sensory substitution training. However, the time course of cross-sensory information transfer in sensory substitution is largely unexplored and may offer insights into the underlying neural mechanisms. In this study, we recorded ERPs to soundscapes before and after sighted participants were trained with the Meijer algorithm. We compared these posttraining versus pretraining ERP differences with those of a control group who received the same set of 80 auditory/visual stimuli but with arbitrary pairings during training. Our behavioral results confirmed the rapid acquisition of cross-sensory mappings, and the group trained with the Meijer algorithm was able to generalize their learning to novel soundscapes at impressive levels of accuracy. The ERP results revealed an early cross-sensory learning effect (150-210 msec) that was significantly enhanced in the algorithm-trained group compared with the control group as well as a later difference (420-480 msec) that was unique to the algorithm-trained group. These ERP modulations are consistent with previous fMRI results and provide additional insight into the time course of cross-sensory information transfer in sensory substitution.
Collapse
|
22
|
Whether the hearing brain hears it or the deaf brain sees it, it’s just the same. Proc Natl Acad Sci U S A 2017; 114:8135-8137. [DOI: 10.1073/pnas.1710492114] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
|
23
|
Hernández-Pérez R, Cuaya LV, Rojas-Hortelano E, Reyes-Aguilar A, Concha L, de Lafuente V. Tactile object categories can be decoded from the parietal and lateral-occipital cortices. Neuroscience 2017; 352:226-235. [DOI: 10.1016/j.neuroscience.2017.03.038] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/10/2016] [Revised: 03/23/2017] [Accepted: 03/24/2017] [Indexed: 01/08/2023]
|
24
|
Tao Q, Chan CCH, Luo YJ, Li JJ, Ting KH, Lu ZL, Whitfield-Gabrieli S, Wang J, Lee TMC. Prior Visual Experience Modulates Learning of Sound Localization Among Blind Individuals. Brain Topogr 2017; 30:364-379. [PMID: 28161728 PMCID: PMC5408050 DOI: 10.1007/s10548-017-0549-z] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/04/2016] [Accepted: 01/19/2017] [Indexed: 11/26/2022]
Abstract
Cross-modal learning requires the use of information from different sensory modalities. This study investigated how the prior visual experience of late blind individuals could modulate neural processes associated with learning of sound localization. Learning was realized by standardized training on sound localization processing, and experience was investigated by comparing brain activations elicited from a sound localization task in individuals with (late blind, LB) and without (early blind, EB) prior visual experience. After the training, EB showed decreased activation in the precuneus, which was functionally connected to a limbic-multisensory network. In contrast, LB showed the increased activation of the precuneus. A subgroup of LB participants who demonstrated higher visuospatial working memory capabilities (LB-HVM) exhibited an enhanced precuneus-lingual gyrus network. This differential connectivity suggests that visuospatial working memory due to the prior visual experience gained via LB-HVM enhanced learning of sound localization. Active visuospatial navigation processes could have occurred in LB-HVM compared to the retrieval of previously bound information from long-term memory for EB. The precuneus appears to play a crucial role in learning of sound localization, disregarding prior visual experience. Prior visual experience, however, could enhance cross-modal learning by extending binding to the integration of unprocessed information, mediated by the cognitive functions that these experiences develop.
Collapse
Affiliation(s)
- Qian Tao
- Psychology Department, School of Medicine, Jinan University, Guangzhou, China
- Applied Cognitive Neuroscience Laboratory, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Chetwyn C H Chan
- Applied Cognitive Neuroscience Laboratory, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong.
| | - Yue-Jia Luo
- National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Jian-Jun Li
- China Rehabilitation Research Center, Beijing, China
| | - Kin-Hung Ting
- Applied Cognitive Neuroscience Laboratory, Department of Rehabilitation Sciences, The Hong Kong Polytechnic University, Hong Kong, Hong Kong
| | - Zhong-Lin Lu
- Center for Cognitive and Behavioral Brain Imaging, Arts, & Sciences, Department of Psychology, The Ohio State University, Ohio, OH, 43210, USA
| | | | - Jun Wang
- National Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, China
| | - Tatia M C Lee
- Laboratory of Neuropsychology, Department of Psychology, The University of Hong Kong, Hong Kong, Hong Kong.
- Laboratory of Cognitive Affective Neuroscience, The University of Hong Kong, Hong Kong, Hong Kong.
- State Key Laboratory of Brain and Cognitive Science, The University of Hong Kong, Hong Kong, Hong Kong.
| |
Collapse
|
25
|
Abstract
The principles that guide large-scale cortical reorganization remain unclear. In the blind, several visual regions preserve their task specificity; ventral visual areas, for example, become engaged in auditory and tactile object-recognition tasks. It remains open whether task-specific reorganization is unique to the visual cortex or, alternatively, whether this kind of plasticity is a general principle applying to other cortical areas. Auditory areas can become recruited for visual and tactile input in the deaf. Although nonhuman data suggest that this reorganization might be task specific, human evidence has been lacking. Here we enrolled 15 deaf and 15 hearing adults into an functional MRI experiment during which they discriminated between temporally complex sequences of stimuli (rhythms). Both deaf and hearing subjects performed the task visually, in the central visual field. In addition, hearing subjects performed the same task in the auditory modality. We found that the visual task robustly activated the auditory cortex in deaf subjects, peaking in the posterior-lateral part of high-level auditory areas. This activation pattern was strikingly similar to the pattern found in hearing subjects performing the auditory version of the task. Although performing the visual task in deaf subjects induced an increase in functional connectivity between the auditory cortex and the dorsal visual cortex, no such effect was found in hearing subjects. We conclude that in deaf humans the high-level auditory cortex switches its input modality from sound to vision but preserves its task-specific activation pattern independent of input modality. Task-specific reorganization thus might be a general principle that guides cortical plasticity in the brain.
Collapse
|
26
|
Debowska W, Wolak T, Nowicka A, Kozak A, Szwed M, Kossut M. Functional and Structural Neuroplasticity Induced by Short-Term Tactile Training Based on Braille Reading. Front Neurosci 2016; 10:460. [PMID: 27790087 PMCID: PMC5061995 DOI: 10.3389/fnins.2016.00460] [Citation(s) in RCA: 29] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2016] [Accepted: 09/26/2016] [Indexed: 11/30/2022] Open
Abstract
Neuroplastic changes induced by sensory learning have been recognized within the cortices of specific modalities as well as within higher ordered multimodal areas. The interplay between these areas is not fully understood, particularly in the case of somatosensory learning. Here we examined functional and structural changes induced by short-term tactile training based of Braille reading, a task that requires both significant tactile expertise and mapping of tactile input onto multimodal representations. Subjects with normal vision were trained for 3 weeks to read Braille exclusively by touch and scanned before and after training, while performing a same-different discrimination task on Braille characters and meaningless characters. Functional and diffusion-weighted magnetic resonance imaging sequences were used to assess resulting changes. The strongest training-induced effect was found in the primary somatosensory cortex (SI), where we observed bilateral augmentation in activity accompanied by an increase in fractional anisotropy (FA) within the contralateral SI. Increases of white matter fractional anisotropy were also observed in the secondary somatosensory area (SII) and the thalamus. Outside of somatosensory system, changes in both structure and function were found in i.e., the fusiform gyrus, the medial frontal gyri and the inferior parietal lobule. Our results provide evidence for functional remodeling of the somatosensory pathway and higher ordered multimodal brain areas occurring as a result of short-lasting tactile learning, and add to them a novel picture of extensive white matter plasticity.
Collapse
Affiliation(s)
- Weronika Debowska
- Laboratory of Neuroplasticity, Nencki Institute of Experimental Biology, Polish Academy of SciencesWarsaw, Poland; CNS Lab, Nalecz Institute of Biocybernetics and Biomedical Engineering, Polish Academy of SciencesWarsaw, Poland
| | - Tomasz Wolak
- Bioimaging Research Center, World Hearing Center, The Institute of Physiology and Pathology of Hearing Warsaw, Poland
| | - Anna Nowicka
- Laboratory of Psychophysiology, Nencki Institute of Experimental Biology Warsaw, Poland
| | - Anna Kozak
- Department of Psychology, University of Social Sciences and Humanities Warsaw, Poland
| | - Marcin Szwed
- Department of Psychology, Jagiellonian University Cracow, Poland
| | - Malgorzata Kossut
- Laboratory of Neuroplasticity, Nencki Institute of Experimental Biology, Polish Academy of SciencesWarsaw, Poland; Department of Psychology, University of Social Sciences and HumanitiesWarsaw, Poland
| |
Collapse
|
27
|
Proulx MJ, Gwinnutt J, Dell'Erba S, Levy-Tzedek S, de Sousa AA, Brown DJ. Other ways of seeing: From behavior to neural mechanisms in the online "visual" control of action with sensory substitution. Restor Neurol Neurosci 2016; 34:29-44. [PMID: 26599473 PMCID: PMC4927905 DOI: 10.3233/rnn-150541] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Vision is the dominant sense for perception-for-action in humans and other higher primates. Advances in sight restoration now utilize the other intact senses to provide information that is normally sensed visually through sensory substitution to replace missing visual information. Sensory substitution devices translate visual information from a sensor, such as a camera or ultrasound device, into a format that the auditory or tactile systems can detect and process, so the visually impaired can see through hearing or touch. Online control of action is essential for many daily tasks such as pointing, grasping and navigating, and adapting to a sensory substitution device successfully requires extensive learning. Here we review the research on sensory substitution for vision restoration in the context of providing the means of online control for action in the blind or blindfolded. It appears that the use of sensory substitution devices utilizes the neural visual system; this suggests the hypothesis that sensory substitution draws on the same underlying mechanisms as unimpaired visual control of action. Here we review the current state of the art for sensory substitution approaches to object recognition, localization, and navigation, and the potential these approaches have for revealing a metamodal behavioral and neural basis for the online control of action.
Collapse
Affiliation(s)
- Michael J Proulx
- Crossmodal Cognition Lab, Department of Psychology, University of Bath, Bath, UK
| | - James Gwinnutt
- Crossmodal Cognition Lab, Department of Psychology, University of Bath, Bath, UK
| | - Sara Dell'Erba
- Crossmodal Cognition Lab, Department of Psychology, University of Bath, Bath, UK
| | - Shelly Levy-Tzedek
- Cognition, Aging and Rehabilitation Lab, Recanati School for Community Health Professions, Department of Physical Therapy & Zlotowski Center for Neuroscience, Ben Gurion University of the Negev, Beer-Sheva, Israel
| | - Alexandra A de Sousa
- Crossmodal Cognition Lab, Department of Psychology, University of Bath, Bath, UK.,Department of Science, Bath Spa University, Bath, UK
| | - David J Brown
- Crossmodal Cognition Lab, Department of Psychology, University of Bath, Bath, UK
| |
Collapse
|
28
|
Bola Ł, Siuda-Krzywicka K, Paplińska M, Sumera E, Hańczur P, Szwed M. Braille in the Sighted: Teaching Tactile Reading to Sighted Adults. PLoS One 2016; 11:e0155394. [PMID: 27187496 PMCID: PMC4871356 DOI: 10.1371/journal.pone.0155394] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/11/2015] [Accepted: 04/28/2016] [Indexed: 11/24/2022] Open
Abstract
Blind people are known to have superior perceptual abilities in their remaining senses. Several studies suggest that these enhancements are dependent on the specific experience of blind individuals, who use those remaining senses more than sighted subjects. In line with this view, sighted subjects, when trained, are able to significantly progress in relatively simple tactile tasks. However, the case of complex tactile tasks is less obvious, as some studies suggest that visual deprivation itself could confer large advantages in learning them. It remains unclear to what extent those complex skills, such as braille reading, can be learnt by sighted subjects. Here we enrolled twenty-nine sighted adults, mostly braille teachers and educators, in a 9-month braille reading course. At the beginning of the course, all subjects were naive in tactile braille reading. After the course, almost all were able to read whole braille words at a mean speed of 6 words-per-minute. Subjects with low tactile acuity did not differ significantly in braille reading speed from the rest of the group, indicating that low tactile acuity is not a limiting factor for learning braille, at least at this early stage of learning. Our study shows that most sighted adults can learn whole-word braille reading, given the right method and a considerable amount of motivation. The adult sensorimotor system can thus adapt, to some level, to very complex tactile tasks without visual deprivation. The pace of learning in our group was comparable to congenitally and early blind children learning braille in primary school, which suggests that the blind’s mastery of complex tactile tasks can, to a large extent, be explained by experience-dependent mechanisms.
Collapse
Affiliation(s)
- Łukasz Bola
- Department of Psychology, Jagiellonian University, Krakow, Poland
- Laboratory of Brain Imaging, Neurobiology Center, Nencki Institute of Experimental Biology, Warsaw, Poland
| | - Katarzyna Siuda-Krzywicka
- Department of Psychology, Jagiellonian University, Krakow, Poland
- INSERM U 1127, CNRS UMR 7225, Sorbonne Universités, and Université Pierre et Marie Curie-Paris 6, UMR S 1127, Institut du Cerveau et de la Moelle épinière (ICM), Paris, France
| | | | - Ewa Sumera
- Institute for the Blind and Partially Sighted Children in Krakow, Krakow, Poland
| | - Paweł Hańczur
- Electrical Drive Division, Institute of Control and Industrial Electronics, Warsaw University of Technology, Warsaw, Poland
| | - Marcin Szwed
- Department of Psychology, Jagiellonian University, Krakow, Poland
- * E-mail:
| |
Collapse
|
29
|
Handjaras G, Ricciardi E, Leo A, Lenci A, Cecchetti L, Cosottini M, Marotta G, Pietrini P. How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge. Neuroimage 2016; 135:232-42. [PMID: 27132545 DOI: 10.1016/j.neuroimage.2016.04.063] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2015] [Revised: 02/12/2016] [Accepted: 04/26/2016] [Indexed: 11/25/2022] Open
Abstract
How conceptual knowledge is represented in the human brain remains to be determined. To address the differential role of low-level sensory-based and high-level abstract features in semantic processing, we combined behavioral studies of linguistic production and brain activity measures by functional magnetic resonance imaging in sighted and congenitally blind individuals while they performed a property-generation task with concrete nouns from eight categories, presented through visual and/or auditory modalities. Patterns of neural activity within a large semantic cortical network that comprised parahippocampal, lateral occipital, temporo-parieto-occipital and inferior parietal cortices correlated with linguistic production and were independent both from the modality of stimulus presentation (either visual or auditory) and the (lack of) visual experience. In contrast, selected modality-dependent differences were observed only when the analysis was limited to the individual regions within the semantic cortical network. We conclude that conceptual knowledge in the human brain relies on a distributed, modality-independent cortical representation that integrates the partial category and modality specific information retained at a regional level.
Collapse
Affiliation(s)
- Giacomo Handjaras
- Dept. Surgical, Medical, Molecular Pathology and Critical Care, University of Pisa, Pisa 56126, Italy
| | - Emiliano Ricciardi
- Dept. Surgical, Medical, Molecular Pathology and Critical Care, University of Pisa, Pisa 56126, Italy
| | - Andrea Leo
- Dept. Surgical, Medical, Molecular Pathology and Critical Care, University of Pisa, Pisa 56126, Italy
| | - Alessandro Lenci
- Department of Philology, Literature, and Linguistics, University of Pisa, Pisa 56126, Italy
| | - Luca Cecchetti
- Dept. Surgical, Medical, Molecular Pathology and Critical Care, University of Pisa, Pisa 56126, Italy
| | | | - Giovanna Marotta
- Department of Philology, Literature, and Linguistics, University of Pisa, Pisa 56126, Italy
| | - Pietro Pietrini
- Dept. Surgical, Medical, Molecular Pathology and Critical Care, University of Pisa, Pisa 56126, Italy; Clinical Psychology Branch, Pisa University Hospital, Pisa 56126, Italy; IMT School for Advanced Studies Lucca, Lucca 55100, Italy.
| |
Collapse
|
30
|
Siuda-Krzywicka K, Bola Ł, Paplińska M, Sumera E, Jednoróg K, Marchewka A, Śliwińska MW, Amedi A, Szwed M. Massive cortical reorganization in sighted Braille readers. eLife 2016; 5:e10762. [PMID: 26976813 PMCID: PMC4805536 DOI: 10.7554/elife.10762] [Citation(s) in RCA: 70] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/12/2015] [Accepted: 01/19/2016] [Indexed: 12/05/2022] Open
Abstract
The brain is capable of large-scale reorganization in blindness or after massive injury. Such reorganization crosses the division into separate sensory cortices (visual, somatosensory...). As its result, the visual cortex of the blind becomes active during tactile Braille reading. Although the possibility of such reorganization in the normal, adult brain has been raised, definitive evidence has been lacking. Here, we demonstrate such extensive reorganization in normal, sighted adults who learned Braille while their brain activity was investigated with fMRI and transcranial magnetic stimulation (TMS). Subjects showed enhanced activity for tactile reading in the visual cortex, including the visual word form area (VWFA) that was modulated by their Braille reading speed and strengthened resting-state connectivity between visual and somatosensory cortices. Moreover, TMS disruption of VWFA activity decreased their tactile reading accuracy. Our results indicate that large-scale reorganization is a viable mechanism recruited when learning complex skills. DOI:http://dx.doi.org/10.7554/eLife.10762.001 According to most textbooks, our brain is divided into separate areas that are dedicated to specific senses. We have a visual cortex for vision, a tactile cortex for touch, and so on. However, researchers suspect that this division might not be as fixed as the textbooks say. For example, blind people can switch their 'leftover' visual cortex to non-visual purposes, such as reading Braille – a tactile alphabet. Can this switch in functional organization also happen in healthy people with normal vision? To investigate this, Siuda-Krzywicka, Bola et al. taught a group of healthy, sighted people to read Braille by touch, and monitored the changes in brain activity that this caused using a technique called functional magnetic resonance imaging. According to textbooks, tactile reading should engage the tactile cortex. Yet, the experiment revealed that the brain activity critical for reading Braille by touch did not occur in the volunteers’ tactile cortex, but in their visual cortex. Further experiments used a technique called transcranial magnetic stimulation to suppress the activity of the visual cortex of the volunteers. This impaired their ability to read Braille by touch. This is a clear-cut proof that sighted adults can re-program their visual cortex for non-visual, tactile purposes. These results show that intensive training in a complex task can overcome the sensory division-of-labor of our brain. This indicates that our brain is much more flexible than previously thought, and that such flexibility might occur when we learn everyday, complex skills such as driving a car or playing a musical instrument. The next question that follows from this work is: what enables the brain’s activity to change after learning to read Braille? To understand this, Siuda-Krzywicka, Bola et al. are currently exploring how the physical structure of the brain changes as a result of a person acquiring the ability to read Braille by touch. DOI:http://dx.doi.org/10.7554/eLife.10762.002
Collapse
Affiliation(s)
- Katarzyna Siuda-Krzywicka
- Department of Psychology, Jagiellonian University, Kraków, Poland.,INSERM U 1127, CNRS UMR 7225, Sorbonne Universités, and Université Pierre et Marie Curie-Paris 6, UMR S 1127, Institut du Cerveau et de la Moelle épinière (ICM), Paris, France
| | - Łukasz Bola
- Department of Psychology, Jagiellonian University, Kraków, Poland.,Laboratory of Brain Imaging, Neurobiology Center, Nencki Institute of Experimental Biology, Warsaw, Poland
| | | | - Ewa Sumera
- Institute for the Blind and Partially Sighted Children in Krakow, Kraków, Poland
| | - Katarzyna Jednoróg
- Laboratory of Psychophysiology, Department of Neurophysiology, Nencki Institute of Experimental Biology, Warsaw, Poland
| | - Artur Marchewka
- Laboratory of Brain Imaging, Neurobiology Center, Nencki Institute of Experimental Biology, Warsaw, Poland
| | - Magdalena W Śliwińska
- Department of Experimental Psychology, University College London, London, United Kingdom
| | - Amir Amedi
- The Cognitive Science Program, The Hebrew University of Jerusalem, Jerusalem, Israel.,Department of Medical Neurobiology, The Institute for Medical Research Israel-Canada, Faculty of Medicine, The Hebrew University of Jerusalem, Jerusalem, Israel.,The Edmond and Lily Safra Center for Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel.,Sorbonne Universite´s, UPMC Univ Paris 06, Institut de la Vision, Paris, France
| | - Marcin Szwed
- Department of Psychology, Jagiellonian University, Kraków, Poland
| |
Collapse
|
31
|
Reich L, Amedi A. 'Visual' parsing can be taught quickly without visual experience during critical periods. Sci Rep 2015; 5:15359. [PMID: 26482105 PMCID: PMC4611203 DOI: 10.1038/srep15359] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2015] [Accepted: 09/15/2015] [Indexed: 12/12/2022] Open
Abstract
Cases of invasive sight-restoration in congenital blind adults demonstrated that acquiring visual abilities is extremely challenging, presumably because visual-experience during critical-periods is crucial for learning visual-unique concepts (e.g. size constancy). Visual rehabilitation can also be achieved using sensory-substitution-devices (SSDs) which convey visual information non-invasively through sounds. We tested whether one critical concept – visual parsing, which is highly-impaired in sight-restored patients – can be learned using SSD. To this end, congenitally blind adults participated in a unique, relatively short (~70 hours), SSD-‘vision’ training. Following this, participants successfully parsed 2D and 3D visual objects. Control individuals naïve to SSDs demonstrated that while some aspects of parsing with SSD are intuitive, the blind’s success could not be attributed to auditory processing alone. Furthermore, we had a unique opportunity to compare the SSD-users’ abilities to those reported for sight-restored patients who performed similar tasks visually, and who had months of eyesight. Intriguingly, the SSD-users outperformed the patients on most criteria tested. These suggest that with adequate training and technologies, key high-order visual features can be quickly acquired in adulthood, and lack of visual-experience during critical-periods can be somewhat compensated for. Practically, these highlight the potential of SSDs as standalone-aids or combined with invasive restoration approaches.
Collapse
Affiliation(s)
- Lior Reich
- Department of Medical Neurobiology, The Institute for Medical Research Israel-Canada, Faculty of Medicine, The Hebrew University of Jerusalem, Jerusalem 91220, Israel
| | - Amir Amedi
- Department of Medical Neurobiology, The Institute for Medical Research Israel-Canada, Faculty of Medicine, The Hebrew University of Jerusalem, Jerusalem 91220, Israel.,The Edmond and Lily Safra Center for Brain Sciences (ELSC), The Hebrew University of Jerusalem, Jerusalem 91220, Israel
| |
Collapse
|
32
|
Paredes LP, Dosen S, Rattay F, Graimann B, Farina D. The impact of the stimulation frequency on closed-loop control with electrotactile feedback. J Neuroeng Rehabil 2015; 12:35. [PMID: 25889752 PMCID: PMC4403675 DOI: 10.1186/s12984-015-0022-8] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2014] [Accepted: 02/24/2015] [Indexed: 11/22/2022] Open
Abstract
BACKGROUND Electrocutaneous stimulation can restore the missing sensory information to prosthetic users. In electrotactile feedback, the information about the prosthesis state is transmitted in the form of pulse trains. The stimulation frequency is an important parameter since it influences the data transmission rate over the feedback channel as well as the form of the elicited tactile sensations. METHODS We evaluated the influence of the stimulation frequency on the subject's ability to utilize the feedback information during electrotactile closed-loop control. Ten healthy subjects performed a real-time compensatory tracking (standard test bench) of sinusoids and pseudorandom signals using either visual feedback (benchmark) or electrocutaneous feedback in seven conditions characterized by different combinations of the stimulation frequency (FSTIM) and tracking error sampling rate (FTE). The tracking error was transmitted using two concentric electrodes placed on the forearm. The quality of tracking was assessed using the Squared Pearson Correlation Coefficient (SPCC), the Normalized Root Mean Square Tracking Error (NRMSTE) and the time delay between the reference and generated trajectories (TDIO). RESULTS The results demonstrated that FSTIM was more important for the control performance than FTE. The quality of tracking deteriorated with a decrease in the stimulation frequency, SPCC and NRMSTE (mean) were 87.5% and 9.4% in the condition 100/100 (FTE/FSTIM), respectively, and deteriorated to 61.1% and 15.3% in 5/5, respectively, while the TDIO increased from 359.8 ms in 100/100 to 1009 ms in 5/5. However, the performance recovered when the tracking error sampled at a low rate was delivered using a high stimulation frequency (SPCC = 83.6%, NRMSTE = 10.3%, TDIO = 415.6 ms, in 5/100). CONCLUSIONS The likely reason for the performance decrease and recovery was that the stimulation frequency critically influenced the tactile perception quality and thereby the effective rate of information transfer through the feedback channel. The outcome of this study can facilitate the selection of optimal system parameters for somatosensory feedback in upper limb prostheses. The results imply that the feedback variables (e.g., grasping force) should be transmitted at relatively high frequencies of stimulation (>25 Hz), but that they can be sampled at much lower rates (e.g., 5 Hz).
Collapse
Affiliation(s)
- Liliana P Paredes
- Laboratorio di Cinematica e Robotica, Fondazione Ospedale San Camillo - I.R.C.C.S., Lido di Venezia, Italy.
| | - Strahinja Dosen
- Department of Neurorehabilitation Engineering, University Medical Center Goettingen, Goettingen, Germany.
| | - Frank Rattay
- Institute for Analysis and Scientific Computing, Vienna University of Technology, Vienna, Austria.
| | - Bernhard Graimann
- Translational Research and Knowledge Management, Otto Bock Healthcare GmbH, Duderstadt, Germany.
| | - Dario Farina
- Department of Neurorehabilitation Engineering, University Medical Center Goettingen, Goettingen, Germany.
| |
Collapse
|
33
|
Abstract
Distinct preference for visual number symbols was recently discovered in the human right inferior temporal gyrus (rITG). It remains unclear how this preference emerges, what is the contribution of shape biases to its formation and whether visual processing underlies it. Here we use congenital blindness as a model for brain development without visual experience. During fMRI, we present blind subjects with shapes encoded using a novel visual-to-music sensory-substitution device (The EyeMusic). Greater activation is observed in the rITG when subjects process symbols as numbers compared with control tasks on the same symbols. Using resting-state fMRI in the blind and sighted, we further show that the areas with preference for numerals and letters exhibit distinct patterns of functional connectivity with quantity and language-processing areas, respectively. Our findings suggest that specificity in the ventral ‘visual’ stream can emerge independently of sensory modality and visual experience, under the influence of distinct connectivity patterns. The human visual cortex includes areas with preference for various object categories. Here, Abboud et al. demonstrate using visual-to-music substitution, that the congenitally blind show a similar preference for numerals in the right inferior temporal cortex as sighted individuals, despite having no visual experience.
Collapse
|
34
|
Multisensory perceptual learning and sensory substitution. Neurosci Biobehav Rev 2014; 41:16-25. [DOI: 10.1016/j.neubiorev.2012.11.017] [Citation(s) in RCA: 73] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/17/2012] [Revised: 11/19/2012] [Accepted: 11/28/2012] [Indexed: 11/23/2022]
|
35
|
Mind the blind brain to understand the sighted one! Is there a supramodal cortical functional architecture? Neurosci Biobehav Rev 2014; 41:64-77. [DOI: 10.1016/j.neubiorev.2013.10.006] [Citation(s) in RCA: 103] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2013] [Revised: 08/13/2013] [Accepted: 10/03/2013] [Indexed: 11/20/2022]
|
36
|
Qin W, Xuan Y, Liu Y, Jiang T, Yu C. Functional Connectivity Density in Congenitally and Late Blind Subjects. Cereb Cortex 2014; 25:2507-16. [PMID: 24642421 DOI: 10.1093/cercor/bhu051] [Citation(s) in RCA: 80] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
Abstract
Visual deprivation during different developmental periods leads to different structural and functional alterations in the brain; however, the effects of visual deprivation on the spontaneous functional organization of the brain remain largely unknown. In this study, we used voxel-based functional connectivity density (FCD) analyses to investigate the effects of visual deprivation during different developmental periods on the spontaneous functional organization of the brain. Compared with the sighted controls (SC), both the congenitally blind (CB) and the late blind (LB) exhibited decreased short- and long-range FCDs in the primary visual cortex (V1) and decreased long-range FCDs in the primary somatosensory and auditory cortices. Although both the CB and LB exhibited increased short-range FCD in the dorsal visual stream, the CB exhibited greater increases in the short- and long-range FCDs in the ventral visual stream and hippocampal complex compared with the LB. Moreover, the short-range FCD of the left V1 exhibited a significant positive correlation with the duration of blindness in the LB. Our findings suggest that visual deprivation before the developmental sensitive period can induce more extensive brain functional reorganization than does visual deprivation after the sensitive period, which may underlie an enhanced capacity for processing nonvisual information in the CB.
Collapse
Affiliation(s)
- Wen Qin
- Department of Radiology and Tianjin Key Laboratory of Functional Imaging
| | - Yun Xuan
- Department of Anatomy, School of Basic Medical Sciences, Tianjin Medical University, Tianjin 300052, China
| | - Yong Liu
- Brainnetome Center, Institute of Automation, National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
| | - Tianzi Jiang
- Brainnetome Center, Institute of Automation, National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
| | - Chunshui Yu
- Department of Radiology and Tianjin Key Laboratory of Functional Imaging
| |
Collapse
|
37
|
Hertz U, Amedi A. Flexibility and Stability in Sensory Processing Revealed Using Visual-to-Auditory Sensory Substitution. Cereb Cortex 2014; 25:2049-64. [PMID: 24518756 PMCID: PMC4494022 DOI: 10.1093/cercor/bhu010] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/24/2022] Open
Abstract
The classical view of sensory processing involves independent processing in sensory cortices and multisensory integration in associative areas. This hierarchical structure has been challenged by evidence of multisensory responses in sensory areas, and dynamic weighting of sensory inputs in associative areas, thus far reported independently. Here, we used a visual-to-auditory sensory substitution algorithm (SSA) to manipulate the information conveyed by sensory inputs while keeping the stimuli intact. During scan sessions before and after SSA learning, subjects were presented with visual images and auditory soundscapes. The findings reveal 2 dynamic processes. First, crossmodal attenuation of sensory cortices changed direction after SSA learning from visual attenuations of the auditory cortex to auditory attenuations of the visual cortex. Secondly, associative areas changed their sensory response profile from strongest response for visual to that for auditory. The interaction between these phenomena may play an important role in multisensory processing. Consistent features were also found in the sensory dominance in sensory areas and audiovisual convergence in associative area Middle Temporal Gyrus. These 2 factors allow for both stability and a fast, dynamic tuning of the system when required.
Collapse
Affiliation(s)
- Uri Hertz
- Department of Medical Neurobiology, Institute for Medical Research Israel-Canada (IMRIC), Hadassah Medical School, Hebrew University of Jerusalem, Jerusalem 91220, Israel Interdisciplinary Center for Neural Computation, The Edmond & Lily Safra Center for Brain Sciences (ELSC), Hebrew University of Jerusalem, Jerusalem 91905, Israel
| | - Amir Amedi
- Department of Medical Neurobiology, Institute for Medical Research Israel-Canada (IMRIC), Hadassah Medical School, Hebrew University of Jerusalem, Jerusalem 91220, Israel Interdisciplinary Center for Neural Computation, The Edmond & Lily Safra Center for Brain Sciences (ELSC), Hebrew University of Jerusalem, Jerusalem 91905, Israel
| |
Collapse
|
38
|
Abstract
Humans typically rely upon vision to identify object shape, but we can also recognize shape via touch (haptics). Our haptic shape recognition ability raises an intriguing question: To what extent do visual cortical shape recognition mechanisms support haptic object recognition? We addressed this question using a haptic fMRI repetition design, which allowed us to identify neuronal populations sensitive to the shape of objects that were touched but not seen. In addition to the expected shape-selective fMRI responses in dorsal frontoparietal areas, we observed widespread shape-selective responses in the ventral visual cortical pathway, including primary visual cortex. Our results indicate that shape processing via touch engages many of the same neural mechanisms as visual object recognition. The shape-specific repetition effects we observed in primary visual cortex show that visual sensory areas are engaged during the haptic exploration of object shape, even in the absence of concurrent shape-related visual input. Our results complement related findings in visually deprived individuals and highlight the fundamental role of the visual system in the processing of object shape.
Collapse
|
39
|
Neural pathways conveying novisual information to the visual cortex. Neural Plast 2013; 2013:864920. [PMID: 23840972 PMCID: PMC3690246 DOI: 10.1155/2013/864920] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2013] [Accepted: 05/22/2013] [Indexed: 11/18/2022] Open
Abstract
The visual cortex has been traditionally considered as a stimulus-driven, unimodal system with a hierarchical organization. However, recent animal and human studies have shown that the visual cortex responds to non-visual stimuli, especially in individuals with visual deprivation congenitally, indicating the supramodal nature of the functional representation in the visual cortex. To understand the neural substrates of the cross-modal processing of the non-visual signals in the visual cortex, we firstly showed the supramodal nature of the visual cortex. We then reviewed how the nonvisual signals reach the visual cortex. Moreover, we discussed if these non-visual pathways are reshaped by early visual deprivation. Finally, the open question about the nature (stimulus-driven or top-down) of non-visual signals is also discussed.
Collapse
|
40
|
Shape-specific activation of occipital cortex in an early blind echolocation expert. Neuropsychologia 2013; 51:938-49. [DOI: 10.1016/j.neuropsychologia.2013.01.024] [Citation(s) in RCA: 45] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2012] [Revised: 01/21/2013] [Accepted: 01/27/2013] [Indexed: 02/02/2023]
|
41
|
Qin W, Liu Y, Jiang T, Yu C. The development of visual areas depends differently on visual experience. PLoS One 2013; 8:e53784. [PMID: 23308283 PMCID: PMC3538632 DOI: 10.1371/journal.pone.0053784] [Citation(s) in RCA: 46] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2012] [Accepted: 12/05/2012] [Indexed: 11/29/2022] Open
Abstract
Visual experience plays an important role in the development of the visual cortex; however, recent functional imaging studies have shown that the functional organization is preserved in several higher-tier visual areas in congenitally blind subjects, indicating that maturation of visual areas depend unequally on visual experience. In this study, we aim to validate this hypothesis using a multimodality MRI approach. We found increased cortical thickness in the congenitally blind was present in the early visual areas and absent in the higher-tier ones, suggesting that the structural development of the visual cortex depends hierarchically on visual experience. In congenitally blind subjects, the decreased resting-state functional connectivity with the primary somatosensory cortex was more prominent in the early visual areas than in the higher-tier ones and were more pronounced in the ventral stream than in the dorsal one, suggesting that the development of functional organization of the visual cortex also depends differently on visual experience. Moreover, congenitally blind subjects showed normal or increased functional connectivity between ipsilateral higher-tier and early visual areas, suggesting an indirect corticocortical pathway through which somatosenroy information can reach the early visual areas. These findings support our hypothesis that the development of visual areas depends differently on visual experience.
Collapse
Affiliation(s)
- Wen Qin
- Department of Radiology, Tianjin Medical University General Hospital, Tianjin, China
| | - Yong Liu
- LIAMA Center for Computational Medicine, National Laboratory of Pattern Recognition Institute of Automation, Chinese Academy of Sciences, Beijing, China
| | - Tianzi Jiang
- LIAMA Center for Computational Medicine, National Laboratory of Pattern Recognition Institute of Automation, Chinese Academy of Sciences, Beijing, China
- * E-mail: (CY); (TJ)
| | - Chunshui Yu
- Department of Radiology, Tianjin Medical University General Hospital, Tianjin, China
- Department of Radiology, Xuanwu Hospital, Capital Medical University, Beijing, China
- * E-mail: (CY); (TJ)
| |
Collapse
|
42
|
Affiliation(s)
- Patrice Voss
- Montreal Neurological Institute, McGill University, Montreal, Canada, 3801 rue University, Montréal, Québec, Canada, H3A 2B4.
| | | |
Collapse
|
43
|
Kassuba T, Menz MM, Röder B, Siebner HR. Multisensory interactions between auditory and haptic object recognition. ACTA ACUST UNITED AC 2012; 23:1097-107. [PMID: 22518017 DOI: 10.1093/cercor/bhs076] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
Object manipulation produces characteristic sounds and causes specific haptic sensations that facilitate the recognition of the manipulated object. To identify the neural correlates of audio-haptic binding of object features, healthy volunteers underwent functional magnetic resonance imaging while they matched a target object to a sample object within and across audition and touch. By introducing a delay between the presentation of sample and target stimuli, it was possible to dissociate haptic-to-auditory and auditory-to-haptic matching. We hypothesized that only semantically coherent auditory and haptic object features activate cortical regions that host unified conceptual object representations. The left fusiform gyrus (FG) and posterior superior temporal sulcus (pSTS) showed increased activation during crossmodal matching of semantically congruent but not incongruent object stimuli. In the FG, this effect was found for haptic-to-auditory and auditory-to-haptic matching, whereas the pSTS only displayed a crossmodal matching effect for congruent auditory targets. Auditory and somatosensory association cortices showed increased activity during crossmodal object matching which was, however, independent of semantic congruency. Together, the results show multisensory interactions at different hierarchical stages of auditory and haptic object processing. Object-specific crossmodal interactions culminate in the left FG, which may provide a higher order convergence zone for conceptual object knowledge.
Collapse
Affiliation(s)
- Tanja Kassuba
- Danish Research Centre for Magnetic Resonance, Copenhagen University Hospital Hvidovre, 2650 Hvidovre, Denmark.
| | | | | | | |
Collapse
|
44
|
Watkins KE, Cowey A, Alexander I, Filippini N, Kennedy JM, Smith SM, Ragge N, Bridge H. Language networks in anophthalmia: maintained hierarchy of processing in 'visual' cortex. ACTA ACUST UNITED AC 2012; 135:1566-77. [PMID: 22427328 DOI: 10.1093/brain/aws067] [Citation(s) in RCA: 70] [Impact Index Per Article: 5.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
Imaging studies in blind subjects have consistently shown that sensory and cognitive tasks evoke activity in the occipital cortex, which is normally visual. The precise areas involved and degree of activation are dependent upon the cause and age of onset of blindness. Here, we investigated the cortical language network at rest and during an auditory covert naming task in five bilaterally anophthalmic subjects, who have never received visual input. When listening to auditory definitions and covertly retrieving words, these subjects activated lateral occipital cortex bilaterally in addition to the language areas activated in sighted controls. This activity was significantly greater than that present in a control condition of listening to reversed speech. The lateral occipital cortex was also recruited into a left-lateralized resting-state network that usually comprises anterior and posterior language areas. Levels of activation to the auditory naming and reversed speech conditions did not differ in the calcarine (striate) cortex. This primary 'visual' cortex was not recruited to the left-lateralized resting-state network and showed high interhemispheric correlation of activity at rest, as is typically seen in unimodal cortical areas. In contrast, the interhemispheric correlation of resting activity in extrastriate areas was reduced in anophthalmia to the level of cortical areas that are heteromodal, such as the inferior frontal gyrus. Previous imaging studies in the congenitally blind show that primary visual cortex is activated in higher-order tasks, such as language and memory to a greater extent than during more basic sensory processing, resulting in a reversal of the normal hierarchy of functional organization across 'visual' areas. Our data do not support such a pattern of organization in anophthalmia. Instead, the patterns of activity during task and the functional connectivity at rest are consistent with the known hierarchy of processing in these areas normally seen for vision. The differences in cortical organization between bilateral anophthalmia and other forms of congenital blindness are considered to be due to the total absence of stimulation in 'visual' cortex by light or retinal activity in the former condition, and suggests development of subcortical auditory input to the geniculo-striate pathway.
Collapse
Affiliation(s)
- Kate E Watkins
- Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, UK.
| | | | | | | | | | | | | | | |
Collapse
|
45
|
|
46
|
Tempere S, Cuzange E, Bougeant JC, de Revel G, Sicard G. Explicit Sensory Training Improves the Olfactory Sensitivity of Wine Experts. CHEMOSENS PERCEPT 2012. [DOI: 10.1007/s12078-012-9120-1] [Citation(s) in RCA: 25] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022]
|
47
|
Striem-Amit E, Dakwar O, Reich L, Amedi A. The large-Scale Organization of “Visual” Streams Emerges Without Visual Experience. Cereb Cortex 2011; 22:1698-709. [DOI: 10.1093/cercor/bhr253] [Citation(s) in RCA: 95] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/26/2023] Open
|