1
|
Damera SR, Malone PS, Stevens BW, Klein R, Eberhardt SP, Auer ET, Bernstein LE, Riesenhuber M. Metamodal Coupling of Vibrotactile and Auditory Speech Processing Systems through Matched Stimulus Representations. J Neurosci 2023; 43:4984-4996. [PMID: 37197979 PMCID: PMC10324991 DOI: 10.1523/jneurosci.1710-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2022] [Revised: 03/10/2023] [Accepted: 04/29/2023] [Indexed: 05/19/2023] Open
Abstract
It has been postulated that the brain is organized by "metamodal," sensory-independent cortical modules capable of performing tasks (e.g., word recognition) in both "standard" and novel sensory modalities. Still, this theory has primarily been tested in sensory-deprived individuals, with mixed evidence in neurotypical subjects, thereby limiting its support as a general principle of brain organization. Critically, current theories of metamodal processing do not specify requirements for successful metamodal processing at the level of neural representations. Specification at this level may be particularly important in neurotypical individuals, where novel sensory modalities must interface with existing representations for the standard sense. Here we hypothesized that effective metamodal engagement of a cortical area requires congruence between stimulus representations in the standard and novel sensory modalities in that region. To test this, we first used fMRI to identify bilateral auditory speech representations. We then trained 20 human participants (12 female) to recognize vibrotactile versions of auditory words using one of two auditory-to-vibrotactile algorithms. The vocoded algorithm attempted to match the encoding scheme of auditory speech while the token-based algorithm did not. Crucially, using fMRI, we found that only in the vocoded group did trained-vibrotactile stimuli recruit speech representations in the superior temporal gyrus and lead to increased coupling between them and somatosensory areas. Our results advance our understanding of brain organization by providing new insight into unlocking the metamodal potential of the brain, thereby benefitting the design of novel sensory substitution devices that aim to tap into existing processing streams in the brain.SIGNIFICANCE STATEMENT It has been proposed that the brain is organized by "metamodal," sensory-independent modules specialized for performing certain tasks. This idea has inspired therapeutic applications, such as sensory substitution devices, for example, enabling blind individuals "to see" by transforming visual input into soundscapes. Yet, other studies have failed to demonstrate metamodal engagement. Here, we tested the hypothesis that metamodal engagement in neurotypical individuals requires matching the encoding schemes between stimuli from the novel and standard sensory modalities. We trained two groups of subjects to recognize words generated by one of two auditory-to-vibrotactile transformations. Critically, only vibrotactile stimuli that were matched to the neural encoding of auditory speech engaged auditory speech areas after training. This suggests that matching encoding schemes is critical to unlocking the brain's metamodal potential.
Collapse
Affiliation(s)
- Srikanth R Damera
- Department of Neuroscience, Georgetown University Medical Center, Washington, DC 20007
| | - Patrick S Malone
- Department of Neuroscience, Georgetown University Medical Center, Washington, DC 20007
| | - Benson W Stevens
- Department of Neuroscience, Georgetown University Medical Center, Washington, DC 20007
| | - Richard Klein
- Department of Neuroscience, Georgetown University Medical Center, Washington, DC 20007
| | - Silvio P Eberhardt
- Department of Speech Language & Hearing Sciences, George Washington University, Washington, DC 20052
| | - Edward T Auer
- Department of Speech Language & Hearing Sciences, George Washington University, Washington, DC 20052
| | - Lynne E Bernstein
- Department of Speech Language & Hearing Sciences, George Washington University, Washington, DC 20052
| | | |
Collapse
|
2
|
Macklin AS, Yau JM, Fischer-Baum S, O'Malley MK. Representational Similarity Analysis for Tracking Neural Correlates of Haptic Learning on a Multimodal Device. IEEE TRANSACTIONS ON HAPTICS 2023; 16:424-435. [PMID: 37556331 PMCID: PMC10605963 DOI: 10.1109/toh.2023.3303838] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/11/2023]
Abstract
A goal of wearable haptic devices has been to enable haptic communication, where individuals learn to map information typically processed visually or aurally to haptic cues via a process of cross-modal associative learning. Neural correlates have been used to evaluate haptic perception and may provide a more objective approach to assess association performance than more commonly used behavioral measures of performance. In this article, we examine Representational Similarity Analysis (RSA) of electroencephalography (EEG) as a framework to evaluate how the neural representation of multifeatured haptic cues changes with association training. We focus on the first phase of cross-modal associative learning, perception of multimodal cues. A participant learned to map phonemes to multimodal haptic cues, and EEG data were acquired before and after training to create neural representational spaces that were compared to theoretical models. Our perceptual model showed better correlations to the neural representational space before training, while the feature-based model showed better correlations with the post-training data. These results suggest that training may lead to a sharpening of the sensory response to haptic cues. Our results show promise that an EEG-RSA approach can capture a shift in the representational space of cues, as a means to track haptic learning.
Collapse
|
3
|
Li G, Yap PT. From descriptive connectome to mechanistic connectome: Generative modeling in functional magnetic resonance imaging analysis. Front Hum Neurosci 2022; 16:940842. [PMID: 36061504 PMCID: PMC9428697 DOI: 10.3389/fnhum.2022.940842] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/10/2022] [Accepted: 07/28/2022] [Indexed: 01/28/2023] Open
Abstract
As a newly emerging field, connectomics has greatly advanced our understanding of the wiring diagram and organizational features of the human brain. Generative modeling-based connectome analysis, in particular, plays a vital role in deciphering the neural mechanisms of cognitive functions in health and dysfunction in diseases. Here we review the foundation and development of major generative modeling approaches for functional magnetic resonance imaging (fMRI) and survey their applications to cognitive or clinical neuroscience problems. We argue that conventional structural and functional connectivity (FC) analysis alone is not sufficient to reveal the complex circuit interactions underlying observed neuroimaging data and should be supplemented with generative modeling-based effective connectivity and simulation, a fruitful practice that we term "mechanistic connectome." The transformation from descriptive connectome to mechanistic connectome will open up promising avenues to gain mechanistic insights into the delicate operating principles of the human brain and their potential impairments in diseases, which facilitates the development of effective personalized treatments to curb neurological and psychiatric disorders.
Collapse
Affiliation(s)
- Guoshi Li
- Department of Radiology, University of North Carolina, Chapel Hill, NC, United States,Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, NC, United States,*Correspondence: Guoshi Li,
| | - Pew-Thian Yap
- Department of Radiology, University of North Carolina, Chapel Hill, NC, United States,Biomedical Research Imaging Center, University of North Carolina, Chapel Hill, NC, United States
| |
Collapse
|
4
|
Velenosi LA, Wu YH, Schmidt TT, Blankenburg F. Intraparietal sulcus maintains working memory representations of somatosensory categories in an adaptive, context-dependent manner. Neuroimage 2020; 221:117146. [DOI: 10.1016/j.neuroimage.2020.117146] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2019] [Revised: 07/03/2020] [Accepted: 07/04/2020] [Indexed: 02/01/2023] Open
|
5
|
Low Cost MR Compatible Haptic Stimulation with Application to fMRI Neurofeedback. Brain Sci 2020; 10:brainsci10110790. [PMID: 33126691 PMCID: PMC7692267 DOI: 10.3390/brainsci10110790] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Revised: 10/23/2020] [Accepted: 10/26/2020] [Indexed: 11/17/2022] Open
Abstract
The most common feedback displays in the fMRI environment are visual, e.g., in which participants try to increase or decrease the level of a thermometer. However, haptic feedback is increasingly valued in computer interaction tasks, particularly for real-time fMRI feedback. fMRI-neurofeedback is a clinical intervention that has not yet taken advantage of this trend. Here we describe a low-cost, user-friendly, MR-compatible system that can provide graded haptic vibrotactile stimulation in an initial application to fMRI neurofeedback. We also present a feasibility demonstration showing that we could successfully set up the system and obtain data in the context of a neurofeedback paradigm. We conclude that vibrotactile stimulation using this low-cost system is a viable method of feedback presentation, and encourage neurofeedback researchers to incorporate this type of feedback into their studies.
Collapse
|
6
|
Jiang X, Wang Y, Li X, Wang L, Zhou YD, Wang H. A Simple and Compact MR-Compatible Electromagnetic Vibrotactile Stimulator. Front Neurosci 2020; 13:1403. [PMID: 32009884 PMCID: PMC6978794 DOI: 10.3389/fnins.2019.01403] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2019] [Accepted: 12/12/2019] [Indexed: 12/26/2022] Open
Abstract
We have developed a low-cost electromagnetic vibrotactile stimulator that uses the magnetic field of an MR scanner as a permanent magnet to power a vibrating motor. A simple variable current power supply is controlled by software using a USB data acquisition controller. In our study, the function of our novel stimulator was verified in a vibration frequency discrimination working memory task, in which various ranges of frequencies and amplitudes are delivered in MRI scanner. Furthermore, our functional MRI study revealed activations of the primary and secondary somatosensory cortices during the perception of tactile stimulation. Therefore, the new designed electromagnetic vibrotactile stimulator is capable of generating various frequencies of tactile stimuli and represents a powerful and useful tool for studying somatosensory functions with functional MRI.
Collapse
Affiliation(s)
- Xinjian Jiang
- Key Laboratory of Brain Functional Genomics (MOE & STCSM), School of Psychology and Cognitive Science, Institute of Cognitive Neuroscience, East China Normal University, Shanghai, China
| | - Yueqian Wang
- Key Laboratory of Brain Functional Genomics (MOE & STCSM), School of Psychology and Cognitive Science, Institute of Cognitive Neuroscience, East China Normal University, Shanghai, China
| | - Xiaojin Li
- Department of Electronic Engineering, School of Information Science and Technology, East China Normal University, Shanghai, China
| | - Liping Wang
- Key Laboratory of Primate Neurobiology, CAS Center for Excellence in Brain Science and Intelligence Technology, Institute of Neuroscience, Chinese Academy of Sciences, Shanghai, China
| | - Yong-Di Zhou
- Key Laboratory of Brain Functional Genomics (MOE & STCSM), School of Psychology and Cognitive Science, Institute of Cognitive Neuroscience, East China Normal University, Shanghai, China
| | - Huimin Wang
- Key Laboratory of Brain Functional Genomics (MOE & STCSM), School of Psychology and Cognitive Science, Institute of Cognitive Neuroscience, East China Normal University, Shanghai, China
| |
Collapse
|
7
|
Malone PS, Eberhardt SP, Wimmer K, Sprouse C, Klein R, Glomb K, Scholl CA, Bokeria L, Cho P, Deco G, Jiang X, Bernstein LE, Riesenhuber M. Neural mechanisms of vibrotactile categorization. Hum Brain Mapp 2019; 40:3078-3090. [PMID: 30920706 PMCID: PMC6865665 DOI: 10.1002/hbm.24581] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2018] [Revised: 02/18/2019] [Accepted: 03/12/2019] [Indexed: 11/11/2022] Open
Abstract
The grouping of sensory stimuli into categories is fundamental to cognition. Previous research in the visual and auditory systems supports a two-stage processing hierarchy that underlies perceptual categorization: (a) a "bottom-up" perceptual stage in sensory cortices where neurons show selectivity for stimulus features and (b) a "top-down" second stage in higher level cortical areas that categorizes the stimulus-selective input from the first stage. In order to test the hypothesis that the two-stage model applies to the somatosensory system, 14 human participants were trained to categorize vibrotactile stimuli presented to their right forearm. Then, during an fMRI scan, participants actively categorized the stimuli. Representational similarity analysis revealed stimulus selectivity in areas including the left precentral and postcentral gyri, the supramarginal gyrus, and the posterior middle temporal gyrus. Crucially, we identified a single category-selective region in the left ventral precentral gyrus. Furthermore, an estimation of directed functional connectivity delivered evidence for robust top-down connectivity from the second to first stage. These results support the validity of the two-stage model of perceptual categorization for the somatosensory system, suggesting common computational principles and a unified theory of perceptual categorization across the visual, auditory, and somatosensory systems.
Collapse
Affiliation(s)
- Patrick S. Malone
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| | - Silvio P. Eberhardt
- Department of Speech, Language, and Hearing SciencesGeorge Washington UniversityWashingtonDistrict of Columbia
| | - Klaus Wimmer
- Center for Brain and Cognition, Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain
- Centre de Recerca MatemàticaBarcelonaSpain
- Barcelona Graduate School of MathematicsBarcelonaSpain
| | - Courtney Sprouse
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| | - Richard Klein
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| | - Katharina Glomb
- Center for Brain and Cognition, Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain
- Department of RadiologyCentre Hospitalier Universitaire VaudoisLausanneSwitzerland
| | - Clara A. Scholl
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| | - Levan Bokeria
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| | - Philip Cho
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| | - Gustavo Deco
- Center for Brain and Cognition, Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain
- Institució Catalana de la Recerca i Estudis Avançats (ICREA)BarcelonaSpain
- Department of NeuropsychologyMax Planck Institute for Human Cognitive and Brain SciencesLeipzigGermany
- School of Psychological SciencesMonash UniversityMelbourneVictoriaAustralia
| | - Xiong Jiang
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| | - Lynne E. Bernstein
- Department of Speech, Language, and Hearing SciencesGeorge Washington UniversityWashingtonDistrict of Columbia
| | - Maximilian Riesenhuber
- Department of NeuroscienceGeorgetown University Medical CenterWashingtonDistrict of Columbia
| |
Collapse
|