1
|
Gruenwald J, Sieghartsleitner S, Kapeller C, Scharinger J, Kamada K, Brunner P, Guger C. Characterization of High-Gamma Activity in Electrocorticographic Signals. Front Neurosci 2023; 17:1206120. [PMID: 37609450 PMCID: PMC10440607 DOI: 10.3389/fnins.2023.1206120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2023] [Accepted: 07/10/2023] [Indexed: 08/24/2023] Open
Abstract
Introduction Electrocorticographic (ECoG) high-gamma activity (HGA) is a widely recognized and robust neural correlate of cognition and behavior. However, fundamental signal properties of HGA, such as the high-gamma frequency band or temporal dynamics of HGA, have never been systematically characterized. As a result, HGA estimators are often poorly adjusted, such that they miss valuable physiological information. Methods To address these issues, we conducted a thorough qualitative and quantitative characterization of HGA in ECoG signals. Our study is based on ECoG signals recorded from 18 epilepsy patients while performing motor control, listening, and visual perception tasks. In this study, we first categorize HGA into HGA types based on the cognitive/behavioral task. For each HGA type, we then systematically quantify three fundamental signal properties of HGA: the high-gamma frequency band, the HGA bandwidth, and the temporal dynamics of HGA. Results The high-gamma frequency band strongly varies across subjects and across cognitive/behavioral tasks. In addition, HGA time courses have lowpass character, with transients limited to 10 Hz. The task-related rise time and duration of these HGA time courses depend on the individual subject and cognitive/behavioral task. Task-related HGA amplitudes are comparable across the investigated tasks. Discussion This study is of high practical relevance because it provides a systematic basis for optimizing experiment design, ECoG acquisition and processing, and HGA estimation. Our results reveal previously unknown characteristics of HGA, the physiological principles of which need to be investigated in further studies.
Collapse
Affiliation(s)
- Johannes Gruenwald
- g.tec medical engineering GmbH, Schiedlberg, Austria
- Institute of Computational Perception, Johannes Kepler University, Linz, Austria
| | - Sebastian Sieghartsleitner
- g.tec medical engineering GmbH, Schiedlberg, Austria
- Institute of Computational Perception, Johannes Kepler University, Linz, Austria
| | | | - Josef Scharinger
- Institute of Computational Perception, Johannes Kepler University, Linz, Austria
| | - Kyousuke Kamada
- Department for Neurosurgery, Asahikawa Medical University, Asahikawa, Japan
- Hokashin Group Megumino Hospital, Sapporo, Japan
| | - Peter Brunner
- National Center for Adaptive Neurotechnologies, Albany, NY, United States
- Department of Neurosurgery, Washington University School of Medicine, St. Louis, MO, United States
| | | |
Collapse
|
2
|
Bellier L, Llorens A, Marciano D, Gunduz A, Schalk G, Brunner P, Knight RT. Music can be reconstructed from human auditory cortex activity using nonlinear decoding models. PLoS Biol 2023; 21:e3002176. [PMID: 37582062 PMCID: PMC10427021 DOI: 10.1371/journal.pbio.3002176] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2022] [Accepted: 05/30/2023] [Indexed: 08/17/2023] Open
Abstract
Music is core to human experience, yet the precise neural dynamics underlying music perception remain unknown. We analyzed a unique intracranial electroencephalography (iEEG) dataset of 29 patients who listened to a Pink Floyd song and applied a stimulus reconstruction approach previously used in the speech domain. We successfully reconstructed a recognizable song from direct neural recordings and quantified the impact of different factors on decoding accuracy. Combining encoding and decoding analyses, we found a right-hemisphere dominance for music perception with a primary role of the superior temporal gyrus (STG), evidenced a new STG subregion tuned to musical rhythm, and defined an anterior-posterior STG organization exhibiting sustained and onset responses to musical elements. Our findings show the feasibility of applying predictive modeling on short datasets acquired in single patients, paving the way for adding musical elements to brain-computer interface (BCI) applications.
Collapse
Affiliation(s)
- Ludovic Bellier
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
| | - Anaïs Llorens
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
| | - Déborah Marciano
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
| | - Aysegul Gunduz
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida, Gainesville, Florida, United States of America
| | - Gerwin Schalk
- Department of Neurology, Albany Medical College, Albany, New York, United States of America
| | - Peter Brunner
- Department of Neurology, Albany Medical College, Albany, New York, United States of America
- Department of Neurosurgery, Washington University School of Medicine, St. Louis, Missouri, United States of America
- National Center for Adaptive Neurotechnologies, Albany, New York, United States of America
| | - Robert T. Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, Berkeley, California, United States of America
- Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America
| |
Collapse
|
3
|
McCarty MJ, Murphy E, Scherschligt X, Woolnough O, Morse CW, Snyder K, Mahon BZ, Tandon N. Intraoperative cortical localization of music and language reveals signatures of structural complexity in posterior temporal cortex. iScience 2023; 26:107223. [PMID: 37485361 PMCID: PMC10362292 DOI: 10.1016/j.isci.2023.107223] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2022] [Revised: 06/01/2023] [Accepted: 06/22/2023] [Indexed: 07/25/2023] Open
Abstract
Language and music involve the productive combination of basic units into structures. It remains unclear whether brain regions sensitive to linguistic and musical structure are co-localized. We report an intraoperative awake craniotomy in which a left-hemispheric language-dominant professional musician underwent cortical stimulation mapping (CSM) and electrocorticography of music and language perception and production during repetition tasks. Musical sequences were melodic or amelodic, and differed in algorithmic compressibility (Lempel-Ziv complexity). Auditory recordings of sentences differed in syntactic complexity (single vs. multiple phrasal embeddings). CSM of posterior superior temporal gyrus (pSTG) disrupted music perception and production, along with speech production. pSTG and posterior middle temporal gyrus (pMTG) activated for language and music (broadband gamma; 70-150 Hz). pMTG activity was modulated by musical complexity, while pSTG activity was modulated by syntactic complexity. This points to shared resources for music and language comprehension, but distinct neural signatures for the processing of domain-specific structural features.
Collapse
Affiliation(s)
- Meredith J. McCarty
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Xavier Scherschligt
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Oscar Woolnough
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Cale W. Morse
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Kathryn Snyder
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
| | - Bradford Z. Mahon
- Department of Psychology, Carnegie Mellon University, Pittsburgh, PA 15213, USA
- Neuroscience Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA
| | - Nitin Tandon
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Texas Institute for Restorative Neurotechnologies, University of Texas Health Science Center at Houston, Houston, TX 77030, USA
- Memorial Hermann Hospital, Texas Medical Center, Houston, TX 77030, USA
| |
Collapse
|
4
|
Bakas S, Adamos DA, Laskaris N. On the estimate of music appraisal from surface EEG: a dynamic-network approach based on cross-sensor PAC measurements. J Neural Eng 2021; 18. [PMID: 33975291 DOI: 10.1088/1741-2552/abffe6] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/26/2020] [Accepted: 05/11/2021] [Indexed: 11/11/2022]
Abstract
Objective.The aesthetic evaluation of music is strongly dependent on the listener and reflects manifold brain processes that go well beyond the perception of incident sound. Being a high-level cognitive reaction, it is difficult to predict merely from the acoustic features of the audio signal and this poses serious challenges to contemporary music recommendation systems. We attempted to decode music appraisal from brain activity, recorded via wearable EEG, during music listening.Approach.To comply with the dynamic nature of music stimuli, cross-frequency coupling measurements were employed in a time-evolving manner to capture the evolving interactions between distinct brain-rhythms during music listening. Brain response to music was first represented as a continuous flow of functional couplings referring to both regional and inter-regional brain dynamics and then modelled as an ensemble of time-varying (sub)networks. Dynamic graph centrality measures were derived, next, as the final feature-engineering step and, lastly, a support-vector machine was trained to decode the subjective music appraisal. A carefully designed experimental paradigm provided the labeled brain signals.Main results.Using data from 20 subjects, dynamic programming to tailor the decoder to each subject individually and cross-validation, we demonstrated highly satisfactory performance (MAE= 0.948,R2= 0.63) that can be attributed, mostly, to interactions of left frontal gamma rhythm. In addition, our music-appraisal decoder was also employed in a part of the DEAP dataset with similar success. Finally, even a generic version of the decoder (common for all subjects) was found to perform sufficiently.Significance.A novel brain signal decoding scheme was introduced and validated empirically on suitable experimental data. It requires simple operations and leaves room for real-time implementation. Both the code and the experimental data are publicly available.
Collapse
Affiliation(s)
- Stylianos Bakas
- Department of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece.,Neuroinformatics GRoup, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Dimitrios A Adamos
- School of Music Studies, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece.,Department of Computing, Imperial College London, SW7 2AZ London, United Kingdom.,Neuroinformatics GRoup, Aristotle University of Thessaloniki, Thessaloniki, Greece
| | - Nikolaos Laskaris
- Department of Informatics, Aristotle University of Thessaloniki, 54124 Thessaloniki, Greece.,Neuroinformatics GRoup, Aristotle University of Thessaloniki, Thessaloniki, Greece
| |
Collapse
|
5
|
Prefrontal High Gamma in ECoG Tags Periodicity of Musical Rhythms in Perception and Imagination. eNeuro 2020; 7:ENEURO.0413-19.2020. [PMID: 32586843 PMCID: PMC7405071 DOI: 10.1523/eneuro.0413-19.2020] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2019] [Revised: 05/19/2020] [Accepted: 06/01/2020] [Indexed: 01/08/2023] Open
Abstract
Rhythmic auditory stimuli are known to elicit matching activity patterns in neural populations. Furthermore, recent research has established the particular importance of high-gamma brain activity in auditory processing by showing its involvement in auditory phrase segmentation and envelope tracking. Here, we use electrocorticographic (ECoG) recordings from eight human listeners to see whether periodicities in high-gamma activity track the periodicities in the envelope of musical rhythms during rhythm perception and imagination. Rhythm imagination was elicited by instructing participants to imagine the rhythm to continue during pauses of several repetitions. To identify electrodes whose periodicities in high-gamma activity track the periodicities in the musical rhythms, we compute the correlation between the autocorrelations (ACCs) of both the musical rhythms and the neural signals. A condition in which participants listened to white noise was used to establish a baseline. High-gamma autocorrelations in auditory areas in the superior temporal gyrus and in frontal areas on both hemispheres significantly matched the autocorrelations of the musical rhythms. Overall, numerous significant electrodes are observed on the right hemisphere. Of particular interest is a large cluster of electrodes in the right prefrontal cortex that is active during both rhythm perception and imagination. This indicates conscious processing of the rhythms’ structure as opposed to mere auditory phenomena. The autocorrelation approach clearly highlights that high-gamma activity measured from cortical electrodes tracks both attended and imagined rhythms.
Collapse
|
6
|
Sonkusare S, Nguyen VT, Moran R, van der Meer J, Ren Y, Koussis N, Dionisio S, Breakspear M, Guo C. Intracranial-EEG evidence for medial temporal pole driving amygdala activity induced by multi-modal emotional stimuli. Cortex 2020; 130:32-48. [PMID: 32640373 DOI: 10.1016/j.cortex.2020.05.018] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2020] [Revised: 05/13/2020] [Accepted: 05/29/2020] [Indexed: 12/13/2022]
Abstract
The temporal pole (TP) is an associative cortical region required for complex cognitive functions such as social and emotional cognition. However, mapping the TP with functional magnetic resonance imaging is technically challenging and thus understanding its interaction with other key emotional circuitry, such as the amygdala, remains elusive. We exploited the unique advantages of stereo-electroencephalography (sEEG) to assess the responses of the TP and the amygdala during the perception of emotionally salient stimuli of pictures, music and movies. These stimuli consistently elicited high gamma responses (70-140 Hz) in both the TP and the amygdala, accompanied by functional connectivity in the low frequency range (2-12 Hz). Computational analyses suggested that the TP drove this effect in the theta frequency range, modulated by the emotional valence of the stimuli. Notably, cross-frequency analysis indicated the phase of theta oscillations in the TP modulated the amplitude of high gamma activity in the amygdala. These results were reproducible across three types of sensory inputs including naturalistic stimuli. Our results suggest that multimodal emotional stimuli induce a hierarchical influence of the TP over the amygdala.
Collapse
Affiliation(s)
- Saurabh Sonkusare
- QIMR Berghofer Medical Research Institute, Brisbane, Australia; School of Medicine, The University of Queensland, Brisbane, Australia.
| | - Vinh T Nguyen
- QIMR Berghofer Medical Research Institute, Brisbane, Australia
| | - Rosalyn Moran
- Department of Neuroimaging, Institute of Psychiatry, Psychology and Neuroscience, King's College London, London, UK
| | | | - Yudan Ren
- QIMR Berghofer Medical Research Institute, Brisbane, Australia; School of Information Science and Technology, Northwest University, Xi'an, China
| | - Nikitas Koussis
- QIMR Berghofer Medical Research Institute, Brisbane, Australia
| | - Sasha Dionisio
- Mater Advanced Epilepsy Unit, Mater Hospital, Brisbane, Australia
| | - Michael Breakspear
- QIMR Berghofer Medical Research Institute, Brisbane, Australia; Hunter Medical Research Institute, University of Newcastle, Newcastle, Australia.
| | - Christine Guo
- QIMR Berghofer Medical Research Institute, Brisbane, Australia
| |
Collapse
|
7
|
Abstract
Intracranial electroencephalography (iEEG) is measured from electrodes placed in or on the brain. These measurements have an excellent signal-to-noise ratio and iEEG signals have often been used to decode brain activity or drive brain-computer interfaces (BCIs). iEEG recordings are typically done for seizure monitoring in epilepsy patients who have these electrodes placed for a clinical purpose: to localize both brain regions that are essential for function and others where seizures start. Brain regions not involved in epilepsy are thought to function normally and provide a unique opportunity to learn about human neurophysiology. Intracranial electrodes measure the aggregate activity of large neuronal populations and recorded signals contain many features. Different features are extracted by analyzing these signals in the time and frequency domain. The time domain may reveal an evoked potential at a particular time after the onset of an event. Decomposition into the frequency domain may show narrowband peaks in the spectrum at specific frequencies or broadband signal changes that span a wide range of frequencies. Broadband power increases are generally observed when a brain region is active while most other features are highly specific to brain regions, inputs, and tasks. Here we describe the spatiotemporal dynamics of several iEEG signals that have often been used to decode brain activity and drive BCIs.
Collapse
|
8
|
Dougherty ME, Nguyen APQ, Baratham VL, Bouchard KE. Laminar origin of evoked ECoG high-gamma activity. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2020; 2019:4391-4394. [PMID: 31946840 DOI: 10.1109/embc.2019.8856786] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/10/2022]
Abstract
High-gamma (Hγ) activity from electrocorticography (ECoG) is a common-used signal for understanding the human brain, but its interpretation is impeded by a lack of spatial localization. To address this, we developed a novel recording approach to simultaneously record μECoG cortical surface electrical potentials (CSEPs) and laminar multiunit activity (MUA). We demonstrate that stimulus evoked CSEPs carry a multi-modal frequency response, peaking in the Hγ range. Laminar MUA responses exhibited similar tuning to CSEP Hγ directly over the intracortical recording site, suggesting a functional relationship. We fit CSEP Hγ to the simultaneously-recorded laminar MUA using a state-of-the-art sparse multi-linear regression model to identify laminar contributions to cortical surface Hγ. Our results indicate that CSEP Hγ recorded by ECoG reflects spiking activity from neurons in layer 3. These results provide initial insight into localizing the sources of CSEPs, which will guide clinical and BMI device decisions.
Collapse
|
9
|
Neural Correlates of Music Listening and Recall in the Human Brain. J Neurosci 2019; 39:8112-8123. [PMID: 31501297 DOI: 10.1523/jneurosci.1468-18.2019] [Citation(s) in RCA: 20] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2018] [Revised: 08/13/2019] [Accepted: 08/14/2019] [Indexed: 11/21/2022] Open
Abstract
Previous neuroimaging studies have identified various brain regions that are activated by music listening or recall. However, little is known about how these brain regions represent the time course and temporal features of music during listening and recall. Here we analyzed neural activity in different brain regions associated with music listening and recall using electrocorticography recordings obtained from 10 epilepsy patients of both genders implanted with subdural electrodes. Electrocorticography signals were recorded while subjects were listening to familiar instrumental music or recalling the same music pieces by imagery. During the onset phase (0-500 ms), music listening initiated cortical activity in high-gamma band in the temporal lobe and supramarginal gyrus, followed by the precentral gyrus and the inferior frontal gyrus. In contrast, during music recall, the high-gamma band activity first appeared in the inferior frontal gyrus and precentral gyrus, and then spread to the temporal lobe, showing a reversed temporal sequential order. During the sustained phase (after 500 ms), delta band and high-gamma band responses in the supramarginal gyrus, temporal and frontal lobes dynamically tracked the intensity envelope of the music during listening or recall with distinct temporal delays. During music listening, the neural tracking by the frontal lobe lagged behind that of the temporal lobe; whereas during music recall, the neural tracking by the frontal lobe preceded that of the temporal lobe. These findings demonstrate bottom-up and top-down processes in the cerebral cortex during music listening and recall and provide important insights into music processing by the human brain.SIGNIFICANCE STATEMENT Understanding how the brain analyzes, stores, and retrieves music remains one of the most challenging problems in neuroscience. By analyzing direct neural recordings obtained from the human brain, we observed dispersed and overlapping brain regions associated with music listening and recall. Music listening initiated cortical activity in high-gamma band starting from the temporal lobe and ending at the inferior frontal gyrus. A reversed temporal flow was observed in high-gamma response during music recall. Neural responses of frontal and temporal lobes dynamically tracked the intensity envelope of music that was presented or imagined during listening or recall. These findings demonstrate bottom-up and top-down processes in the cerebral cortex during music listening and recall.
Collapse
|
10
|
Yang Y, Sani OG, Chang EF, Shanechi MM. Dynamic network modeling and dimensionality reduction for human ECoG activity. J Neural Eng 2019; 16:056014. [DOI: 10.1088/1741-2552/ab2214] [Citation(s) in RCA: 28] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/31/2022]
|
11
|
Kapeller C, Ogawa H, Schalk G, Kunii N, Coon WG, Scharinger J, Guger C, Kamada K. Real-time detection and discrimination of visual perception using electrocorticographic signals. J Neural Eng 2018; 15:036001. [PMID: 29359711 DOI: 10.1088/1741-2552/aaa9f6] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/30/2023]
Abstract
OBJECTIVE Several neuroimaging studies have demonstrated that the ventral temporal cortex contains specialized regions that process visual stimuli. This study investigated the spatial and temporal dynamics of electrocorticographic (ECoG) responses to different types and colors of visual stimulation that were presented to four human participants, and demonstrated a real-time decoder that detects and discriminates responses to untrained natural images. APPROACH ECoG signals from the participants were recorded while they were shown colored and greyscale versions of seven types of visual stimuli (images of faces, objects, bodies, line drawings, digits, and kanji and hiragana characters), resulting in 14 classes for discrimination (experiment I). Additionally, a real-time system asynchronously classified ECoG responses to faces, kanji and black screens presented via a monitor (experiment II), or to natural scenes (i.e. the face of an experimenter, natural images of faces and kanji, and a mirror) (experiment III). Outcome measures in all experiments included the discrimination performance across types based on broadband γ activity. MAIN RESULTS Experiment I demonstrated an offline classification accuracy of 72.9% when discriminating among the seven types (without color separation). Further discrimination of grey versus colored images reached an accuracy of 67.1%. Discriminating all colors and types (14 classes) yielded an accuracy of 52.1%. In experiment II and III, the real-time decoder correctly detected 73.7% responses to face, kanji and black computer stimuli and 74.8% responses to presented natural scenes. SIGNIFICANCE Seven different types and their color information (either grey or color) could be detected and discriminated using broadband γ activity. Discrimination performance maximized for combined spatial-temporal information. The discrimination of stimulus color information provided the first ECoG-based evidence for color-related population-level cortical broadband γ responses in humans. Stimulus categories can be detected by their ECoG responses in real time within 500 ms with respect to stimulus onset.
Collapse
Affiliation(s)
- C Kapeller
- Guger Technologies OG, Graz, Austria. Department of Computational Perception, Johannes Kepler University, Linz, Austria
| | | | | | | | | | | | | | | |
Collapse
|
12
|
Todaro C, Marzetti L, Valdés Sosa PA, Valdés-Hernandez PA, Pizzella V. Mapping Brain Activity with Electrocorticography: Resolution Properties and Robustness of Inverse Solutions. Brain Topogr 2018; 32:583-598. [PMID: 29362974 DOI: 10.1007/s10548-018-0623-1] [Citation(s) in RCA: 12] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2017] [Accepted: 01/16/2018] [Indexed: 10/18/2022]
Abstract
Electrocorticography (ECoG) is an electrophysiological technique that records brain activity directly from the cortical surface with high temporal (ms) and spatial (mm) resolution. Its major limitations are in the high invasiveness and in the restricted field-of-view of the electrode grid, which partially covers the cortex. To infer brain activity at locations different from just below the electrodes, it is necessary to solve the electromagnetic inverse problem. Limitations in the performance of source reconstruction algorithms from ECoG have been, to date, only partially addressed in the literature, and a systematic evaluation is still lacking. The main goal of this study is to provide a quantitative evaluation of resolution properties of widely used inverse methods (eLORETA and MNE) for various ECoG grid sizes, in terms of localization error, spatial dispersion, and overall amplitude. Additionally, this study aims at evaluating how the use of simultaneous electroencephalography (EEG) affects the above properties. For these purposes, we take advantage of a unique dataset in which a monkey underwent a simultaneous recording with a 128 channel ECoG grid and an 18 channel EEG grid. Our results show that, in general conditions, the reconstruction of cortical activity located more than 1 cm away from the ECoG grid is not accurate, since the localization error increases linearly with the distance from the electrodes. This problem can be partially overcome by recording simultaneously ECoG and EEG. However, this analysis enlightens the necessity to design inverse algorithms specifically targeted at taking into account the limited field-of-view of the ECoG grid.
Collapse
|
13
|
Cui Z, Wang Q, Gao Y, Wang J, Wang M, Teng P, Guan Y, Zhou J, Li T, Luan G, Li L. Dynamic Correlations between Intrinsic Connectivity and Extrinsic Connectivity of the Auditory Cortex in Humans. Front Hum Neurosci 2017; 11:407. [PMID: 28848415 PMCID: PMC5554526 DOI: 10.3389/fnhum.2017.00407] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2017] [Accepted: 07/25/2017] [Indexed: 12/31/2022] Open
Abstract
The arrival of sound signals in the auditory cortex (AC) triggers both local and inter-regional signal propagations over time up to hundreds of milliseconds and builds up both intrinsic functional connectivity (iFC) and extrinsic functional connectivity (eFC) of the AC. However, interactions between iFC and eFC are largely unknown. Using intracranial stereo-electroencephalographic recordings in people with drug-refractory epilepsy, this study mainly investigated the temporal dynamic of the relationships between iFC and eFC of the AC. The results showed that a Gaussian wideband-noise burst markedly elicited potentials in both the AC and numerous higher-order cortical regions outside the AC (non-auditory cortices). Granger causality analyses revealed that in the earlier time window, iFC of the AC was positively correlated with both eFC from the AC to the inferior temporal gyrus and that to the inferior parietal lobule. While in later periods, the iFC of the AC was positively correlated with eFC from the precentral gyrus to the AC and that from the insula to the AC. In conclusion, dual-directional interactions occur between iFC and eFC of the AC at different time windows following the sound stimulation and may form the foundation underlying various central auditory processes, including auditory sensory memory, object formation, integrations between sensory, perceptional, attentional, motor, emotional, and executive processes.
Collapse
Affiliation(s)
- Zhuang Cui
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China.,Beijing HospitalBeijing, China
| | - Qian Wang
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China.,School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Key Laboratory of Machine Perception (Ministry of Education), Peking UniversityBeijing, China
| | - Yayue Gao
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Key Laboratory of Machine Perception (Ministry of Education), Peking UniversityBeijing, China
| | - Jing Wang
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China
| | - Mengyang Wang
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China
| | - Pengfei Teng
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China
| | - Yuguang Guan
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China
| | - Jian Zhou
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China
| | - Tianfu Li
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China.,Beijing Institute for Brain DisordersBeijing, China
| | - Guoming Luan
- Beijing Key Laboratory of Epilepsy, Epilepsy Center, Department of Functional Neurosurgery, Sanbo Brain Hospital, Capital Medical UniversityBeijing, China.,Beijing Institute for Brain DisordersBeijing, China
| | - Liang Li
- School of Psychological and Cognitive Sciences and Beijing Key Laboratory of Behavior and Mental Health, Key Laboratory of Machine Perception (Ministry of Education), Peking UniversityBeijing, China.,Beijing Institute for Brain DisordersBeijing, China
| |
Collapse
|
14
|
Ramírez MIG, Rodríguez-Arias LR, Santiago AO, Pizano AL, Zamora RL, Gregorio RV, Trenado C, Sánchez HMG, San-Juan D. Correlation Between Bispectral Index and Electrocorticographic Features During Epilepsy Surgery. Clin EEG Neurosci 2017; 48:272-279. [PMID: 27325591 DOI: 10.1177/1550059416654850] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Surgical resection guided by intraoperative electrocorticography (iECoG) has been in clinical use for many decades. The use of the bispectral index (BIS) for monitoring depth of anesthesia during different types of surgery, including epilepsy surgery, is increasing nowadays. The BIS is an EEG-derived variable indicating cortical electrical activity. However, the correlation between the BIS score and the iECoG score, with the purpose of optimizing the quality and time of the iECoG recordings in epilepsy surgery is unknown. The goal of this study was to evaluate the correlation between BIS values and iECoG parameters during the epilepsy surgery under anesthesia with propofol and fentanyl. This is a prospective study that included patients with epilepsy who underwent epilepsy surgery guided by BIS and iECoG (September 2008 to October 2013). Clinical, physiological, and sociodemographic characteristics are shown. We correlated the iECoG parameters (presence of burst suppressions [BS], suppression time [seconds], background frequency [Hz], and type of iECoG score by Mathern et al) with BIS values. We included 28 patients, 15/28 (53.5%) female, general mean age of 30.5 years (range 13-56 years). Patients underwent epilepsy surgery: 22/28 (79%) temporal and 6/28 (21%) extratemporal. We found a nonlinear polynomial cubic relationship between the mentioned variables noting that a BIS range of 40 to 60 gave the following results: iECoG BS periods <5 seconds, background frequency 10 to 17 Hz, and iECoG score 2 characterized by lack of >20-Hz background frequencies. No BS were observed with a BIS > 60. In conclusion BIS values and iECoG parameters during the epilepsy surgery under anesthesia with propofol and fentanyl have a nonlinear correlation. BS patterns were not found with a BIS > 60. These findings show that BIS is a nonlinear multidimensional measure, which possesses high variability with the iECoG parameters. BS patterns are not found with BIS > 60.
Collapse
Affiliation(s)
| | | | - Areli O Santiago
- 1 Neuroanesthesiology Department. National Institute of Neurology, Mexico City, Mexico
| | | | | | - Rafael V Gregorio
- 3 Clinical Neurophysiology Department, National Institute of Neurology, Mexico City, Mexico
| | - Carlos Trenado
- 4 Institute of Clinical Neuroscience and Medical Psychology, University Hospital Düsseldorf, Dusseldorf, Germany
| | - Héctor Manuel G Sánchez
- 5 Faculty of Medicine of the Autonomous University of Baja California, Campus of Mexicali, Mexicali, Baja California, Mexico
| | - Daniel San-Juan
- 3 Clinical Neurophysiology Department, National Institute of Neurology, Mexico City, Mexico
| |
Collapse
|
15
|
Schalk G, Marple J, Knight RT, Coon WG. Instantaneous voltage as an alternative to power- and phase-based interpretation of oscillatory brain activity. Neuroimage 2017. [PMID: 28624646 DOI: 10.1016/j.neuroimage.2017.06.014] [Citation(s) in RCA: 18] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022] Open
Abstract
For decades, oscillatory brain activity has been characterized primarily by measurements of power and phase. While many studies have linked those measurements to cortical excitability, their relationship to each other and to the physiological underpinnings of excitability is unclear. The recently proposed Function-through-Biased-Oscillations (FBO) hypothesis (Schalk, 2015) addressed these issues by suggesting that the voltage potential at the cortical surface directly reflects the excitability of cortical populations, that this voltage is rhythmically driven away from a low resting potential (associated with depolarized cortical populations) towards positivity (associated with hyperpolarized cortical populations). This view explains how oscillatory power and phase together influence the instantaneous voltage potential that directly regulates cortical excitability. This implies that the alternative measurement of instantaneous voltage of oscillatory activity should better predict cortical excitability compared to either of the more traditional measurements of power or phase. Using electrocorticographic (ECoG) data from 28 human subjects, the results of our study confirm this prediction: compared to oscillatory power and phase, the instantaneous voltage explained 20% and 31% more of the variance in broadband gamma, respectively, and power and phase together did not produce better predictions than the instantaneous voltage. These results synthesize the previously separate power- and phase-based interpretations and associate oscillatory activity directly with a physiological interpretation of cortical excitability. This alternative view has implications for the interpretation of studies of oscillatory activity and for current theories of cortical information transmission.
Collapse
Affiliation(s)
- Gerwin Schalk
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Dept. of Health, Albany, NY, United States; Dept. of Neurology, Albany Medical College, Albany, NY, United States; Dept. of Biomedical Sciences, State University of New York, Albany, NY, United States.
| | - Joshua Marple
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Dept. of Health, Albany, NY, United States; Dept. of Computer Science, University of Kansas, Lawrence, KS, United States
| | - Robert T Knight
- Dept. of Psychology and The Helen Wills Neuroscience Institute, University of California at Berkeley, Berkeley, CA, United States
| | - William G Coon
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Dept. of Health, Albany, NY, United States; Dept. of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, MA, United States
| |
Collapse
|
16
|
Qiu Z, Allison BZ, Jin J, Zhang Y, Wang X, Li W, Cichocki A. Optimized Motor Imagery Paradigm Based on Imagining Chinese Characters Writing Movement. IEEE Trans Neural Syst Rehabil Eng 2017; 25:1009-1017. [PMID: 28113345 DOI: 10.1109/tnsre.2017.2655542] [Citation(s) in RCA: 36] [Impact Index Per Article: 5.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
BACKGROUND motor imagery (MI) is a mental representation of motor behavior. The MI-based brain computer interfaces (BCIs) can provide communication for the physically impaired. The performance of MI-based BCI mainly depends on the subject's ability to self-modulate electroencephalogram signals. Proper training can help naive subjects learn to modulate brain activity proficiently. However, training subjects typically involve abstract motor tasks and are time-consuming. METHODS to improve the performance of naive subjects during motor imagery, a novel paradigm was presented that would guide naive subjects to modulate brain activity effectively. In this new paradigm, pictures of the left or right hand were used as cues for subjects to finish the motor imagery task. Fourteen healthy subjects (11 male, aged 22-25 years, and mean 23.6±1.16) participated in this study. The task was to imagine writing a Chinese character. Specifically, subjects could imagine hand movements corresponding to the sequence of writing strokes in the Chinese character. This paradigm was meant to find an effective and familiar action for most Chinese people, to provide them with a specific, extensively practiced task and help them modulate brain activity. RESULTS results showed that the writing task paradigm yielded significantly better performance than the traditional arrow paradigm (p < 0.001). Questionnaire replies indicated that most subjects thought that the new paradigm was easier. CONCLUSION the proposed new motor imagery paradigm could guide subjects to help them modulate brain activity effectively. Results showed that there were significant improvements using new paradigm, both in classification accuracy and usability.
Collapse
|
17
|
Brumberg JS, Krusienski DJ, Chakrabarti S, Gunduz A, Brunner P, Ritaccio AL, Schalk G. Spatio-Temporal Progression of Cortical Activity Related to Continuous Overt and Covert Speech Production in a Reading Task. PLoS One 2016; 11:e0166872. [PMID: 27875590 PMCID: PMC5119784 DOI: 10.1371/journal.pone.0166872] [Citation(s) in RCA: 32] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2016] [Accepted: 11/04/2016] [Indexed: 11/18/2022] Open
Abstract
How the human brain plans, executes, and monitors continuous and fluent speech has remained largely elusive. For example, previous research has defined the cortical locations most important for different aspects of speech function, but has not yet yielded a definition of the temporal progression of involvement of those locations as speech progresses either overtly or covertly. In this paper, we uncovered the spatio-temporal evolution of neuronal population-level activity related to continuous overt speech, and identified those locations that shared activity characteristics across overt and covert speech. Specifically, we asked subjects to repeat continuous sentences aloud or silently while we recorded electrical signals directly from the surface of the brain (electrocorticography (ECoG)). We then determined the relationship between cortical activity and speech output across different areas of cortex and at sub-second timescales. The results highlight a spatio-temporal progression of cortical involvement in the continuous speech process that initiates utterances in frontal-motor areas and ends with the monitoring of auditory feedback in superior temporal gyrus. Direct comparison of cortical activity related to overt versus covert conditions revealed a common network of brain regions involved in speech that may implement orthographic and phonological processing. Our results provide one of the first characterizations of the spatiotemporal electrophysiological representations of the continuous speech process, and also highlight the common neural substrate of overt and covert speech. These results thereby contribute to a refined understanding of speech functions in the human brain.
Collapse
Affiliation(s)
- Jonathan S. Brumberg
- Department of Speech-Language-Hearing: Sciences & Disorders, University of Kansas, Lawrence, KS, United States of America
- * E-mail:
| | - Dean J. Krusienski
- Department of Electrical & Computer Engineering, Old Dominion University, Norfolk, VA, United States of America
| | - Shreya Chakrabarti
- Department of Electrical & Computer Engineering, Old Dominion University, Norfolk, VA, United States of America
| | - Aysegul Gunduz
- J. Crayton Pruitt Family Dept. of Biomedical Engineering, University of Florida, Gainesville, FL, United States of America
| | - Peter Brunner
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, United States of America
- Department of Neurology, Albany Medical College, Albany, NY, United States of America
| | - Anthony L. Ritaccio
- Department of Neurology, Albany Medical College, Albany, NY, United States of America
| | - Gerwin Schalk
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, United States of America
- Department of Neurology, Albany Medical College, Albany, NY, United States of America
| |
Collapse
|
18
|
Herff C, Schultz T. Automatic Speech Recognition from Neural Signals: A Focused Review. Front Neurosci 2016; 10:429. [PMID: 27729844 PMCID: PMC5037201 DOI: 10.3389/fnins.2016.00429] [Citation(s) in RCA: 60] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2016] [Accepted: 09/05/2016] [Indexed: 11/13/2022] Open
Abstract
Speech interfaces have become widely accepted and are nowadays integrated in various real-life applications and devices. They have become a part of our daily life. However, speech interfaces presume the ability to produce intelligible speech, which might be impossible due to either loud environments, bothering bystanders or incapabilities to produce speech (i.e., patients suffering from locked-in syndrome). For these reasons it would be highly desirable to not speak but to simply envision oneself to say words or sentences. Interfaces based on imagined speech would enable fast and natural communication without the need for audible speech and would give a voice to otherwise mute people. This focused review analyzes the potential of different brain imaging techniques to recognize speech from neural signals by applying Automatic Speech Recognition technology. We argue that modalities based on metabolic processes, such as functional Near Infrared Spectroscopy and functional Magnetic Resonance Imaging, are less suited for Automatic Speech Recognition from neural signals due to low temporal resolution but are very useful for the investigation of the underlying neural mechanisms involved in speech processes. In contrast, electrophysiologic activity is fast enough to capture speech processes and is therefor better suited for ASR. Our experimental results indicate the potential of these signals for speech recognition from neural data with a focus on invasively measured brain activity (electrocorticography). As a first example of Automatic Speech Recognition techniques used from neural signals, we discuss the Brain-to-text system.
Collapse
Affiliation(s)
- Christian Herff
- Cognitive Systems Lab, Department for Mathematics and Computer Science, University of Bremen Bremen, Germany
| | - Tanja Schultz
- Cognitive Systems Lab, Department for Mathematics and Computer Science, University of Bremen Bremen, Germany
| |
Collapse
|
19
|
de Pesters A, Coon WG, Brunner P, Gunduz A, Ritaccio AL, Brunet NM, de Weerd P, Roberts MJ, Oostenveld R, Fries P, Schalk G. Alpha power indexes task-related networks on large and small scales: A multimodal ECoG study in humans and a non-human primate. Neuroimage 2016; 134:122-131. [PMID: 27057960 PMCID: PMC4912924 DOI: 10.1016/j.neuroimage.2016.03.074] [Citation(s) in RCA: 66] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/02/2016] [Accepted: 03/28/2016] [Indexed: 12/19/2022] Open
Abstract
Performing different tasks, such as generating motor movements or processing sensory input, requires the recruitment of specific networks of neuronal populations. Previous studies suggested that power variations in the alpha band (8-12Hz) may implement such recruitment of task-specific populations by increasing cortical excitability in task-related areas while inhibiting population-level cortical activity in task-unrelated areas (Klimesch et al., 2007; Jensen and Mazaheri, 2010). However, the precise temporal and spatial relationships between the modulatory function implemented by alpha oscillations and population-level cortical activity remained undefined. Furthermore, while several studies suggested that alpha power indexes task-related populations across large and spatially separated cortical areas, it was largely unclear whether alpha power also differentially indexes smaller networks of task-related neuronal populations. Here we addressed these questions by investigating the temporal and spatial relationships of electrocorticographic (ECoG) power modulations in the alpha band and in the broadband gamma range (70-170Hz, indexing population-level activity) during auditory and motor tasks in five human subjects and one macaque monkey. In line with previous research, our results confirm that broadband gamma power accurately tracks task-related behavior and that alpha power decreases in task-related areas. More importantly, they demonstrate that alpha power suppression lags population-level activity in auditory areas during the auditory task, but precedes it in motor areas during the motor task. This suppression of alpha power in task-related areas was accompanied by an increase in areas not related to the task. In addition, we show for the first time that these differential modulations of alpha power could be observed not only across widely distributed systems (e.g., motor vs. auditory system), but also within the auditory system. Specifically, alpha power was suppressed in the locations within the auditory system that most robustly responded to particular sound stimuli. Altogether, our results provide experimental evidence for a mechanism that preferentially recruits task-related neuronal populations by increasing cortical excitability in task-related cortical areas and decreasing cortical excitability in task-unrelated areas. This mechanism is implemented by variations in alpha power and is common to humans and the non-human primate under study. These results contribute to an increasingly refined understanding of the mechanisms underlying the selection of the specific neuronal populations required for task execution.
Collapse
Affiliation(s)
- A de Pesters
- Nat Ctr for Adapt Neurotech, Wadsworth Center, NY State Dept of Health, Albany, NY, USA; Dept of Biomed Sci, State Univ of New York at Albany, Albany, NY, USA.
| | - W G Coon
- Nat Ctr for Adapt Neurotech, Wadsworth Center, NY State Dept of Health, Albany, NY, USA.
| | - P Brunner
- Nat Ctr for Adapt Neurotech, Wadsworth Center, NY State Dept of Health, Albany, NY, USA; Dept of Neurology, Albany Medical College, Albany, NY, USA.
| | - A Gunduz
- Dept of Biomed Eng, Univ of Florida, Gainesville, FL, USA.
| | - A L Ritaccio
- Dept of Neurology, Albany Medical College, Albany, NY, USA.
| | - N M Brunet
- SUNY Downstate Med Ctr, Brooklyn, NY, USA.
| | - P de Weerd
- Dept of Cogn Neurosci, Maastricht Univ, Maastricht, Netherlands; Donders Inst for Brain, Cognition and Behaviour, Nijmegen, Netherlands.
| | - M J Roberts
- Donders Inst for Brain, Cognition and Behaviour, Nijmegen, Netherlands.
| | - R Oostenveld
- Donders Inst for Brain, Cognition and Behaviour, Nijmegen, Netherlands.
| | - P Fries
- Donders Inst for Brain, Cognition and Behaviour, Nijmegen, Netherlands; Ernst Strüngmann Inst for Neurosci, Frankfurt, Germany.
| | - G Schalk
- Nat Ctr for Adapt Neurotech, Wadsworth Center, NY State Dept of Health, Albany, NY, USA; Dept of Biomed Sci, State Univ of New York at Albany, Albany, NY, USA; Dept of Neurology, Albany Medical College, Albany, NY, USA.
| |
Collapse
|
20
|
Foo F, King-Stephens D, Weber P, Laxer K, Parvizi J, Knight RT. Differential Processing of Consonance and Dissonance within the Human Superior Temporal Gyrus. Front Hum Neurosci 2016; 10:154. [PMID: 27148011 PMCID: PMC4829599 DOI: 10.3389/fnhum.2016.00154] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2016] [Accepted: 03/29/2016] [Indexed: 11/23/2022] Open
Abstract
The auditory cortex is well-known to be critical for music perception, including the perception of consonance and dissonance. Studies on the neural correlates of consonance and dissonance perception have largely employed non-invasive electrophysiological and functional imaging techniques in humans as well as neurophysiological recordings in animals, but the fine-grained spatiotemporal dynamics within the human auditory cortex remain unknown. We recorded electrocorticographic (ECoG) signals directly from the lateral surface of either the left or right temporal lobe of eight patients undergoing neurosurgical treatment as they passively listened to highly consonant and highly dissonant musical chords. We assessed ECoG activity in the high gamma (γhigh, 70–150 Hz) frequency range within the superior temporal gyrus (STG) and observed two types of cortical sites of interest in both hemispheres: one type showed no significant difference in γhigh activity between consonant and dissonant chords, and another type showed increased γhigh responses to dissonant chords between 75 and 200 ms post-stimulus onset. Furthermore, a subset of these sites exhibited additional sensitivity towards different types of dissonant chords, and a positive correlation between changes in γhigh power and the degree of stimulus roughness was observed in both hemispheres. We also observed a distinct spatial organization of cortical sites in the right STG, with dissonant-sensitive sites located anterior to non-sensitive sites. In sum, these findings demonstrate differential processing of consonance and dissonance in bilateral STG with the right hemisphere exhibiting robust and spatially organized sensitivity toward dissonance.
Collapse
Affiliation(s)
- Francine Foo
- Biophysics Graduate Group, University of CaliforniaBerkeley, CA, USA; Helen Wills Neuroscience Institute, University of CaliforniaBerkeley, CA, USA
| | - David King-Stephens
- Department of Neurology and Neurosurgery, California Pacific Medical Center San Francisco, CA, USA
| | - Peter Weber
- Department of Neurology and Neurosurgery, California Pacific Medical Center San Francisco, CA, USA
| | - Kenneth Laxer
- Department of Neurology and Neurosurgery, California Pacific Medical Center San Francisco, CA, USA
| | - Josef Parvizi
- Stanford Human Intracranial Cognitive Electrophysiology Program, Department of Neurology and Neurological Sciences, Stanford University Stanford, CA, USA
| | - Robert T Knight
- Biophysics Graduate Group, University of CaliforniaBerkeley, CA, USA; Helen Wills Neuroscience Institute, University of CaliforniaBerkeley, CA, USA; Department of Psychology, University of CaliforniaBerkeley, CA, USA
| |
Collapse
|
21
|
de Pesters A, Taplin AM, Adamo MA, Ritaccio AL, Schalk G. Electrocorticographic mapping of expressive language function without requiring the patient to speak: A report of three cases. EPILEPSY & BEHAVIOR CASE REPORTS 2016; 6:13-8. [PMID: 27408803 PMCID: PMC4925928 DOI: 10.1016/j.ebcr.2016.02.002] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 02/16/2016] [Revised: 02/25/2016] [Accepted: 02/26/2016] [Indexed: 01/24/2023]
Abstract
Objective Patients requiring resective brain surgery often undergo functional brain mapping during perioperative planning to localize expressive language areas. Currently, all established protocols to perform such mapping require substantial time and patient participation during verb generation or similar tasks. These issues can make language mapping impractical in certain clinical circumstances (e.g., during awake craniotomies) or with certain populations (e.g., pediatric patients). Thus, it is important to develop new techniques that reduce mapping time and the requirement for active patient participation. Several neuroscientific studies reported that the mere auditory presentation of speech stimuli can engage not only receptive but also expressive language areas. Here, we tested the hypothesis that submission of electrocorticographic (ECoG) recordings during a short speech listening task to an appropriate analysis procedure can identify eloquent expressive language cortex without requiring the patient to speak. Methods Three patients undergoing temporary placement of subdural electrode grids passively listened to stories while we recorded their ECoG activity. We identified those sites whose activity in the broadband gamma range (70–170 Hz) changed immediately after presentation of the speech stimuli with respect to a prestimulus baseline. Results Our analyses revealed increased broadband gamma activity at distinct locations in the inferior frontal cortex, superior temporal gyrus, and/or perisylvian areas in all three patients and premotor and/or supplementary motor areas in two patients. The sites in the inferior frontal cortex that we identified with our procedure were either on or immediately adjacent to locations identified using electrical cortical stimulation (ECS) mapping. Conclusions The results of this study provide encouraging preliminary evidence that it may be possible that a brief and practical protocol can identify expressive language areas without requiring the patient to speak. This protocol could provide the clinician with a map of expressive language cortex within a few minutes. This may be useful as an adjunct to ECS interrogation or as an alternative to mapping using functional magnetic resonance imaging (fMRI). In conclusion, with further development and validation in more subjects, the approach presented here could help in identifying expressive language areas in situations where patients cannot speak in response to task instructions.
Collapse
Affiliation(s)
- Adriana de Pesters
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA; Department of Biomedical Sciences, State University of New York at Albany, Albany, NY, USA
| | - AmiLyn M Taplin
- Department of Neurosurgery, Albany Medical College, Albany, NY, USA
| | - Matthew A Adamo
- Department of Neurosurgery, Albany Medical College, Albany, NY, USA
| | | | - Gerwin Schalk
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA; Department of Biomedical Sciences, State University of New York at Albany, Albany, NY, USA; Department of Neurology, Albany Medical College, Albany, NY, USA
| |
Collapse
|
22
|
Liu Y, Coon WG, de Pesters A, Brunner P, Schalk G. The effects of spatial filtering and artifacts on electrocorticographic signals. J Neural Eng 2015; 12. [PMID: 26268446 PMCID: PMC5485665 DOI: 10.1088/1741-2560/12/5/056008 10.1088/1741-2560/12/5/056008] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
Abstract
OBJECTIVE Electrocorticographic (ECoG) signals contain noise that is common to all channels and noise that is specific to individual channels. Most published ECoG studies use common average reference (CAR) spatial filters to remove common noise, but CAR filters may introduce channel-specific noise into other channels. To address this concern, scientists often remove artifactual channels prior to data analysis. However, removing these channels depends on expert-based labeling and may also discard useful data. Thus, the effects of spatial filtering and artifacts on ECoG signals have been largely unknown. This study aims to quantify these effects and thereby address this gap in knowledge. APPROACH In this study, we address these issues by exploring the effects of application of two types of unsupervised spatial filters and three methods of detecting signal artifacts using a large ECoG data set (20 subjects, four task conditions in each subject). MAIN RESULTS Our results confirm that spatial filtering improves performance, i.e., it reduces ECoG signal variance that is not related to the task. They also show that removing artifactual channels automatically (using quantitatively defined rejection criteria) or manually (using expert opinion) does not increase the total amount of task-related information, but does avoid potential contamination from one or more noisy channels. Finally, applying a novel 'median average reference' filter does not require the elimination of artifactual channels prior to spatial filtering and still mitigates the influence of channels with channel-specific noise. Thus, it allows the investigator to retain more potentially useful task-related data. SIGNIFICANCE In summary, our results show that appropriately designed spatial filters that account for both common noise and channel-specific noise greatly improve the quality of ECoG signal analyses, and that artifacts in only a single channel can result in profound and undesired effects on all other channels.
Collapse
Affiliation(s)
- Y Liu
- School of Mechanical Engineering, Beijing Institute of Technology, Beijing, China,National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA
| | - W G Coon
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA,Department of Biomedical Sciences, State University of New York at Albany, Albany, NY, USA
| | - A de Pesters
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA,Department of Biomedical Sciences, State University of New York at Albany, Albany, NY, USA
| | - P Brunner
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA,Department of Neurology, Albany Medical College, Albany, NY, USA
| | - G Schalk
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA,Department of Biomedical Sciences, State University of New York at Albany, Albany, NY, USA,Department of Neurology, Albany Medical College, Albany, NY, USA
| |
Collapse
|
23
|
Dijkstra K, Brunner P, Gunduz A, Coon W, Ritaccio A, Farquhar J, Schalk G. Identifying the Attended Speaker Using Electrocorticographic (ECoG) Signals. BRAIN-COMPUTER INTERFACES 2015; 2:161-173. [PMID: 26949710 PMCID: PMC4776341 DOI: 10.1080/2326263x.2015.1063363] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
People affected by severe neuro-degenerative diseases (e.g., late-stage amyotrophic lateral sclerosis (ALS) or locked-in syndrome) eventually lose all muscular control. Thus, they cannot use traditional assistive communication devices that depend on muscle control, or brain-computer interfaces (BCIs) that depend on the ability to control gaze. While auditory and tactile BCIs can provide communication to such individuals, their use typically entails an artificial mapping between the stimulus and the communication intent. This makes these BCIs difficult to learn and use. In this study, we investigated the use of selective auditory attention to natural speech as an avenue for BCI communication. In this approach, the user communicates by directing his/her attention to one of two simultaneously presented speakers. We used electrocorticographic (ECoG) signals in the gamma band (70-170 Hz) to infer the identity of attended speaker, thereby removing the need to learn such an artificial mapping. Our results from twelve human subjects show that a single cortical location over superior temporal gyrus or pre-motor cortex is typically sufficient to identify the attended speaker within 10 s and with 77% accuracy (50% accuracy due to chance). These results lay the groundwork for future studies that may determine the real-time performance of BCIs based on selective auditory attention to speech.
Collapse
Affiliation(s)
- K. Dijkstra
- Ctr for Adapt Neurotech, Wadsworth Center, New York State Department of Health, Albany, NY
- Dept of Neurology, Albany Medical College, Albany, NY
- Donders Inst for Brain, Cognition and Behaviour, Radboud Univ Nijmegen, The Netherlands
| | - P. Brunner
- Ctr for Adapt Neurotech, Wadsworth Center, New York State Department of Health, Albany, NY
- Dept of Neurology, Albany Medical College, Albany, NY
| | - A. Gunduz
- Ctr for Adapt Neurotech, Wadsworth Center, New York State Department of Health, Albany, NY
- J. Crayton Pruitt Family Dept of Biomed Eng, Univ of Florida, Gainesville, FL
| | - W. Coon
- Ctr for Adapt Neurotech, Wadsworth Center, New York State Department of Health, Albany, NY
- Dept of Biomed Sci, State Univ of New York at Albany, Albany, NY
| | - A.L. Ritaccio
- Dept of Neurology, Albany Medical College, Albany, NY
| | - J. Farquhar
- Donders Inst for Brain, Cognition and Behaviour, Radboud Univ Nijmegen, The Netherlands
| | - G. Schalk
- Ctr for Adapt Neurotech, Wadsworth Center, New York State Department of Health, Albany, NY
- Dept of Neurology, Albany Medical College, Albany, NY
- Dept of Biomed Sci, State Univ of New York at Albany, Albany, NY
| |
Collapse
|
24
|
Liu Y, Coon WG, de Pesters A, Brunner P, Schalk G. The effects of spatial filtering and artifacts on electrocorticographic signals. J Neural Eng 2015; 12:056008. [PMID: 26268446 DOI: 10.1088/1741-2560/12/5/056008] [Citation(s) in RCA: 32] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Electrocorticographic (ECoG) signals contain noise that is common to all channels and noise that is specific to individual channels. Most published ECoG studies use common average reference (CAR) spatial filters to remove common noise, but CAR filters may introduce channel-specific noise into other channels. To address this concern, scientists often remove artifactual channels prior to data analysis. However, removing these channels depends on expert-based labeling and may also discard useful data. Thus, the effects of spatial filtering and artifacts on ECoG signals have been largely unknown. This study aims to quantify these effects and thereby address this gap in knowledge. APPROACH In this study, we address these issues by exploring the effects of application of two types of unsupervised spatial filters and three methods of detecting signal artifacts using a large ECoG data set (20 subjects, four task conditions in each subject). MAIN RESULTS Our results confirm that spatial filtering improves performance, i.e., it reduces ECoG signal variance that is not related to the task. They also show that removing artifactual channels automatically (using quantitatively defined rejection criteria) or manually (using expert opinion) does not increase the total amount of task-related information, but does avoid potential contamination from one or more noisy channels. Finally, applying a novel 'median average reference' filter does not require the elimination of artifactual channels prior to spatial filtering and still mitigates the influence of channels with channel-specific noise. Thus, it allows the investigator to retain more potentially useful task-related data. SIGNIFICANCE In summary, our results show that appropriately designed spatial filters that account for both common noise and channel-specific noise greatly improve the quality of ECoG signal analyses, and that artifacts in only a single channel can result in profound and undesired effects on all other channels.
Collapse
Affiliation(s)
- Y Liu
- School of Mechanical Engineering, Beijing Institute of Technology, Beijing, People's Republic of China. National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health, Albany, NY, USA
| | | | | | | | | |
Collapse
|
25
|
Herff C, Heger D, de Pesters A, Telaar D, Brunner P, Schalk G, Schultz T. Brain-to-text: decoding spoken phrases from phone representations in the brain. Front Neurosci 2015; 9:217. [PMID: 26124702 PMCID: PMC4464168 DOI: 10.3389/fnins.2015.00217] [Citation(s) in RCA: 144] [Impact Index Per Article: 16.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2015] [Accepted: 05/18/2015] [Indexed: 11/24/2022] Open
Abstract
It has long been speculated whether communication between humans and machines based on natural speech related cortical activity is possible. Over the past decade, studies have suggested that it is feasible to recognize isolated aspects of speech from neural signals, such as auditory features, phones or one of a few isolated words. However, until now it remained an unsolved challenge to decode continuously spoken speech from the neural substrate associated with speech and language processing. Here, we show for the first time that continuously spoken speech can be decoded into the expressed words from intracranial electrocorticographic (ECoG) recordings.Specifically, we implemented a system, which we call Brain-To-Text that models single phones, employs techniques from automatic speech recognition (ASR), and thereby transforms brain activity while speaking into the corresponding textual representation. Our results demonstrate that our system can achieve word error rates as low as 25% and phone error rates below 50%. Additionally, our approach contributes to the current understanding of the neural basis of continuous speech production by identifying those cortical regions that hold substantial information about individual phones. In conclusion, the Brain-To-Text system described in this paper represents an important step toward human-machine communication based on imagined speech.
Collapse
Affiliation(s)
- Christian Herff
- Cognitive Systems Lab, Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology Karlsruhe, Germany
| | - Dominic Heger
- Cognitive Systems Lab, Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology Karlsruhe, Germany
| | - Adriana de Pesters
- New York State Department of Health, National Center for Adaptive Neurotechnologies, Wadsworth Center Albany, NY, USA ; Department of Biomedical Sciences, State University of New York at Albany Albany, NY, USA
| | - Dominic Telaar
- Cognitive Systems Lab, Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology Karlsruhe, Germany
| | - Peter Brunner
- New York State Department of Health, National Center for Adaptive Neurotechnologies, Wadsworth Center Albany, NY, USA ; Department of Neurology, Albany Medical College Albany, NY, USA
| | - Gerwin Schalk
- New York State Department of Health, National Center for Adaptive Neurotechnologies, Wadsworth Center Albany, NY, USA ; Department of Biomedical Sciences, State University of New York at Albany Albany, NY, USA ; Department of Neurology, Albany Medical College Albany, NY, USA
| | - Tanja Schultz
- Cognitive Systems Lab, Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology Karlsruhe, Germany
| |
Collapse
|
26
|
Lotte F, Brumberg JS, Brunner P, Gunduz A, Ritaccio AL, Guan C, Schalk G. Electrocorticographic representations of segmental features in continuous speech. Front Hum Neurosci 2015; 9:97. [PMID: 25759647 PMCID: PMC4338752 DOI: 10.3389/fnhum.2015.00097] [Citation(s) in RCA: 57] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/18/2014] [Accepted: 02/06/2015] [Indexed: 11/25/2022] Open
Abstract
Acoustic speech output results from coordinated articulation of dozens of muscles, bones and cartilages of the vocal mechanism. While we commonly take the fluency and speed of our speech productions for granted, the neural mechanisms facilitating the requisite muscular control are not completely understood. Previous neuroimaging and electrophysiology studies of speech sensorimotor control has typically concentrated on speech sounds (i.e., phonemes, syllables and words) in isolation; sentence-length investigations have largely been used to inform coincident linguistic processing. In this study, we examined the neural representations of segmental features (place and manner of articulation, and voicing status) in the context of fluent, continuous speech production. We used recordings from the cortical surface [electrocorticography (ECoG)] to simultaneously evaluate the spatial topography and temporal dynamics of the neural correlates of speech articulation that may mediate the generation of hypothesized gestural or articulatory scores. We found that the representation of place of articulation involved broad networks of brain regions during all phases of speech production: preparation, execution and monitoring. In contrast, manner of articulation and voicing status were dominated by auditory cortical responses after speech had been initiated. These results provide a new insight into the articulatory and auditory processes underlying speech production in terms of their motor requirements and acoustic correlates.
Collapse
Affiliation(s)
| | - Jonathan S Brumberg
- Department of Speech-Language-Hearing, University of Kansas Lawrence, KS, USA
| | - Peter Brunner
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health Albany, NY, USA ; Department of Neurology, Albany Medical College Albany, NY, USA
| | - Aysegul Gunduz
- J. Crayton Pruitt Family Department of Biomedical Engineering, University of Florida Gainesville, FL, USA
| | | | - Cuntai Guan
- ASTAR Agency for Science, Technology and Research, Institute for Infocomm Research, Singapore Singapore
| | - Gerwin Schalk
- National Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health Albany, NY, USA ; Department of Neurology, Albany Medical College Albany, NY, USA
| |
Collapse
|
27
|
Sturm I, Blankertz B, Potes C, Schalk G, Curio G. ECoG high gamma activity reveals distinct cortical representations of lyrics passages, harmonic and timbre-related changes in a rock song. Front Hum Neurosci 2014; 8:798. [PMID: 25352799 PMCID: PMC4195312 DOI: 10.3389/fnhum.2014.00798] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/22/2014] [Accepted: 09/19/2014] [Indexed: 11/13/2022] Open
Abstract
Listening to music moves our minds and moods, stirring interest in its neural underpinnings. A multitude of compositional features drives the appeal of natural music. How such original music, where a composer's opus is not manipulated for experimental purposes, engages a listener's brain has not been studied until recently. Here, we report an in-depth analysis of two electrocorticographic (ECoG) data sets obtained over the left hemisphere in ten patients during presentation of either a rock song or a read-out narrative. First, the time courses of five acoustic features (intensity, presence/absence of vocals with lyrics, spectral centroid, harmonic change, and pulse clarity) were extracted from the audio tracks and found to be correlated with each other to varying degrees. In a second step, we uncovered the specific impact of each musical feature on ECoG high-gamma power (70-170 Hz) by calculating partial correlations to remove the influence of the other four features. In the music condition, the onset and offset of vocal lyrics in ongoing instrumental music was consistently identified within the group as the dominant driver for ECoG high-gamma power changes over temporal auditory areas, while concurrently subject-individual activation spots were identified for sound intensity, timbral, and harmonic features. The distinct cortical activations to vocal speech-related content embedded in instrumental music directly demonstrate that song integrated in instrumental music represents a distinct dimension in complex music. In contrast, in the speech condition, the full sound envelope was reflected in the high gamma response rather than the onset or offset of the vocal lyrics. This demonstrates how the contributions of stimulus features that modulate the brain response differ across the two examples of a full-length natural stimulus, which suggests a context-dependent feature selection in the processing of complex auditory stimuli.
Collapse
Affiliation(s)
- Irene Sturm
- Berlin School of Mind and Brain, Humboldt Universität zu Berlin Berlin, Germany ; Neurotechnology Group, Department of Electrical Engineering and Computer Science, Berlin Institute of Technology Berlin, Germany ; Neurophysics Group, Department of Neurology and Clinical Neurophysiology, Charité - University Medicine Berlin Berlin, Germany
| | - Benjamin Blankertz
- Neurotechnology Group, Department of Electrical Engineering and Computer Science, Berlin Institute of Technology Berlin, Germany ; Bernstein Focus: Neurotechnology Berlin, Germany
| | - Cristhian Potes
- National Resource Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health Albany, NY, USA ; Department of Electrical and Computer Engineering, University of Texas at El Paso El Paso, TX, USA
| | - Gerwin Schalk
- National Resource Center for Adaptive Neurotechnologies, Wadsworth Center, New York State Department of Health Albany, NY, USA ; Department of Electrical and Computer Engineering, University of Texas at El Paso El Paso, TX, USA ; Department of Neurosurgery, Washington University in St. Louis St. Louis, MO, USA ; Department of Biomedical Engineering, Rensselaer Polytechnic Institute Troy, NY, USA ; Department of Neurology, Albany Medical College Albany, NY, USA ; Department of Neurosurgery, Washington University in St. Louis St. Louis, MO, USA
| | - Gabriel Curio
- Berlin School of Mind and Brain, Humboldt Universität zu Berlin Berlin, Germany ; Neurophysics Group, Department of Neurology and Clinical Neurophysiology, Charité - University Medicine Berlin Berlin, Germany ; Bernstein Focus: Neurotechnology Berlin, Germany
| |
Collapse
|
28
|
Potes C, Brunner P, Gunduz A, Knight RT, Schalk G. Spatial and temporal relationships of electrocorticographic alpha and gamma activity during auditory processing. Neuroimage 2014; 97:188-95. [PMID: 24768933 PMCID: PMC4065821 DOI: 10.1016/j.neuroimage.2014.04.045] [Citation(s) in RCA: 65] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/17/2013] [Revised: 03/22/2014] [Accepted: 04/13/2014] [Indexed: 11/24/2022] Open
Abstract
Neuroimaging approaches have implicated multiple brain sites in musical perception, including the posterior part of the superior temporal gyrus and adjacent perisylvian areas. However, the detailed spatial and temporal relationship of neural signals that support auditory processing is largely unknown. In this study, we applied a novel inter-subject analysis approach to electrophysiological signals recorded from the surface of the brain (electrocorticography (ECoG)) in ten human subjects. This approach allowed us to reliably identify those ECoG features that were related to the processing of a complex auditory stimulus (i.e., continuous piece of music) and to investigate their spatial, temporal, and causal relationships. Our results identified stimulus-related modulations in the alpha (8-12 Hz) and high gamma (70-110 Hz) bands at neuroanatomical locations implicated in auditory processing. Specifically, we identified stimulus-related ECoG modulations in the alpha band in areas adjacent to primary auditory cortex, which are known to receive afferent auditory projections from the thalamus (80 of a total of 15,107 tested sites). In contrast, we identified stimulus-related ECoG modulations in the high gamma band not only in areas close to primary auditory cortex but also in other perisylvian areas known to be involved in higher-order auditory processing, and in superior premotor cortex (412/15,107 sites). Across all implicated areas, modulations in the high gamma band preceded those in the alpha band by 280 ms, and activity in the high gamma band causally predicted alpha activity, but not vice versa (Granger causality, p<1e(-8)). Additionally, detailed analyses using Granger causality identified causal relationships of high gamma activity between distinct locations in early auditory pathways within superior temporal gyrus (STG) and posterior STG, between posterior STG and inferior frontal cortex, and between STG and premotor cortex. Evidence suggests that these relationships reflect direct cortico-cortical connections rather than common driving input from subcortical structures such as the thalamus. In summary, our inter-subject analyses defined the spatial and temporal relationships between music-related brain activity in the alpha and high gamma bands. They provide experimental evidence supporting current theories about the putative mechanisms of alpha and gamma activity, i.e., reflections of thalamo-cortical interactions and local cortical neural activity, respectively, and the results are also in agreement with existing functional models of auditory processing.
Collapse
Affiliation(s)
- Cristhian Potes
- BCI R&D Program, Wadsworth Center, New York State Department of Health, Albany, NY, USA; Department of Electrical and Computer Engineering, University of Texas at El Paso, TX, USA.
| | - Peter Brunner
- BCI R&D Program, Wadsworth Center, New York State Department of Health, Albany, NY, USA; Department of Neurology, Albany Medical College, Albany, NY, USA; Department of Computer Science, Graz University of Technology, Graz, Austria.
| | - Aysegul Gunduz
- BCI R&D Program, Wadsworth Center, New York State Department of Health, Albany, NY, USA; Department of Neurology, Albany Medical College, Albany, NY, USA; J. Crayton Pruitt Family, Department of Biomedical Engineering, University of Florida, Gainesville, FL, USA.
| | - Robert T Knight
- Department of Psychology, University of California at Berkeley, CA, USA.
| | - Gerwin Schalk
- BCI R&D Program, Wadsworth Center, New York State Department of Health, Albany, NY, USA; Department of Electrical and Computer Engineering, University of Texas at El Paso, TX, USA; Department of Neurology, Albany Medical College, Albany, NY, USA; Department of Biomedical Science, State University of NY at Albany, Albany, NY, USA.
| |
Collapse
|
29
|
Gupta D, Jeremy Hill N, Brunner P, Gunduz A, Ritaccio AL, Schalk G. Simultaneous real-time monitoring of multiple cortical systems. J Neural Eng 2014; 11:056001. [PMID: 25080161 DOI: 10.1088/1741-2560/11/5/056001] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022]
Abstract
OBJECTIVE Real-time monitoring of the brain is potentially valuable for performance monitoring, communication, training or rehabilitation. In natural situations, the brain performs a complex mix of various sensory, motor or cognitive functions. Thus, real-time brain monitoring would be most valuable if (a) it could decode information from multiple brain systems simultaneously, and (b) this decoding of each brain system were robust to variations in the activity of other (unrelated) brain systems. Previous studies showed that it is possible to decode some information from different brain systems in retrospect and/or in isolation. In our study, we set out to determine whether it is possible to simultaneously decode important information about a user from different brain systems in real time, and to evaluate the impact of concurrent activity in different brain systems on decoding performance. APPROACH We study these questions using electrocorticographic signals recorded in humans. We first document procedures for generating stable decoding models given little training data, and then report their use for offline and for real-time decoding from 12 subjects (six for offline parameter optimization, six for online experimentation). The subjects engage in tasks that involve movement intention, movement execution and auditory functions, separately, and then simultaneously. Main Results: Our real-time results demonstrate that our system can identify intention and movement periods in single trials with an accuracy of 80.4% and 86.8%, respectively (where 50% would be expected by chance). Simultaneously, the decoding of the power envelope of an auditory stimulus resulted in an average correlation coefficient of 0.37 between the actual and decoded power envelopes. These decoders were trained separately and executed simultaneously in real time. SIGNIFICANCE This study yielded the first demonstration that it is possible to decode simultaneously the functional activity of multiple independent brain systems. Our comparison of univariate and multivariate decoding strategies, and our analysis of the influence of their decoding parameters, provides benchmarks and guidelines for future research on this topic.
Collapse
Affiliation(s)
- Disha Gupta
- Wadsworth Center, New York State Department of Health, Albany, NY, USA. Department of Neurology, Albany Medical College, Albany, NY, USA. Early Brain Injury Recovery Program, Burke-Cornell Medical Research Institute, White Plains, NY, USA
| | | | | | | | | | | |
Collapse
|
30
|
Ritaccio A, Beauchamp M, Bosman C, Brunner P, Chang E, Crone N, Gunduz A, Gupta D, Knight R, Leuthardt E, Litt B, Moran D, Ojemann J, Parvizi J, Ramsey N, Rieger J, Viventi J, Voytek B, Williams J, Schalk G. Proceedings of the Third International Workshop on Advances in Electrocorticography. Epilepsy Behav 2012; 25:605-13. [PMID: 23160096 PMCID: PMC4041796 DOI: 10.1016/j.yebeh.2012.09.016] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/06/2012] [Accepted: 09/08/2012] [Indexed: 10/27/2022]
Abstract
The Third International Workshop on Advances in Electrocorticography (ECoG) was convened in Washington, DC, on November 10-11, 2011. As in prior meetings, a true multidisciplinary fusion of clinicians, scientists, and engineers from many disciplines gathered to summarize contemporary experiences in brain surface recordings. The proceedings of this meeting serve as evidence of a very robust and transformative field but will yet again require revision to incorporate the advances that the following year will surely bring.
Collapse
Affiliation(s)
| | | | | | - Peter Brunner
- Albany Medical College, Albany, NY, USA, Wadsworth Center, New York State Department of Health, Albany, NY, USA
| | - Edward Chang
- University of California, San Francisco, CA, USA
| | - Nathan Crone
- Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Aysegul Gunduz
- Albany Medical College, Albany, NY, USA, Wadsworth Center, New York State Department of Health, Albany, NY, USA
| | - Disha Gupta
- Albany Medical College, Albany, NY, USA, Wadsworth Center, New York State Department of Health, Albany, NY, USA
| | - Robert Knight
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA
| | | | - Brian Litt
- University of Pennsylvania, Pittsburgh, PA, USA
| | | | | | | | - Nick Ramsey
- University Medical Center, Utrecht University, Utrecht, The Netherlands
| | - Jochem Rieger
- Helen Wills Neuroscience Institute, University of California, Berkeley, CA, USA, University of Oldenburg, Oldenburg, Germany
| | | | | | | | - Gerwin Schalk
- Albany Medical College, Albany, NY, USA, Wadsworth Center, New York State Department of Health, Albany, NY, USA
| |
Collapse
|