1
|
Gao M, Zhu W, Drewes J. The temporal dynamics of conscious and unconscious audio-visual semantic integration. Heliyon 2024; 10:e33828. [PMID: 39055801 PMCID: PMC11269866 DOI: 10.1016/j.heliyon.2024.e33828] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2023] [Revised: 06/11/2024] [Accepted: 06/27/2024] [Indexed: 07/28/2024] Open
Abstract
We compared the time course of cross-modal semantic effects induced by both naturalistic sounds and spoken words on the processing of visual stimuli, whether visible or suppressed form awareness through continuous flash suppression. We found that, under visible conditions, spoken words elicited audio-visual semantic effects over longer time (-1000, -500, -250 ms SOAs) than naturalistic sounds (-500, -250 ms SOAs). Performance was generally better with auditory primes, but more so with congruent stimuli. Spoken words presented in advance (-1000, -500 ms) outperformed naturalistic sounds; the opposite was true for (near-)simultaneous presentations. Congruent spoken words demonstrated superior categorization performance compared to congruent naturalistic sounds. The audio-visual semantic congruency effect still occurred with suppressed visual stimuli, although without significant variations in the temporal patterns between auditory types. These findings indicate that: 1. Semantically congruent auditory input can enhance visual processing performance, even when the visual stimulus is imperceptible to conscious awareness. 2. The temporal dynamics is contingent on the auditory types only when the visual stimulus is visible. 3. Audiovisual semantic integration requires sufficient time for processing auditory information.
Collapse
Affiliation(s)
- Mingjie Gao
- School of Information Science, Yunnan University, Kunming, China
| | - Weina Zhu
- School of Information Science, Yunnan University, Kunming, China
| | - Jan Drewes
- Institute of Brain and Psychological Sciences, Sichuan Normal University, Chengdu, China
| |
Collapse
|
2
|
Basma B, Savage R, Bertone A. The N400 in readers with dyslexia: A systematic review and meta-analysis. Int J Psychophysiol 2024; 196:112283. [PMID: 38128616 DOI: 10.1016/j.ijpsycho.2023.112283] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Revised: 12/11/2023] [Accepted: 12/14/2023] [Indexed: 12/23/2023]
Abstract
This systematic review and meta-analysis aimed to assess whether (i) significant differences exist in the N400 response to lexico-semantic tasks between typically developing (TD) readers and readers with dyslexia, and (ii) whether these differences are moderated by the modality of task presentation (visual vs. auditory), the type of task, age, or opaque orthography (shallow and transparent alphabets vs Chinese morpho-syllabary). Twenty studies were included in the meta-analysis, and the analysis did not demonstrate strong evidence of publication bias. An overall effect size of Hedge's g = 0.66, p < .001, was found between typically developing readers and readers with dyslexia. All moderators were found to be significant; larger effects were associated with visual modality (g = 0.692, p < .001), semantically incongruent sentence tasks (g = 0.948, p < .001), pseudowords/characters tasks (g = 0.971, p < .001), and orthography [Chinese (g = 1.015, p < .001) vs. alphabets (g = 0.539, p < .001)]. Analysis of reaction time showed Hedge's g = 1.613, p < .001. Results suggest that the N400 reliably differentiated between typically developing readers and readers with dyslexia. Implications for future research and practice are discussed.
Collapse
Affiliation(s)
- Badriah Basma
- Department of Educational and Counselling Psychology, McGill University, Montreal, Quebec, Canada.
| | - Robert Savage
- Department of Education, York University, Toronto, Canada.
| | - Armando Bertone
- Department of Educational and Counselling Psychology, McGill University, Montreal, Quebec, Canada.
| |
Collapse
|
3
|
Manfredi M, Boggio PS. Neural correlates of sex differences in communicative gestures and speech comprehension: A preliminary study. Soc Neurosci 2021; 16:653-667. [PMID: 34697990 DOI: 10.1080/17470919.2021.1997800] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
Abstract
The goal of this study was to investigate whether the semantic processing of the audiovisual combination of communicative gestures with speech differs between men and women. We recorded event-related brain potentials in women and men during the presentation of communicative gestures that were either congruent or incongruent with the speech.Our results showed that incongruent gestures elicited an N400 effect over frontal sites compared to congruent ones in both groups. Moreover, the females showed an earlier N2 response to incongruent stimuli than congruent ones, while larger sustained negativity and late positivity in response to incongruent stimuli was observed only in males. These results suggest that women rapidly recognize and process audiovisual combinations of communicative gestures and speech (as early as 300 ms) whereas men analyze them at the later stages of the process.
Collapse
Affiliation(s)
- Mirella Manfredi
- Department of Psychology, University of Zurich, Zurich, Switzerland
| | - Paulo Sergio Boggio
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil
| |
Collapse
|
4
|
Manfredi M, Sanchez Mello de Pinho P, Murrins Marques L, de Oliveira Ribeiro B, Boggio PS. Crossmodal processing of environmental sounds and everyday life actions: An ERP study. Heliyon 2021; 7:e07937. [PMID: 34541349 PMCID: PMC8436072 DOI: 10.1016/j.heliyon.2021.e07937] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2021] [Revised: 07/06/2021] [Accepted: 09/01/2021] [Indexed: 12/03/2022] Open
Abstract
To investigate the processing of environmental sounds, previous researchers have compared the semantic processing of words and sounds, yielding mixed results. This study aimed to specifically investigate the electrophysiological mechanism underlying the semantic processing of environmental sounds presented in a naturalistic visual scene. We recorded event-related brain potentials in a group of young adults over the presentation of everyday life actions that were either congruent or incongruent with environmental sounds. Our results showed that incongruent environmental sounds evoked both a P400 and an N400 effect, reflecting sensitivity to physical and semantic violations of environmental sounds’ properties, respectively. In addition, our findings showed an enhanced late positivity in response to incongruous environmental sounds, probably reflecting additional reanalysis costs. In conclusion, these results indicate that the crossmodal processing of the environmental sounds might require the simultaneous involvement of different cognitive processes.
Collapse
Affiliation(s)
- Mirella Manfredi
- Department of Psychology, University of Zurich, Zurich, Switzerland
- Corresponding author.
| | - Pamella Sanchez Mello de Pinho
- Social and Cognitive Neuroscience Laboratory, Developmental Disorders Program, Center for Health and Biological Sciences, Mackenzie Presbyterian University, Sao Paulo, Brazil
| | - Lucas Murrins Marques
- Social and Cognitive Neuroscience Laboratory, Developmental Disorders Program, Center for Health and Biological Sciences, Mackenzie Presbyterian University, Sao Paulo, Brazil
| | - Beatriz de Oliveira Ribeiro
- Social and Cognitive Neuroscience Laboratory, Developmental Disorders Program, Center for Health and Biological Sciences, Mackenzie Presbyterian University, Sao Paulo, Brazil
| | - Paulo Sergio Boggio
- Social and Cognitive Neuroscience Laboratory, Developmental Disorders Program, Center for Health and Biological Sciences, Mackenzie Presbyterian University, Sao Paulo, Brazil
- Corresponding author.
| |
Collapse
|
5
|
Manfredi M, Cohn N, Ribeiro B, Sanchez Pinho P, Fernandes Rodrigues Pereira E, Boggio PS. The electrophysiology of audiovisual processing in visual narratives in adolescents with autism spectrum disorder. Brain Cogn 2021; 151:105730. [PMID: 33892434 DOI: 10.1016/j.bandc.2021.105730] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Revised: 02/15/2021] [Accepted: 04/03/2021] [Indexed: 12/24/2022]
Abstract
We investigated the semantic processing of the multimodal audiovisual combination of visual narratives with auditory descriptive words and auditory sounds in individuals with ASD. To this aim, we recorded ERPs to critical auditory words and sounds associated with events in visual narrative that were either semantically congruent or incongruent with the climactic visual event. A similar N400 effect was found both in adolescents with ASD and neurotypical adolescents (ages 9-16) when accessing different types of auditory information (i.e. words and sounds) into a visual narrative. This result might suggest that verbal information processing in ASD adolescents could be facilitated by direct association with meaningful visual information. In addition, we observed differences in scalp distribution of later brain responses between ASD and neurotypical adolescents. This finding might suggest ASD adolescents differ from neurotypical adolescents during the processing of the multimodal combination of visual narratives with auditory information at later stages of the process. In conclusion, the semantic processing of verbal information, typically impaired in individuals with ASD, can be facilitated when embedded into a meaningful visual information.
Collapse
Affiliation(s)
- Mirella Manfredi
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil; Department of Psychology, University of Zurich, Zurich, Switzerland.
| | - Neil Cohn
- Department of Communication and Cognition, Tilburg University, Tilburg, Netherlands
| | - Beatriz Ribeiro
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil
| | - Pamella Sanchez Pinho
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil
| | | | - Paulo Sergio Boggio
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil
| |
Collapse
|
6
|
Multimodal feature binding in object memory retrieval using event-related potentials: Implications for models of semantic memory. Int J Psychophysiol 2020; 153:116-126. [PMID: 32389620 DOI: 10.1016/j.ijpsycho.2020.04.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/16/2020] [Revised: 03/30/2020] [Accepted: 04/29/2020] [Indexed: 11/23/2022]
Abstract
To test the hypothesis that semantic processes are represented in multiple subsystems, we recorded electroencephalogram (EEG) as we elicited object memories using the modified Semantic Object Retrieval Test, during which an object feature, presented as a visual word [VW], an auditory word [AW], or a picture [Pic], was followed by a second feature always presented as a visual word. We performed both hypothesis-driven and data-driven analyses using event-related potentials (ERPs) time locked to the second stimulus. We replicated a previously reported left fronto-temporal ERP effect (750-1000 ms post-stimulus) in the VW task, and also found that this ERP component was only present during object memory retrieval in verbal (VW, AW) as opposed to non-verbal (Pic) stimulus types. We also found a right temporal ERP effect (850-1000 ms post-stimulus) that was present in auditory (AW) but not in visual (VW, Pic) stimulus types. In addition, we found an earlier left temporo-parietal ERP effect between 350 and 700 ms post-stimulus and a later midline parietal ERP effect between 700 and 1100 ms post-stimulus, present in all stimulus types, suggesting common neural mechanisms for object retrieval processes and object activation, respectively. These findings support multiple semantic subsystems that respond to varying stimulus modalities, and argue against an ultimate unitary amodal semantic analysis.
Collapse
|
7
|
Pitt KM, Brumberg JS, Burnison JD, Mehta J, Kidwai J. Behind the Scenes of Noninvasive Brain-Computer Interfaces: A Review of Electroencephalography Signals, How They Are Recorded, and Why They Matter. ACTA ACUST UNITED AC 2019; 4:1622-1636. [PMID: 32529035 DOI: 10.1044/2019_pers-19-00059] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023]
Abstract
Purpose Brain-computer interface (BCI) techniques may provide computer access for individuals with severe physical impairments. However, the relatively hidden nature of BCI control obscures how BCI systems work behind the scenes, making it difficult to understand how electroencephalography (EEG) records the BCI related brain signals, what brain signals are recorded by EEG, and why these signals are targeted for BCI control. Furthermore, in the field of speech-language-hearing, signals targeted for BCI application have been of primary interest to clinicians and researchers in the area of augmentative and alternative communication (AAC). However, signals utilized for BCI control reflect sensory, cognitive and motor processes, which are of interest to a range of related disciplines including speech science. Method This tutorial was developed by a multidisciplinary team emphasizing primary and secondary BCI-AAC related signals of interest to speech-language-hearing. Results An overview of BCI-AAC related signals are provided discussing 1) how BCI signals are recorded via EEG, 2) what signals are targeted for non-invasive BCI control, including the P300, sensorimotor rhythms, steady state evoked potentials, contingent negative variation, and the N400, and 3) why these signals are targeted. During tutorial creation, attention was given to help support EEG and BCI understanding for those without an engineering background. Conclusion Tutorials highlighting how BCI-AAC signals are elicited and recorded can help increase interest and familiarity with EEG and BCI techniques and provide a framework for understanding key principles behind BCI-AAC design and implementation.
Collapse
Affiliation(s)
- Kevin M Pitt
- Department of Special Education and Communication Disorders, University of Nebraska-Lincoln, Lincoln, NE
| | - Jonathan S Brumberg
- Department of Speech-Language-Hearing: Sciences & Disorders, University of Kansas, Lawrence, KS
| | | | - Jyutika Mehta
- Department of Communication Sciences & Disorders, Texas Woman's University, Denton, TX
| | - Juhi Kidwai
- Department of Speech-Language-Hearing: Sciences & Disorders, University of Kansas, Lawrence, KS
| |
Collapse
|
8
|
Hendrickson K, Love T, Walenski M, Friend M. The organization of words and environmental sounds in the second year: Behavioral and electrophysiological evidence. Dev Sci 2019; 22:e12746. [PMID: 30159958 PMCID: PMC6294716 DOI: 10.1111/desc.12746] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/20/2016] [Accepted: 05/18/2018] [Indexed: 11/30/2022]
Abstract
The majority of research examining early auditory-semantic processing and organization is based on studies of meaningful relations between words and referents. However, a thorough investigation into the fundamental relation between acoustic signals and meaning requires an understanding of how meaning is associated with both lexical and non-lexical sounds. Indeed, it is unknown how meaningful auditory information that is not lexical (e.g., environmental sounds) is processed and organized in the young brain. To capture the structure of semantic organization for words and environmental sounds, we record event-related potentials as 20-month-olds view images of common nouns (e.g., dog) while hearing words or environmental sounds that match the picture (e.g., "dog" or barking), that are within-category violations (e.g., "cat" or meowing), or that are between-category violations (e.g., "pen" or scribbling). Results show both words and environmental sounds exhibit larger negative amplitudes to between-category violations relative to matches. Unlike words, which show a greater negative response early and consistently to within-category violations, such an effect for environmental sounds occurs late in semantic processing. Thus, as in adults, the young brain represents semantic relations between words and between environmental sounds, though it more readily differentiates semantically similar words compared to environmental sounds.
Collapse
Affiliation(s)
- Kristi Hendrickson
- Department of Communication Sciences & Disorders, University of Iowa, USA
| | - Tracy Love
- Center for Research in Language, University of California, San Diego, USA
- School of Speech, Language, and Hearing Sciences, San Diego State University, USA
| | | | | |
Collapse
|
9
|
Manfredi M, Cohn N, De Araújo Andreoli M, Boggio PS. Listening beyond seeing: Event-related potentials to audiovisual processing in visual narrative. BRAIN AND LANGUAGE 2018; 185:1-8. [PMID: 29986168 DOI: 10.1016/j.bandl.2018.06.008] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/08/2017] [Revised: 06/28/2018] [Accepted: 06/28/2018] [Indexed: 06/08/2023]
Abstract
Every day we integrate meaningful information coming from different sensory modalities, and previous work has debated whether conceptual knowledge is represented in modality-specific neural stores specialized for specific types of information, and/or in an amodal, shared system. In the current study, we investigated semantic processing through a cross-modal paradigm which asked whether auditory semantic processing could be modulated by the constraints of context built up across a meaningful visual narrative sequence. We recorded event-related brain potentials (ERPs) to auditory words and sounds associated to events in visual narratives-i.e., seeing images of someone spitting while hearing either a word (Spitting!) or a sound (the sound of spitting)-which were either semantically congruent or incongruent with the climactic visual event. Our results showed that both incongruent sounds and words evoked an N400 effect, however, the distribution of the N400 effect to words (centro-parietal) differed from that of sounds (frontal). In addition, words had an earlier latency N400 than sounds. Despite these differences, a sustained late frontal negativity followed the N400s and did not differ between modalities. These results support the idea that semantic memory balances a distributed cortical network accessible from multiple modalities, yet also engages amodal processing insensitive to specific modalities.
Collapse
Affiliation(s)
- Mirella Manfredi
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil.
| | - Neil Cohn
- Tilburg Center for Cognition and Communication, Tilburg University, Tilburg, Netherlands
| | - Mariana De Araújo Andreoli
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil
| | - Paulo Sergio Boggio
- Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil
| |
Collapse
|
10
|
Haebig E, Weber C, Leonard LB, Deevy P, Tomblin JB. Neural patterns elicited by sentence processing uniquely characterize typical development, SLI recovery, and SLI persistence. J Neurodev Disord 2017. [PMID: 28630655 PMCID: PMC5470275 DOI: 10.1186/s11689-017-9201-1] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Background A substantial amount of work has examined language abilities in young children with specific language impairment (SLI); however, our understanding of the developmental trajectory of language impairment is limited. Along with studying the behavioral changes that occur across development, it is important to examine the neural indices of language processing for children with different language trajectories. The current study sought to examine behavioral and neural bases of language processing in adolescents showing three different trajectories: those with normal language development (NL), those exhibiting persistent SLI (SLI-Persistent), and those with a history of SLI who appear to have recovered (SLI-Recovered). Methods Through a sentence judgment task, we examined semantic and syntactic processing. Adolescents judged whether or not each sentence was semantically and syntactically correct. Stimuli consisted of naturally spoken sentences that were either correct, contained a semantic verb error, or contained a syntactic verb agreement error. Verb agreement errors consisted of omission and commission violations of the third-person singular -s. Behavioral button-press responses and electroencephalographic recordings were collected. Behavioral judgments and mean amplitude of the N400 and P600 components were examined. Results Adolescents in the SLI-Persistent group had lower sentence judgment accuracy overall, relative to the NL and SLI-Recovered groups. Accuracy in judging omission and commission syntactic errors were marginally different, with marginally lower accuracy for commission errors. All groups demonstrated an N400 component elicited by semantic violations. However, adolescents in the SLI-Persistent group demonstrated a less robust P600 component for syntactic violations. Furthermore, adolescents in the SLI-Recovered group exhibited a similar neural profile to the NL group for the semantic and syntactic omission violations. However, a unique profile with initial negativity was observed in the SLI-Recovered group in the commission violation condition. Conclusions Adolescents with persistent language impairment continue to demonstrate delays in language processing at the behavioral and neural levels. Conversely, the adolescents in the SLI-Recovered group appear to have made gains in language processing skills to overcome their initial impairments. However, our findings suggest that the adolescents in the SLI-Recovered group may have compensatory processing strategies for some aspects of language, as evidenced by a unique event-related potential profile.
Collapse
|
11
|
Manfredi M, Cohn N, Kutas M. When a hit sounds like a kiss: An electrophysiological exploration of semantic processing in visual narrative. BRAIN AND LANGUAGE 2017; 169:28-38. [PMID: 28242517 PMCID: PMC5465314 DOI: 10.1016/j.bandl.2017.02.001] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/26/2016] [Revised: 02/02/2017] [Accepted: 02/07/2017] [Indexed: 06/06/2023]
Abstract
Researchers have long questioned whether information presented through different sensory modalities involves distinct or shared semantic systems. We investigated uni-sensory cross-modal processing by recording event-related brain potentials to words replacing the climactic event in a visual narrative sequence (comics). We compared Onomatopoeic words, which phonetically imitate action sounds (Pow!), with Descriptive words, which describe an action (Punch!), that were (in)congruent within their sequence contexts. Across two experiments, larger N400s appeared to Anomalous Onomatopoeic or Descriptive critical panels than to their congruent counterparts, reflecting a difficulty in semantic access/retrieval. Also, Descriptive words evinced a greater late frontal positivity compared to Onomatopoetic words, suggesting that, though plausible, they may be less predictable/expected in visual narratives. Our results indicate that uni-sensory cross-model integration of word/letter-symbol strings within visual narratives elicit ERP patterns typically observed for written sentence processing, thereby suggesting the engagement of similar domain-independent integration/interpretation mechanisms.
Collapse
Affiliation(s)
- Mirella Manfredi
- Department of Psychology, University of Milano-Bicocca, Milan, Italy; Social and Cognitive Neuroscience Laboratory, Center for Biological Science and Health, Mackenzie Presbyterian University, São Paulo, Brazil.
| | - Neil Cohn
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA; Tilburg Center for Cognition and Communication, Tilburg University, Tilburg, Netherlands
| | - Marta Kutas
- Department of Cognitive Science, University of California, San Diego, La Jolla, CA, USA
| |
Collapse
|
12
|
Hendrickson K, Walenski M, Friend M, Love T. The organization of words and environmental sounds in memory. Neuropsychologia 2015; 69:67-76. [PMID: 25624059 DOI: 10.1016/j.neuropsychologia.2015.01.035] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2014] [Revised: 11/21/2014] [Accepted: 01/23/2015] [Indexed: 11/24/2022]
Abstract
In the present study we used event-related potentials to compare the organization of linguistic and meaningful nonlinguistic sounds in memory. We examined N400 amplitudes as adults viewed pictures presented with words or environmental sounds that matched the picture (Match), that shared semantic features with the expected match (Near Violation), and that shared relatively few semantic features with the expected match (Far Violation). Words demonstrated incremental N400 amplitudes based on featural similarity from 300-700ms, such that both Near and Far Violations exhibited significant N400 effects, however Far Violations exhibited greater N400 effects than Near Violations. For environmental sounds, Far Violations but not Near Violations elicited significant N400 effects, in both early (300-400ms) and late (500-700ms) time windows, though a graded pattern similar to that of words was seen in the mid-latency time window (400-500ms). These results indicate that the organization of words and environmental sounds in memory is differentially influenced by featural similarity, with a consistently fine-grained graded structure for words but not sounds.
Collapse
Affiliation(s)
- Kristi Hendrickson
- Center for Research in Language, University of California, San Diego, USA; School of Speech, Language, and Hearing Sciences, San Diego State University, USA; Joint Doctoral Program in Language and Communicative Disorders, San Diego State University, USA.
| | - Matthew Walenski
- School of Speech, Language, and Hearing Sciences, San Diego State University, USA.
| | | | - Tracy Love
- Center for Research in Language, University of California, San Diego, USA; School of Speech, Language, and Hearing Sciences, San Diego State University, USA.
| |
Collapse
|
13
|
Zhou L, Jiang C, Delogu F, Yang Y. Spatial conceptual associations between music and pictures as revealed by N400 effect. Psychophysiology 2014; 51:520-8. [DOI: 10.1111/psyp.12195] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/04/2013] [Accepted: 12/13/2013] [Indexed: 11/29/2022]
Affiliation(s)
- Linshu Zhou
- Key Laboratory of Behavioral Science, Institute of Psychology; Chinese Academy of Sciences; Beijing China
- University of Chinese Academy of Sciences; Beijing China
| | - Cunmei Jiang
- Music College; Shanghai Normal University; Shanghai China
| | - Franco Delogu
- College of Arts and Sciences; Lawrence Technological University; Southfield Michigan USA
| | - Yufang Yang
- Key Laboratory of Behavioral Science, Institute of Psychology; Chinese Academy of Sciences; Beijing China
| |
Collapse
|
14
|
Frey A, Aramaki M, Besson M. Conceptual priming for realistic auditory scenes and for auditory words. Brain Cogn 2013; 84:141-52. [PMID: 24378910 DOI: 10.1016/j.bandc.2013.11.013] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/24/2012] [Revised: 11/22/2013] [Accepted: 11/26/2013] [Indexed: 10/25/2022]
Abstract
Two experiments were conducted using both behavioral and Event-Related brain Potentials methods to examine conceptual priming effects for realistic auditory scenes and for auditory words. Prime and target sounds were presented in four stimulus combinations: Sound-Sound, Word-Sound, Sound-Word and Word-Word. Within each combination, targets were conceptually related to the prime, unrelated or ambiguous. In Experiment 1, participants were asked to judge whether the primes and targets fit together (explicit task) and in Experiment 2 they had to decide whether the target was typical or ambiguous (implicit task). In both experiments and in the four stimulus combinations, reaction times and/or error rates were longer/higher and the N400 component was larger to ambiguous targets than to conceptually related targets, thereby pointing to a common conceptual system for processing auditory scenes and linguistic stimuli in both explicit and implicit tasks. However, fine-grained analyses also revealed some differences between experiments and conditions in scalp topography and duration of the priming effects possibly reflecting differences in the integration of perceptual and cognitive attributes of linguistic and nonlinguistic sounds. These results have clear implications for the building-up of virtual environments that need to convey meaning without words.
Collapse
Affiliation(s)
- Aline Frey
- Laboratoire Cognitions Humaine & Artificielle, Université Paris 8, Saint-Denis, France.
| | - Mitsuko Aramaki
- Laboratoire de Mécanique et d'Acoustique, CNRS, Marseille, France
| | - Mireille Besson
- Laboratoire de Neurosciences Cognitives, CNRS & Aix-Marseille Université, Marseille, France; Cuban Neuroscience Center, Habana, Cuba
| |
Collapse
|
15
|
Macizo P, Van Petten C, O'Rourke PL. Semantic access to embedded words? Electrophysiological and behavioral evidence from Spanish and English. BRAIN AND LANGUAGE 2012; 123:113-124. [PMID: 22925681 DOI: 10.1016/j.bandl.2012.07.010] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2012] [Revised: 06/12/2012] [Accepted: 07/23/2012] [Indexed: 06/01/2023]
Abstract
Many multisyllabic words contain shorter words that are not semantic units, like the CAP in HANDICAP and the DURA (hard) in VERDURA (vegetable). The spaces between printed words identify word boundaries, but spurious identification of these embedded words is a potentially greater challenge for spoken language comprehension, a challenge that is handled by different mechanisms in different models of auditory word recognition. Subphonemic acoustic differences--subtle differences in pronunciation--often differentiate embedded words from genuine words. We examined semantic access to embedded words in two languages with different phonology by presenting carrier words followed by targets related to the embedded words and recording event-related potentials and lexical decision times in 34 Spanish/English bilinguals. No evidence of embedded word access was observed in brain activity or behavior, and this could not be attributed to subphonemic acoustic factors. The data place constraints on models of speech segmentation.
Collapse
|
16
|
Gygi B, Shafiro V. The incongruency advantage for environmental sounds presented in natural auditory scenes. J Exp Psychol Hum Percept Perform 2011; 37:551-65. [PMID: 21355664 DOI: 10.1037/a0020671] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The effect of context on the identification of common environmental sounds (e.g., dogs barking or cars honking) was tested by embedding them in familiar auditory background scenes (street ambience, restaurants). Initial results with subjects trained on both the scenes and the sounds to be identified showed a significant advantage of about five percentage points better accuracy for sounds that were contextually incongruous with the background scene (e.g., a rooster crowing in a hospital). Further studies with naive (untrained) listeners showed that this incongruency advantage (IA) is level-dependent: there is no advantage for incongruent sounds lower than a Sound/Scene ratio (So/Sc) of -7.5 dB, but there is about five percentage points better accuracy for sounds with greater So/Sc. Testing a new group of trained listeners on a larger corpus of sounds and scenes showed that the effect is robust and not confined to a specific stimulus set. Modeling using spectral-temporal measures showed that neither analyses based on acoustic features, nor semantic assessments of sound-scene congruency can account for this difference, indicating the IA is a complex effect, possibly arising from the sensitivity of the auditory system to new and unexpected events, under particular listening conditions.
Collapse
Affiliation(s)
- Brian Gygi
- Speech and Hearing Research, Veterans Affairs Northern California Health Care System, 150 Muir Road, Martinez, CA 94553, USA.
| | | |
Collapse
|
17
|
Liu B, Wang Z, Wu G, Meng X. Cognitive integration of asynchronous natural or non-natural auditory and visual information in videos of real-world events: an event-related potential study. Neuroscience 2011; 180:181-90. [DOI: 10.1016/j.neuroscience.2011.01.066] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2010] [Revised: 01/20/2011] [Accepted: 01/31/2011] [Indexed: 11/25/2022]
|
18
|
Schirmer A, Soh YH, Penney TB, Wyse L. Perceptual and conceptual priming of environmental sounds. J Cogn Neurosci 2011; 23:3241-53. [PMID: 21281092 DOI: 10.1162/jocn.2011.21623] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
It is still unknown whether sonic environments influence the processing of individual sounds in a similar way as discourse or sentence context influences the processing of individual words. One obstacle to answering this question has been the failure to dissociate perceptual (i.e., how similar are sonic environment and target sound?) and conceptual (i.e., how related are sonic environment and target?) priming effects. In this study, we dissociate these effects by creating prime-target pairs with a purely perceptual or both a perceptual and conceptual relationship. Perceptual prime-target pairs were derived from perceptual-conceptual pairs (i.e., meaningful environmental sounds) by shuffling the spectral composition of primes and targets so as to preserve their perceptual relationship while making them unrecognizable. Hearing both original and shuffled targets elicited a more positive N1/P2 complex in the ERP when targets were related to a preceding prime as compared with unrelated. Only related original targets reduced the N400 amplitude. Related shuffled targets tended to decrease the amplitude of a late temporo-parietal positivity. Taken together, these effects indicate that sonic environments influence first the perceptual and then the conceptual processing of individual sounds. Moreover, the influence on conceptual processing is comparable to the influence linguistic context has on the processing of individual words.
Collapse
|
19
|
Aramaki M, Marie C, Kronland-Martinet R, Ystad S, Besson M. Sound categorization and conceptual priming for nonlinguistic and linguistic sounds. J Cogn Neurosci 2010; 22:2555-69. [PMID: 19929328 DOI: 10.1162/jocn.2009.21398] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
The aim of these experiments was to compare conceptual priming for linguistic and for a homogeneous class of nonlinguistic sounds, impact sounds, by using both behavioral (percentage errors and RTs) and electrophysiological measures (ERPs). Experiment 1 aimed at studying the neural basis of impact sound categorization by creating typical and ambiguous sounds from different material categories (wood, metal, and glass). Ambiguous sounds were associated with slower RTs and larger N280, smaller P350/P550 components, and larger negative slow wave than typical impact sounds. Thus, ambiguous sounds were more difficult to categorize than typical sounds. A category membership task was used in Experiment 2. Typical sounds were followed by sounds from the same or from a different category or by ambiguous sounds. Words were followed by words, pseudowords, or nonwords. Error rate was highest for ambiguous sounds and for pseudowords and both elicited larger N400-like components than same typical sounds and words. Moreover, both different typical sounds and nonwords elicited P300 components. These results are discussed in terms of similar conceptual priming effects for nonlinguistic and linguistic stimuli.
Collapse
Affiliation(s)
- Mitsuko Aramaki
- CNRS-Institut de Neurosciences Cognitives de la Méditerranée, Marseille Cedex, France.
| | | | | | | | | |
Collapse
|
20
|
Pinheiro AP, Galdo-Álvarez S, Sampaio A, Niznikiewicz M, Gonçalves OF. Electrophysiological correlates of semantic processing in Williams syndrome. RESEARCH IN DEVELOPMENTAL DISABILITIES 2010; 31:1412-1425. [PMID: 20674263 DOI: 10.1016/j.ridd.2010.06.017] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/05/2010] [Revised: 06/15/2010] [Accepted: 06/21/2010] [Indexed: 05/29/2023]
Abstract
Williams syndrome (WS), a genetic neurodevelopmental disorder due to microdeletion in chromosome 7, has been described as a syndrome with an intriguing socio-cognitive phenotype. Cognitively, the relative preservation of language and face processing abilities coexists with severe deficits in visual-spatial tasks, as well as in tasks involving abstract reasoning. However, in spite of early claims of the independence of language from general cognition in WS, a detailed investigation of language subcomponents has demonstrated several abnormalities in lexical-semantic processing. Nonetheless, the neurobiological processes underlying language processing in Williams syndrome remain to be clarified. The aim of this study was to examine the electrophysiological correlates of semantic processing in WS, taking typical development as a reference. A group of 12 individuals diagnosed with Williams syndrome, with age range between 9 and 31 years, was compared with a group of typically developing participants, individually matched in chronological age, gender and handedness. Participants were presented with sentences that ended with words incongruent (50%) with the previous sentence context or with words judged to be its best completion (50%), and they were asked to decide if the sentence made sense or not. Results in WS suggest atypical sensory ERP components (N100 and P200), preserved N400 amplitude, and abnormal P600 in WS, with the latter being related to late integration and re-analysis processes. These results may represent a physiological signature of underlying impaired on-line language processing in this disorder.
Collapse
Affiliation(s)
- Ana P Pinheiro
- Neuropsychophysiology Lab, CiPsi, School of Psychology, University of Minho, Braga, Portugal.
| | | | | | | | | |
Collapse
|
21
|
Liu L, Vira A, Friedman E, Minas J, Bolger D, Bitan T, Booth J. Children with reading disability show brain differences in effective connectivity for visual, but not auditory word comprehension. PLoS One 2010; 5:e13492. [PMID: 21049093 PMCID: PMC2963599 DOI: 10.1371/journal.pone.0013492] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2010] [Accepted: 09/27/2010] [Indexed: 11/18/2022] Open
Abstract
BACKGROUND Previous literature suggests that those with reading disability (RD) have more pronounced deficits during semantic processing in reading as compared to listening comprehension. This discrepancy has been supported by recent neuroimaging studies showing abnormal activity in RD during semantic processing in the visual but not in the auditory modality. Whether effective connectivity between brain regions in RD could also show this pattern of discrepancy has not been investigated. METHODOLOGY/PRINCIPAL FINDINGS Children (8- to 14-year-olds) were given a semantic task in the visual and auditory modality that required an association judgment as to whether two sequentially presented words were associated. Effective connectivity was investigated using Dynamic Causal Modeling (DCM) on functional magnetic resonance imaging (fMRI) data. Bayesian Model Selection (BMS) was used separately for each modality to find a winning family of DCM models separately for typically developing (TD) and RD children. BMS yielded the same winning family with modulatory effects on bottom-up connections from the input regions to middle temporal gyrus (MTG) and inferior frontal gyrus(IFG) with inconclusive evidence regarding top-down modulations. Bayesian Model Averaging (BMA) was thus conducted across models in this winning family and compared across groups. The bottom-up effect from the fusiform gyrus (FG) to MTG rather than the top-down effect from IFG to MTG was stronger in TD compared to RD for the visual modality. The stronger bottom-up influence in TD was only evident for related word pairs but not for unrelated pairs. No group differences were noted in the auditory modality. CONCLUSIONS/SIGNIFICANCE This study revealed a modality-specific deficit for children with RD in bottom-up effective connectivity from orthographic to semantic processing regions. There were no group differences in connectivity from frontal regions, suggesting that the core deficit in RD is not in top-down modulation.
Collapse
Affiliation(s)
- Li Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University, Beijing, People's Republic of China
- Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Amit Vira
- Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Emma Friedman
- Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Jennifer Minas
- Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| | - Donald Bolger
- Department of Human Development, Maryland University, College Park, Maryland, United States of America
| | - Tali Bitan
- Department of Communication Disorders, Haifa University, Haifa, Israel
| | - James Booth
- Department of Communication Sciences and Disorders, Northwestern University, Evanston, Illinois, United States of America
| |
Collapse
|
22
|
Plante E, Bahl M, Vance R, Gerken L. Children with specific language impairment show rapid, implicit learning of stress assignment rules. JOURNAL OF COMMUNICATION DISORDERS 2010; 43:397-406. [PMID: 20542518 PMCID: PMC2922431 DOI: 10.1016/j.jcomdis.2010.04.012] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/07/2009] [Revised: 04/13/2010] [Accepted: 04/23/2010] [Indexed: 05/15/2023]
Abstract
UNLABELLED An implicit learning paradigm was used to assess children's sensitivity to syllable stress information in an artificial language. Study 1 demonstrated that preschool children, with and without specific language impairment (SLI), can generalize patterns of stress heard during a brief period of familiarization, and can also abstract underlying ordered rules by which stress patterns were assigned to syllables. In Study 2, the salience of stressed elements was acoustically enhanced. Counter to expectations, there was no evidence of learning with this manipulation for either the typically developing children or children with SLI. The results suggest that children with SLI and their typically developing peers are sensitive to syllable stress cues to language structure. However, attempts to draw attention to these patterns by making them more salient may prompt children to use alternate learning strategies that do not lead to an implicit understanding of how stress contributes to the structure of language. LEARNING OUTCOMES The reader will be able to understand: (1) that children with SLI can learn and generalize the rules for assigning word-level stress patterns within minutes of hearing examples, but (2) strategies to enhance learning may actually have the opposite effect for these children.
Collapse
Affiliation(s)
- Elena Plante
- University of Arizona, Tucson, AZ, United States.
| | | | | | | |
Collapse
|
23
|
Schön D, Ystad S, Kronland-Martinet R, Besson M. The evocative power of sounds: conceptual priming between words and nonverbal sounds. J Cogn Neurosci 2010; 22:1026-35. [PMID: 19583472 DOI: 10.1162/jocn.2009.21302] [Citation(s) in RCA: 30] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Two experiments were conducted to examine the conceptual relation between words and nonmeaningful sounds. In order to reduce the role of linguistic mediation, sounds were recorded in such a way that it was highly unlikely to identify the source that produced them. Related and unrelated sound-word pairs were presented in Experiment 1 and the order of presentation was reversed in Experiment 2 (word-sound). Results showed that, in both experiments, participants were sensitive to the conceptual relation between the two items. They were able to correctly categorize items as related or unrelated with good accuracy. Moreover, a relatedness effect developed in the event-related brain potentials between 250 and 600 msec, although with a slightly different scalp topography for word and sound targets. Results are discussed in terms of similar conceptual processing networks and we propose a tentative model of the semiotics of sounds.
Collapse
Affiliation(s)
- Daniele Schön
- CNRS & Université de la Méditerranée, Marseille, France.
| | | | | | | |
Collapse
|
24
|
Dobel C, Lagemann L, Zwitserlood P. Non-native phonemes in adult word learning: evidence from the N400m. Philos Trans R Soc Lond B Biol Sci 2010; 364:3697-709. [PMID: 19933141 DOI: 10.1098/rstb.2009.0158] [Citation(s) in RCA: 22] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Newborns are equipped with a large phonemic inventory that becomes tuned to one's native language early in life. We review and add new data about how learning of a non-native phoneme can be accomplished in adults and how the efficiency of word learning can be assessed by neurophysiological measures. For this purpose, we studied the acquisition of the voiceless, bilabial fricative /Phi/ via a statistical-learning paradigm. Phonemes were embedded in minimal pairs of pseudowords, differing only with respect to the fricative (/aPhio/ versus /afo/). During learning, pseudowords were combined with pictures of objects with some combinations of pseudowords and pictures occurring more frequently than others. Behavioural data and the N400m component, as an index of lexical activation/semantic access, showed that participants had learned to associate the pseudowords with the pictures. However, they could not discriminate within the minimal pairs. Importantly, before learning, the novel words with the sound /Phi/ showed smaller N400 amplitudes than those with native phonemes, evidencing their non-word status. Learning abolished this difference indicating that /Phi/ had become integrated into the native category /f/, instead of establishing a novel category. Our data and review demonstrate that native phonemic categories are powerful attractors hampering the mastery of non-native contrasts.
Collapse
Affiliation(s)
- Christian Dobel
- Institute for Biomagnetism and Biosignal Analysis, Westfälische Wilhelms-Universität Münster, 48149 Münster, Germany.
| | | | | |
Collapse
|
25
|
McCleery JP, Ceponiene R, Burner KM, Townsend J, Kinnear M, Schreibman L. Neural correlates of verbal and nonverbal semantic integration in children with autism spectrum disorders. J Child Psychol Psychiatry 2010; 51:277-86. [PMID: 20025622 DOI: 10.1111/j.1469-7610.2009.02157.x] [Citation(s) in RCA: 53] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
BACKGROUND Autism is a pervasive developmental disorder characterized by deficits in social-emotional, social-communicative, and language skills. Behavioral and neuroimaging studies have found that children with autism spectrum disorders (ASD) evidence abnormalities in semantic processing, with particular difficulties in verbal comprehension. However, it is not known whether these semantic deficits are confined to the verbal domain or represent a more general problem with semantic processing. The focus of the current study was to investigate verbal and meaningful nonverbal semantic processing in high-functioning children with autism (mean age = 5.8 years) using event-related potentials (ERPs). METHOD ERPs were recorded while children attended to semantically matching and mismatching picture-word and picture-environmental sound pairs. RESULTS ERPs of typically developing children exhibited evidence of semantic incongruency detection in both the word and environmental sound conditions, as indexed by elicitation of an N400 effect. In contrast, children with ASD showed an N400 effect in the environmental sound condition but not in the word condition. CONCLUSIONS These results provide evidence for a deficiency in the automatic activation of semantic representations in children with ASD, and suggest that this deficit is somewhat more selective to, or more severe in, the verbal than the nonverbal domain.
Collapse
|
26
|
Cummings A, Ceponiene R. Verbal and nonverbal semantic processing in children with developmental language impairment. Neuropsychologia 2010; 48:77-85. [PMID: 19698728 PMCID: PMC2794944 DOI: 10.1016/j.neuropsychologia.2009.08.012] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/16/2009] [Revised: 06/03/2009] [Accepted: 08/13/2009] [Indexed: 11/21/2022]
Abstract
In an effort to clarify whether semantic integration is impaired in verbal and nonverbal auditory domains in children with developmental language impairment (a.k.a., LI and SLI), the present study obtained behavioral and neural responses to words and environmental sounds in children with language impairment and their typically developing age-matched controls (ages 7-15 years). Event-related brain potentials (ERPs) were recorded while children performed a forced-choice matching task on semantically matching and mismatching visual-auditory, picture-word and picture-environmental sound pairs. Behavioral accuracy and reaction time measures were similar for both groups of children, with environmental sounds eliciting more accurate responses than words. In picture-environmental sound trials, behavioral performance and the brain's response to semantic incongruency (i.e., the N400 effect) of the children with language impairment were comparable to those of their typically developing peers. However, in picture-word trials, children with LI tended to be less accurate than their controls and their N400 effect was significantly delayed in latency. Thus, the children with LI demonstrated a semantic integration deficit that was somewhat specific to the verbal domain. The particular finding of a delayed N400 effect is consistent with the storage deficit hypothesis of language impairment (Kail & Leonard, 1986) suggesting weakened and/or less efficient connections within the language networks of children with LI.
Collapse
Affiliation(s)
- Alycia Cummings
- San Diego State University/University of California, San Diego Joint Doctoral Program in Language and Communicative Disorders, San Diego, CA, USA.
| | | |
Collapse
|
27
|
Duncan CC, Barry RJ, Connolly JF, Fischer C, Michie PT, Näätänen R, Polich J, Reinvang I, Van Petten C. Event-related potentials in clinical research: guidelines for eliciting, recording, and quantifying mismatch negativity, P300, and N400. Clin Neurophysiol 2009; 120:1883-1908. [PMID: 19796989 DOI: 10.1016/j.clinph.2009.07.045] [Citation(s) in RCA: 740] [Impact Index Per Article: 49.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2008] [Revised: 06/26/2009] [Accepted: 07/04/2009] [Indexed: 01/17/2023]
Abstract
This paper describes recommended methods for the use of event-related brain potentials (ERPs) in clinical research and reviews applications to a variety of psychiatric and neurological disorders. Techniques are presented for eliciting, recording, and quantifying three major cognitive components with confirmed clinical utility: mismatch negativity (MMN), P300, and N400. Also highlighted are applications of each of the components as methods of investigating central nervous system pathology. The guidelines are intended to assist investigators who use ERPs in clinical research, in an effort to provide clear and concise recommendations and thereby to standardize methodology and facilitate comparability of data across laboratories.
Collapse
Affiliation(s)
- Connie C Duncan
- Clinical Psychophysiology and Psychopharmacology Laboratory, Department of Psychiatry, Uniformed Services University of the Health Sciences, Bethesda, MD, USA.
| | - Robert J Barry
- School of Psychology and Brain & Behaviour Research Institute, University of Wollongong, Wollongong, NSW, Australia
| | - John F Connolly
- Department of Linguistics and Languages, McMaster University, Hamilton, Ont., Canada
| | - Catherine Fischer
- Hospices Civils de Lyon, Neurological Hospital and INSERM U821, Lyon, France
| | - Patricia T Michie
- School of Psychology, The University of Newcastle, Callaghan, NSW, Australia
| | - Risto Näätänen
- Department of Psychology, University of Tartu, Tartu, Estonia; Center of Functionally Integrative Neuroscience (CFIN), University of Aarhus, Aarhus, Denmark; Cognitive Brain Research Unit, Department of Psychology, University of Helsinki, Helsinki, Finland
| | - John Polich
- Cognitive Electrophysiology Laboratory, Molecular and Integrative Neurosciences Department, The Scripps Research Institute, La Jolla, CA, USA
| | - Ivar Reinvang
- Department of Psychology, University of Oslo, Oslo, Norway
| | - Cyma Van Petten
- Department of Psychology, Binghamton University, Binghamton, New York, USA
| |
Collapse
|
28
|
Alt M, Gutmann ML. Fast mapping semantic features: performance of adults with normal language, history of disorders of spoken and written language, and attention deficit hyperactivity disorder on a word-learning task. JOURNAL OF COMMUNICATION DISORDERS 2009; 42:347-64. [PMID: 19439319 PMCID: PMC2771630 DOI: 10.1016/j.jcomdis.2009.03.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/18/2008] [Revised: 02/18/2009] [Accepted: 03/26/2009] [Indexed: 05/12/2023]
Abstract
PURPOSE This study was designed to test the word learning abilities of adults with typical language abilities, those with a history of disorders of spoken or written language (hDSWL), and hDSWL plus attention deficit hyperactivity disorder (+ADHD). METHODS Sixty-eight adults were required to associate a novel object with a novel label, and then recognize semantic features of the object and phonological features of the label. Participants were tested for overt ability (accuracy) and covert processing (reaction time). RESULTS The +ADHD group was less accurate at mapping semantic features and slower to respond to lexical labels than both other groups. Different factors correlated with word learning performance for each group. CONCLUSIONS Adults with language and attention deficits are more impaired at word learning than adults with language deficits only. Despite behavioral profiles like typical peers, adults with hDSWL may use different processing strategies than their peers. LEARNING OUTCOMES Readers will be able to: (1) recognize the influence of a dual disability (hDSWL and ADHD) on word learning outcomes; (2) identify factors that may contribute to word learning in adults in terms of (a) the nature of the words to be learned and (b) the language processing of the learner.
Collapse
Affiliation(s)
- Mary Alt
- University of Arizona, Department of Speech, Language, and Hearing Sciences, 1131 E. 2St., Tucson, AZ 85721 USA, ,
| | - Michelle L. Gutmann
- University of Arizona, Department of Speech, Language, and Hearing Sciences, 1131 E. 2St., Tucson, AZ 85721 USA, ,
| |
Collapse
|
29
|
Abstract
Measuring event-related potentials (ERPs) has been fundamental to our understanding of how language is encoded in the brain. One particular ERP response, the N400 response, has been especially influential as an index of lexical and semantic processing. However, there remains a lack of consensus on the interpretation of this component. Resolving this issue has important consequences for neural models of language comprehension. Here we show that evidence bearing on where the N400 response is generated provides key insights into what it reflects. A neuroanatomical model of semantic processing is used as a guide to interpret the pattern of activated regions in functional MRI, magnetoencephalography and intracranial recordings that are associated with contextual semantic manipulations that lead to N400 effects.
Collapse
|
30
|
Cummings A, Ceponiene R, Dick F, Saygin AP, Townsend J. A developmental ERP study of verbal and non-verbal semantic processing. Brain Res 2008; 1208:137-49. [PMID: 18387601 DOI: 10.1016/j.brainres.2008.02.015] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2007] [Revised: 02/06/2008] [Accepted: 02/06/2008] [Indexed: 10/22/2022]
Abstract
To clarify how different the processing of verbal information is from the processing of meaningful non-verbal information, the present study characterized the developmental changes in neural responses to words and environmental sounds from pre-adolescence (7-9 years) through adolescence (12-14 years) to adulthood (18-25 years). Children and adults' behavioral and electrophysiological responses (the N400 effect of event-related potentials) were compared during the processing of words and environmental sounds presented in semantically matching and mismatching picture contexts. Behavioral accuracy of picture-sound matching improved until adulthood, while reaction time measures leveled out by age 12. No major electrophysiological changes in the N400 effect were observed between pre-adolescence and adolescence. When compared to adults, children demonstrated significant maturational changes including longer latencies and larger amplitudes of the N400 effect. Interestingly, these developmental differences were driven by stimulus type: the Environmental Sound N400 effect decreased in latency from adolescence to adulthood, while no age effects were observed in response to Words. Thus, while the semantic processing of single words is well established by 7 years of age, the processing of environmental sounds continues to improve throughout development.
Collapse
Affiliation(s)
- Alycia Cummings
- Project in Cognitive and Neural Development, University of California, San Diego, USA.
| | | | | | | | | |
Collapse
|
31
|
Dick F, Saygin AP, Galati G, Pitzalis S, Bentrovato S, D'Amico S, Wilson S, Bates E, Pizzamiglio L. What is Involved and What is Necessary for Complex Linguistic and Nonlinguistic Auditory Processing: Evidence from Functional Magnetic Resonance Imaging and Lesion Data. J Cogn Neurosci 2007; 19:799-816. [PMID: 17488205 DOI: 10.1162/jocn.2007.19.5.799] [Citation(s) in RCA: 51] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2022]
Abstract
Abstract
We used functional magnetic resonance imaging (fMRI) in conjunction with a voxel-based approach to lesion symptom mapping to quantitatively evaluate the similarities and differences between brain areas involved in language and environmental sound comprehension. In general, we found that language and environmental sounds recruit highly overlapping cortical regions, with cross-domain differences being graded rather than absolute. Within language-based regions of interest, we found that in the left hemisphere, language and environmental sound stimuli evoked very similar volumes of activation, whereas in the right hemisphere, there was greater activation for environmental sound stimuli. Finally, lesion symptom maps of aphasic patients based on environmental sounds or linguistic deficits [Saygin, A. P., Dick, F., Wilson, S. W., Dronkers, N. F., & Bates, E. Shared neural resources for processing language and environmental sounds: Evidence from aphasia. Brain, 126, 928–945, 2003] were generally predictive of the extent of blood oxygenation level dependent fMRI activation across these regions for sounds and linguistic stimuli in young healthy subjects.
Collapse
|
32
|
Cummings A, Čeponienė R, Koyama A, Saygin A, Townsend J, Dick F. Auditory semantic networks for words and natural sounds. Brain Res 2006; 1115:92-107. [DOI: 10.1016/j.brainres.2006.07.050] [Citation(s) in RCA: 67] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2006] [Revised: 07/12/2006] [Accepted: 07/13/2006] [Indexed: 10/24/2022]
|
33
|
Van Petten C, Luka BJ. Neural localization of semantic context effects in electromagnetic and hemodynamic studies. BRAIN AND LANGUAGE 2006; 97:279-93. [PMID: 16343606 DOI: 10.1016/j.bandl.2005.11.003] [Citation(s) in RCA: 241] [Impact Index Per Article: 13.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/20/2005] [Revised: 10/31/2005] [Accepted: 11/03/2005] [Indexed: 05/05/2023]
Abstract
Measures of electrical brain activity (event-related potentials, ERPs) have been useful in understanding language processing for several decades. Extant data suggest that the amplitude of the N400 component of the ERP is a general index of the ease or difficulty of retrieving stored conceptual knowledge associated with a word, which is dependent on both the stored representation itself, and the retrieval cues provided by the preceding context. Recordings from patients with brain damage, intracranial recordings, and magnetoencephalographic data implicate a (probably large portion of) the left temporal lobe as the largest source of the N400 semantic context effect, with a substantial but lesser contribution from the right temporal lobe. Event-related functional magnetic resonance (fMRI) studies using semantic context manipulations are dominated by observations of greater hemodynamic activity for incongruent sentence completions or semantically unrelated words than congruent or related words, consistent with the direction of the ERP effect. The locations of the hemodynamic effects show some variability across studies, but one commonly identified region is the left superior temporal gyrus, which is compatible with the electrophysiological results. A second commonly identified region in the fMRI studies is the left inferior frontal gyrus, which does not appear to make a substantial contribution to the N400 effect.
Collapse
Affiliation(s)
- Cyma Van Petten
- Department of Psychology, University of Arizona, Tucson, AZ 85721, USA.
| | | |
Collapse
|
34
|
Ellis Weismer S, Plante E, Jones M, Tomblin JB. A functional magnetic resonance imaging investigation of verbal working memory in adolescents with specific language impairment. JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2005; 48:405-25. [PMID: 15989401 DOI: 10.1044/1092-4388(2005/028)] [Citation(s) in RCA: 37] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/22/2003] [Revised: 04/16/2004] [Accepted: 07/27/2004] [Indexed: 05/03/2023]
Abstract
This study used neuroimaging and behavioral techniques to examine the claim that processing capacity limitations underlie specific language impairment (SLI). Functional magnetic resonance imaging (fMRI) was used to investigate verbal working memory in adolescents with SLI and normal language (NL) controls. The experimental task involved a modified listening span measure that included sentence encoding and recognition of final words in prior sets of sentences. The SLI group performed significantly poorer than the NL group for both encoding and recognition and displayed slower reaction times for correct responses on high complexity encoding items. fMRI results revealed that the SLI group exhibited significant hypoactivation during encoding in regions that have been implicated in attentional and memory processes, as well as hypoactivation during recognition in regions associated with language processing. Correlational analyses indicated that adolescents with SLI exhibited different patterns of coordinating activation among brain regions relative to controls for both encoding and recognition, suggesting reliance on a less functional network. These findings are interpreted as supporting the notion that constraints in nonlinguistic systems play a role in SLI.
Collapse
|
35
|
Pankratz M, Morrison A, Plante E. Difference in standard scores of adults on the Peabody Picture Vocabulary Test (Revised and Third Edition). JOURNAL OF SPEECH, LANGUAGE, AND HEARING RESEARCH : JSLHR 2004; 47:714-718. [PMID: 15212579 DOI: 10.1044/1092-4388(2004/054)] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/24/2023]
Abstract
Differences in the standard scores for the Peabody Picture Vocabulary Test-Revised (PPVT-R; L. M. Dunn and L. M. Dunn, 1981) and the PPVT-Third Edition (PPVT-III; Dunn and Dunn, 1997b) are known to exist for children, with typically higher scores occurring on the PPVT-III. However, these tests are administered into adulthood as well, and score equivalence must be evaluated for this age range. Analysis of data from the PPVT-R and PPVT-III tests from 76 adult participants revealed significant score differences. Participants with poor language skills scored significantly higher on the PPVT-III than on the PPVT-R. The control group showed no significant difference between the PPVT-R and PPVT-III scores. The results suggest that the two tests should not be considered interchangeable.
Collapse
Affiliation(s)
- Mary Pankratz
- Speech & Hearing Sciences, University of Arizona, Tucson, 85721, USA.
| | | | | |
Collapse
|
36
|
Halgren E, Dhond RP, Christensen N, Van Petten C, Marinkovic K, Lewine JD, Dale AM. N400-like magnetoencephalography responses modulated by semantic context, word frequency, and lexical class in sentences. Neuroimage 2002; 17:1101-16. [PMID: 12414253 DOI: 10.1006/nimg.2002.1268] [Citation(s) in RCA: 297] [Impact Index Per Article: 13.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Words have been found to elicit a negative potential at the scalp peaking at approximately 400 ms that is strongly modulated by semantic context. The current study used whole-head magnetoencephalography (MEG) as male subjects read sentences ending with semantically congruous or incongruous words. Compared with congruous words, sentence-terminal incongruous words consistently evoked a large magnetic field over the left hemisphere, peaking at approximately 450 ms. Source modeling at this latency with conventional equivalent current dipoles (ECDs) placed the N400 m generator in or near the left superior temporal sulcus. A distributed solution constrained to the cortical surface suggested a sequence of differential activation, beginning in Wernicke's area at approximately 250 ms, spreading to anterior temporal sites at approximately 270 ms, to Broca's area by approximately 300 ms, to dorsolateral prefrontal cortices by approximately 320 ms, and to anterior orbital and frontopolar cortices by approximately 370 ms. Differential activity was exclusively left-sided until >370 ms, and then involved right anterior temporal and orbital cortices. At the peak of the N400 m, activation in the left hemisphere was estimated to be widespread in the anterior temporal, perisylvian, orbital, frontopolar, and dorsolateral prefrontal cortices. In the right hemisphere, the orbital, as well as, weakly, the right anterior temporal cortices were activated. Similar but weaker field patterns were evoked by intermediate words in the sentences, especially to low-frequency words occurring in early sentence positions where there is little preceding context. The locations of the N400 m sources identified with the distributed solution correspond well with those previously demonstrated with direct intracranial recordings, and suggested by functional magnetic resonance imaging (fMRI). These results help identify a distributed cortical network that supports online semantic processing.
Collapse
Affiliation(s)
- Eric Halgren
- MGH-NMR Center, Harvard Medical School, Boston, Massachusetts, USA.
| | | | | | | | | | | | | |
Collapse
|