1
|
Roberts K, Jentzsch I, Otto TU. Semantic congruency modulates the speed-up of multisensory responses. Sci Rep 2024; 14:567. [PMID: 38177170 PMCID: PMC10766646 DOI: 10.1038/s41598-023-50674-4] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/07/2023] [Accepted: 12/22/2023] [Indexed: 01/06/2024] Open
Abstract
Responses to multisensory signals are often faster compared to their unisensory components. This speed-up is typically attributed to target redundancy in that a correct response can be triggered by one or the other signal. In addition, semantic congruency of signals can also modulate multisensory responses; however, the contribution of semantic content is difficult to isolate as its manipulation commonly changes signal redundancy as well. To disentangle the effects of redundancy and semantic congruency, we manipulated semantic content but kept redundancy constant. We presented semantically congruent/incongruent animal pictures and sounds and asked participants to respond with the same response to two target animals (cats and dogs). We find that the speed-up of multisensory responses is larger for congruent (e.g., barking dogs) than incongruent combinations (e.g., barking cats). We then used a computational modelling approach to analyse audio-visual processing interferences that may underlie the effect. Our data is best described by a model that explains the semantic congruency modulation with a parameter that was previously linked to trial sequence effects, which in our experiment occur from the repetition/switching of both sensory modality and animal category. Yet, a systematic analysis of such trial sequence effects shows that the reported congruency effect is an independent phenomenon. Consequently, we discuss potential contributors to the semantic modulation of multisensory responses.
Collapse
Affiliation(s)
- Kalvin Roberts
- School of Psychology and Neuroscience, University of St. Andrews, St. Mary's Quad, South Street, St. Andrews, KY16 9JP, UK.
| | - Ines Jentzsch
- School of Psychology and Neuroscience, University of St. Andrews, St. Mary's Quad, South Street, St. Andrews, KY16 9JP, UK
| | - Thomas U Otto
- School of Psychology and Neuroscience, University of St. Andrews, St. Mary's Quad, South Street, St. Andrews, KY16 9JP, UK.
| |
Collapse
|
2
|
Letts E, Basharat A, Barnett-Cowan M. Evaluating the Effect of Semantic Congruency and Valence on Multisensory Integration. Multisens Res 2022; 35:309-334. [PMID: 35393374 DOI: 10.1163/22134808-bja10073] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2021] [Accepted: 02/21/2022] [Indexed: 11/19/2022]
Abstract
Previous studies have found that semantics, the higher-level meaning of stimuli, can impact multisensory integration; however, less is known about the effect of valence, an affective response to stimuli. This study investigated the effects of both semantic congruency and valence of non-speech audiovisual stimuli on multisensory integration via response time (RT) and temporal-order judgement (TOJ) tasks [assessing processing speed (RT), Point of Subjective Simultaneity (PSS), and time window when multisensory stimuli are likely to be perceived as simultaneous (temporal binding window; TBW)]. Through an online study with 40 participants (mean age: 26.25 years; females = 17), we found that both congruence and valence had a significant main effect on RT (congruency and positive valence decrease RT) and an interaction effect (congruent/positive valence condition being significantly faster than all others). For TOJ, there was a significant main effect of valence and a significant interaction effect where positive valence (compared to negative valence) and the congruent/positive condition (compared to all other conditions) required visual stimuli to be presented significantly earlier than auditory stimuli to be perceived as simultaneous. A subsequent analysis showed a positive correlation between TBW width and RT (as TBW widens, RT increases) for the categories that were furthest from true simultaneity in their PSS (Congruent/Positive and Incongruent/Negative). This study provides new evidence that supports previous research on semantic congruency and presents a novel incorporation of valence into behavioural responses.
Collapse
Affiliation(s)
- Elyse Letts
- Faculty of Health, University of Waterloo, 200 University Ave. W, Waterloo, ON, Canada, N2L3G1
| | - Aysha Basharat
- Faculty of Health, University of Waterloo, 200 University Ave. W, Waterloo, ON, Canada, N2L3G1
| | - Michael Barnett-Cowan
- Faculty of Health, University of Waterloo, 200 University Ave. W, Waterloo, ON, Canada, N2L3G1
| |
Collapse
|
3
|
Wiech K, Eippert F, Vandekerckhove J, Zaman J, Placek K, Tuerlinckx F, Vlaeyen JWS, Tracey I. Cortico-Brainstem Mechanisms of Biased Perceptual Decision-Making in the Context of Pain. THE JOURNAL OF PAIN 2022; 23:680-692. [PMID: 34856408 DOI: 10.1016/j.jpain.2021.11.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/03/2021] [Revised: 10/29/2021] [Accepted: 11/16/2021] [Indexed: 06/13/2023]
Abstract
Prior expectations can bias how we perceive pain. Using a drift diffusion model, we recently showed that this influence is primarily based on changes in perceptual decision-making (indexed as shift in starting point). Only during unexpected application of high-intensity noxious stimuli, altered information processing (indexed as increase in drift rate) explained the expectancy effect on pain processing. Here, we employed functional magnetic resonance imaging to investigate the neural basis of both these processes in healthy volunteers. On each trial, visual cues induced the expectation of high- or low-intensity noxious stimulation or signaled equal probability for both intensities. Participants categorized a subsequently applied electrical stimulus as either low- or high-intensity pain. A shift in starting point towards high pain correlated negatively with right dorsolateral prefrontal cortex activity during cue presentation underscoring its proposed role of "keeping pain out of mind". This anticipatory right dorsolateral prefrontal cortex signal increase was positively correlated with periaqueductal gray (PAG) activity when the expected high-intensity stimulation was applied. A drift rate increase during unexpected high-intensity pain was reflected in amygdala engagement and increased functional connectivity between amygdala and PAG. Our findings suggest involvement of the PAG in both decision-making bias and altered information processing to implement expectancy effects on pain. PERSPECTIVE: Modulation of pain through expectations has been linked to changes in perceptual decision-making and altered processing of afferent information. Our results suggest involvement of the dorsolateral prefrontal cortex, amygdala, and periaqueductal gray in these processes.
Collapse
Affiliation(s)
- Katja Wiech
- Wellcome Centre for Integrative Neuroimaging (WIN), Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford, UK.
| | - Falk Eippert
- Wellcome Centre for Integrative Neuroimaging (WIN), Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford, UK; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Joachim Vandekerckhove
- Department of Cognitive Sciences, University of California, Irvine, California; Research Group of Quantitative Psychology and Individual Differences, KU Leuven, Leuven, Belgium
| | - Jonas Zaman
- Research Group Health Psychology, KU Leuven, Leuven, Belgium
| | - Katerina Placek
- Takeda Pharmaceuticals, Statistics and Quantitative Sciences, Cambridge, Massachusetts
| | - Francis Tuerlinckx
- Research Group of Quantitative Psychology and Individual Differences, KU Leuven, Leuven, Belgium
| | - Johan W S Vlaeyen
- Research Group Health Psychology, KU Leuven, Leuven, Belgium; Research Group Experimental Health Psychology, Maastricht University, Maastricht, Netherlands
| | - Irene Tracey
- Wellcome Centre for Integrative Neuroimaging (WIN), Nuffield Department of Clinical Neurosciences, University of Oxford, John Radcliffe Hospital, Oxford, UK
| |
Collapse
|
4
|
Abstract
Increasing research has revealed that uninformative spatial sounds facilitate the early processing of visual stimuli. This study examined the crossmodal interactions of semantically congruent stimuli by assessing whether the presentation of event-related characteristic sounds facilitated or interfered with the visual search for corresponding event scenes in pictures. The search array consisted of four images: one target and three non-target pictures. Auditory stimuli were presented to participants in synchronization with picture onset using three types of sounds: a sound congruent with a target, a sound congruent with a distractor, or a control sound. The control sound varied across six experiments, alternating between a sound unrelated to the search stimuli, white noise, and no sound. Participants were required to swiftly localize a target position while ignoring the sound presentation. Visual localization resulted in rapid responses when a sound that was semantically related to the target was played. Furthermore, when a sound was semantically related to a distractor picture, the response times were longer. When the distractor-congruent sound was used, participants incorrectly localized the distractor position more often than at the chance level. These findings were replicated when the experiments ruled out the possibility that participants would learn picture-sound pairs during the visual tasks (i.e., the possibility of brief training during the experiments). Overall, event-related crossmodal interactions occur based on semantic representations, and audiovisual associations may develop as a result of long-term experiences rather than brief training in a laboratory.
Collapse
|
5
|
Kemény F, Landerl K. Phonology-independent general orthographic knowledge. Q J Exp Psychol (Hove) 2021; 74:2075-2083. [PMID: 34009059 PMCID: PMC8531945 DOI: 10.1177/17470218211018438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
While reading is among the most important and well-researched topics of
developmental psychology, sublexical regularities and how these
regularities relate to reading skills have attracted less interest so
far. This study tested general orthographic knowledge (GOK) using an
indirect reaction time (RT)-based task, in which participants had to
detect letters appearing within frequent and infrequent letter
clusters. The aim of the method was to minimise the roles of
phonological activation and metalinguistic decision. Three different
age-groups of German-speaking individuals were tested: first graders
(N = 60), third graders
(N = 68), and adults (N = 44).
Orthographic regularity affected RTs in all three groups, with
significantly lower RTs for frequent than for infrequent clusters. The
indirect measure of GOK did not show an association with reading
measures in first graders and adults, but in the case of third graders
it explained variance over and above age and phonological skills. This
study provides evidence for phonology-independent GOK, at least in
third graders.
Collapse
Affiliation(s)
- Ferenc Kemény
- Institute of Psychology,
University of Graz, Graz, Austria
- Ferenc Kemény, Institute of
Psychology, University of Graz, Universitätsplatz 2, DG, 8010 Graz,
Austria.
| | - Karin Landerl
- Institute of Psychology,
University of Graz, Graz, Austria
- BioTechMed-Graz, Graz,
Austria
- Department of Cognitive Science,
Macquarie University, Sydney, NSW, Australia
| |
Collapse
|
6
|
Poncet M, Fabre‐Thorpe M, Chakravarthi R. A simple rule to describe interactions between visual categories. Eur J Neurosci 2020; 52:4639-4666. [DOI: 10.1111/ejn.14890] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2019] [Revised: 06/12/2020] [Accepted: 06/24/2020] [Indexed: 11/27/2022]
Affiliation(s)
- Marlene Poncet
- CerCo Université de ToulouseCNRSUPS Toulouse France
- School of Psychology University of St Andrews St Andrews UK
| | | | | |
Collapse
|
7
|
Spilcke-Liss J, Zhu J, Gluth S, Spezio M, Gläscher J. Semantic Incongruency Interferes With Endogenous Attention in Cross-Modal Integration of Semantically Congruent Objects. Front Integr Neurosci 2019; 13:53. [PMID: 31572138 PMCID: PMC6749080 DOI: 10.3389/fnint.2019.00053] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2019] [Accepted: 08/29/2019] [Indexed: 12/04/2022] Open
Abstract
Efficient multisensory integration is often influenced by other cognitive processes including, but not limited to, semantic congruency and focused endogenous attention. Semantic congruency can re-allocate processing resources to the location of a congruent stimulus, while attention can prioritize the integration of multi-sensory stimuli under focus. Here, we explore the robustness of this phenomenon in the context of three stimuli, two of which are in the focus of endogenous attention. Participants completed an endogenous attention task with a stimulus compound consisting of 3 different objects: (1) a visual object (V) in the foreground, (2) an auditory object (A), and (3) a visual background scene object (B). Three groups of participants focused their attention on either the visual object and auditory sound (Group VA, n = 30), the visual object and the background (VB, n = 27), or the auditory sound and the background (AB, n = 30), and judged the semantic congruency of the objects under focus. Congruency varied systematically across all 3 stimuli: All stimuli could be semantically incongruent (e.g., V, ambulance; A, church bell; and B, swimming-pool) or all could be congruent (e.g., V, lion; A, roar; and B, savannah), or two objects could be congruent with the remaining one incongruent to the other two (e.g., V, duck; A, quack; and B, phone booth). Participants exhibited a distinct pattern of errors: when participants attended two congruent objects (e.g., group VA: V, lion; A, roar), in the presence of an unattended, incongruent third object (e.g., B, bath room) they tended to make more errors than in any other stimulus combination. Drift diffusion modeling of the behavioral data revealed a significantly smaller drift rate in two-congruent-attended condition, indicating slower evidence accumulation, which was likely due to interference from the unattended, incongruent object. Interference with evidence accumulation occurred independently of which pair of objects was in the focus of attention, which suggests that the vulnerability of congruency judgments to incongruent unattended distractors is not affected by sensory modalities. A control analysis ruled out the simple explanation of a negative response bias. These findings implicate that our perceptual system is highly sensitive to semantic incongruencies even when they are not endogenously attended.
Collapse
Affiliation(s)
- Julia Spilcke-Liss
- Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| | - Jun Zhu
- Department of Computer Science, Tsinghua University, Beijing, China
| | - Sebastian Gluth
- Department of Psychology, University of Basel, Basel, Switzerland
| | - Michael Spezio
- Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.,Psychology and Neuroscience, Scripps College, Claremont, CA, United States
| | - Jan Gläscher
- Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany
| |
Collapse
|
8
|
Eördegh G, Őze A, Bodosi B, Puszta A, Pertich Á, Rosu A, Godó G, Nagy A. Multisensory guided associative learning in healthy humans. PLoS One 2019; 14:e0213094. [PMID: 30861023 PMCID: PMC6413907 DOI: 10.1371/journal.pone.0213094] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/07/2018] [Accepted: 02/14/2019] [Indexed: 12/15/2022] Open
Abstract
Associative learning is a basic cognitive function by which discrete and often different percepts are linked together. The Rutgers Acquired Equivalence Test investigates a specific kind of associative learning, visually guided equivalence learning. The test consists of an acquisition (pair learning) and a test (rule transfer) phase, which are associated primarily with the function of the basal ganglia and the hippocampi, respectively. Earlier studies described that both fundamentally-involved brain structures in the visual associative learning, the basal ganglia and the hippocampi, receive not only visual but also multisensory information. However, no study has investigated whether there is a priority for multisensory guided equivalence learning compared to unimodal ones. Thus we had no data about the modality-dependence or independence of the equivalence learning. In the present study, we have therefore introduced the auditory- and multisensory (audiovisual)-guided equivalence learning paradigms and investigated the performance of 151 healthy volunteers in the visual as well as in the auditory and multisensory paradigms. Our results indicated that visual, auditory and multisensory guided associative learning is similarly effective in healthy humans, which suggest that the acquisition phase is fairly independent from the modality of the stimuli. On the other hand, in the test phase, where participants were presented with acquisitions that were learned earlier and associations that were until then not seen or heard but predictable, the multisensory stimuli elicited the best performance. The test phase, especially its generalization part, seems to be a harder cognitive task, where the multisensory information processing could improve the performance of the participants.
Collapse
Affiliation(s)
- Gabriella Eördegh
- Department of Operative and Esthetic Dentistry, Faculty of Dentistry, University of Szeged, Szeged, Hungary
| | - Attila Őze
- Department of Physiology, Faculty of Medicine, University of Szeged, Szeged, Hungary
| | - Balázs Bodosi
- Department of Physiology, Faculty of Medicine, University of Szeged, Szeged, Hungary
| | - András Puszta
- Department of Physiology, Faculty of Medicine, University of Szeged, Szeged, Hungary
| | - Ákos Pertich
- Department of Physiology, Faculty of Medicine, University of Szeged, Szeged, Hungary
| | - Anett Rosu
- Department of Psychiatry, Faculty of Medicine, University of Szeged, Szeged, Hungary
| | - György Godó
- Csongrád County Health Care Center, Psychiatric Outpatient Care, Hódmezővásárhely, Hungary
| | - Attila Nagy
- Department of Physiology, Faculty of Medicine, University of Szeged, Szeged, Hungary
- * E-mail:
| |
Collapse
|