1
|
Szaszkó B, Habeler M, Forstinger M, Pomper U, Scheftner M, Stolte M, Grüner M, Ansorge U. 10 Hz rhythmic stimulation modulates electrophysiological, but not behavioral markers of suppression. Front Psychol 2024; 15:1376664. [PMID: 38831943 PMCID: PMC11144928 DOI: 10.3389/fpsyg.2024.1376664] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Accepted: 04/08/2024] [Indexed: 06/05/2024] Open
Abstract
We investigated the role of alpha in the suppression of attention capture by salient but to-be-suppressed (negative and nonpredictive) color cues, expecting a potential boosting effect of alpha-rhythmic entrainment on feature-specific cue suppression. We did so by presenting a rhythmically flickering visual bar of 10 Hz before the cue - either on the cue's side or opposite the cue -while an arrhythmically flickering visual bar was presented on the respective other side. We hypothesized that rhythmic entrainment at cue location could enhance the suppression of the cue. Testing 27 participants ranging from 18 to 39 years of age, we found both behavioral and electrophysiological evidence of suppression: Search times for a target at a negatively cued location were delayed relative to a target away from the cued location (inverse validity effects). In addition, an event-related potential indicative for suppression (the Distractor Positivity, Pd) was observed following rhythmic but not arrhythmic stimulation, indicating that suppression was boosted by the stimulation. This was also echoed in higher spectral power and intertrial phase coherence of EEG at rhythmically versus arrhythmically stimulated electrode sites, albeit only at the second harmonic (20 Hz), but not at the stimulation frequency. In addition, inverse validity effects were not modulated by rhythmic entrainment congruent with the cue side. Hence, we propose that rhythmic visual stimulation in the alpha range could support suppression, though behavioral evidence remains elusive, in contrast to electrophysiological findings.
Collapse
Affiliation(s)
- Bence Szaszkó
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
| | - Martin Habeler
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
| | - Marlene Forstinger
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
| | - Ulrich Pomper
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
| | - Manuel Scheftner
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
| | - Moritz Stolte
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
| | - Markus Grüner
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
| | - Ulrich Ansorge
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Vienna, Austria
- Vienna Cognitive Science Hub, University of Vienna, Vienna, Austria
- Research Platform Mediatised Lifeworlds, University of Vienna, Vienna, Austria
| |
Collapse
|
2
|
Gehmacher Q, Schubert J, Schmidt F, Hartmann T, Reisinger P, Rösch S, Schwarz K, Popov T, Chait M, Weisz N. Eye movements track prioritized auditory features in selective attention to natural speech. Nat Commun 2024; 15:3692. [PMID: 38693186 PMCID: PMC11063150 DOI: 10.1038/s41467-024-48126-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2023] [Accepted: 04/22/2024] [Indexed: 05/03/2024] Open
Abstract
Over the last decades, cognitive neuroscience has identified a distributed set of brain regions that are critical for attention. Strong anatomical overlap with brain regions critical for oculomotor processes suggests a joint network for attention and eye movements. However, the role of this shared network in complex, naturalistic environments remains understudied. Here, we investigated eye movements in relation to (un)attended sentences of natural speech. Combining simultaneously recorded eye tracking and magnetoencephalographic data with temporal response functions, we show that gaze tracks attended speech, a phenomenon we termed ocular speech tracking. Ocular speech tracking even differentiates a target from a distractor in a multi-speaker context and is further related to intelligibility. Moreover, we provide evidence for its contribution to neural differences in speech processing, emphasizing the necessity to consider oculomotor activity in future research and in the interpretation of neural differences in auditory cognition.
Collapse
Affiliation(s)
- Quirin Gehmacher
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Salzburg, Austria.
| | - Juliane Schubert
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Salzburg, Austria
| | - Fabian Schmidt
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Salzburg, Austria
| | - Thomas Hartmann
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Salzburg, Austria
| | - Patrick Reisinger
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Salzburg, Austria
| | - Sebastian Rösch
- Department of Otorhinolaryngology, Head and Neck Surgery, Paracelsus Medical University Salzburg, 5020, Salzburg, Austria
| | | | - Tzvetan Popov
- Methods of Plasticity Research, Department of Psychology, University of Zurich, CH-8050, Zurich, Switzerland
- Department of Psychology, University of Konstanz, DE- 78464, Konstanz, Germany
| | - Maria Chait
- Ear Institute, University College London, London, UK
| | - Nathan Weisz
- Paris-Lodron-University of Salzburg, Department of Psychology, Centre for Cognitive Neuroscience, Salzburg, Austria
- Neuroscience Institute, Christian Doppler University Hospital, Paracelsus Medical University, Salzburg, Austria
| |
Collapse
|
3
|
Rotaru I, Geirnaert S, Heintz N, Van de Ryck I, Bertrand A, Francart T. What are we reallydecoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention. J Neural Eng 2024; 21:016017. [PMID: 38266281 DOI: 10.1088/1741-2552/ad2214] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Accepted: 01/24/2024] [Indexed: 01/26/2024]
Abstract
Objective.Spatial auditory attention decoding (Sp-AAD) refers to the task of identifying the direction of the speaker to which a person is attending in a multi-talker setting, based on the listener's neural recordings, e.g. electroencephalography (EEG). The goal of this study is to thoroughly investigate potential biases when training such Sp-AAD decoders on EEG data, particularly eye-gaze biases and latent trial-dependent confounds, which may result in Sp-AAD models that decode eye-gaze or trial-specific fingerprints rather than spatial auditory attention.Approach.We designed a two-speaker audiovisual Sp-AAD protocol in which the spatial auditory and visual attention were enforced to be either congruent or incongruent, and we recorded EEG data from sixteen participants undergoing several trials recorded at distinct timepoints. We trained a simple linear model for Sp-AAD based on common spatial patterns filters in combination with either linear discriminant analysis (LDA) or k-means clustering, and evaluated them both across- and within-trial.Main results.We found that even a simple linear Sp-AAD model is susceptible to overfitting to confounding signal patterns such as eye-gaze and trial fingerprints (e.g. due to feature shifts across trials), resulting in artificially high decoding accuracies. Furthermore, we found that changes in the EEG signal statistics across trials deteriorate the trial generalization of the classifier, even when the latter is retrained on the test trial with an unsupervised algorithm.Significance.Collectively, our findings confirm that there exist subtle biases and confounds that can strongly interfere with the decoding of spatial auditory attention from EEG. It is expected that more complicated non-linear models based on deep neural networks, which are often used for Sp-AAD, are even more vulnerable to such biases. Future work should perform experiments and model evaluations that avoid and/or control for such biases in Sp-AAD tasks.
Collapse
Affiliation(s)
- Iustina Rotaru
- Department of Neurosciences, ExpORL, KU Leuven, Herestraat 49 bus 721, B-3000 Leuven, Belgium
- Department of Electrical Engineering (ESAT), Stadius Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium
| | - Simon Geirnaert
- Department of Neurosciences, ExpORL, KU Leuven, Herestraat 49 bus 721, B-3000 Leuven, Belgium
- Department of Electrical Engineering (ESAT), Stadius Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium
- Leuven.AI-KU Leuven Institute for AI, Leuven, Belgium
| | - Nicolas Heintz
- Department of Neurosciences, ExpORL, KU Leuven, Herestraat 49 bus 721, B-3000 Leuven, Belgium
- Department of Electrical Engineering (ESAT), Stadius Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium
- Leuven.AI-KU Leuven Institute for AI, Leuven, Belgium
| | - Iris Van de Ryck
- Department of Neurosciences, ExpORL, KU Leuven, Herestraat 49 bus 721, B-3000 Leuven, Belgium
| | - Alexander Bertrand
- Department of Electrical Engineering (ESAT), Stadius Center for Dynamical Systems, Signal Processing and Data Analytics, KU Leuven, Kasteelpark Arenberg 10, B-3001 Leuven, Belgium
- Leuven.AI-KU Leuven Institute for AI, Leuven, Belgium
| | - Tom Francart
- Department of Neurosciences, ExpORL, KU Leuven, Herestraat 49 bus 721, B-3000 Leuven, Belgium
- Leuven.AI-KU Leuven Institute for AI, Leuven, Belgium
| |
Collapse
|
4
|
Szaszkó B, Schmid RR, Pomper U, Maiworm M, Laiber S, Tschenett H, Nater UM, Ansorge U. The influence of hatha yoga on stress, anxiety, and suppression: A randomized controlled trial. Acta Psychol (Amst) 2023; 241:104075. [PMID: 37931334 DOI: 10.1016/j.actpsy.2023.104075] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2023] [Accepted: 10/30/2023] [Indexed: 11/08/2023] Open
Abstract
Engaging in yoga may mitigate stress and anxiety in individuals while potentially enhancing one's capacity to manage distractions. Our research aimed to explore the relation between these two outcomes: Can an eight-week yoga program foster distraction suppression, thereby reducing stress and discomfort? To answer this question, we used Hatha Yoga, the most commonly practiced form of yoga. We tested if the intervention improved participants' ability to suppress distractions and selectively decrease self-reported stress and stress reactivity. In Addition, we investigated whether such an intervention would increase participants' mindfulness. Our study included 98 healthy yoga novices between 18 and 40 years who were randomly assigned to either an experimental or a waitlist condition, with each participant completing pre- and post-intervention assessments, including questionnaires, as well as electrophysiological and behavioral measures. After eight weeks of yoga practice, significant reductions in self-reported stress and stress reactivity levels, as well as increased mindfulness, were observed among those participating in the intervention relative to those in the waitlist control group. There were, however, no significant changes in state or trait anxiety due to the intervention. Changes in stress measures could not be explained by changes in participants' ability to suppress distractors, which was not affected by the intervention. Overall, our findings suggest that regular participation in Hatha Yoga can improve mental health outcomes without impacting cognitive functioning directly related to distractor suppression. CLINICAL TRIAL REGISTRATION NUMBER: NCT05232422.
Collapse
Affiliation(s)
- Bence Szaszkó
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria.
| | - Rebecca Rosa Schmid
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria
| | - Ulrich Pomper
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria
| | - Mira Maiworm
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria
| | - Sophia Laiber
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria
| | - Hannah Tschenett
- Department of Clinical and Health Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria; University Research Platform "The Stress of Life - Processes and Mechanisms Underlying Everyday Life Stress", University of Vienna, Liebiggasse 5, 1010 Vienna, Austria
| | - Urs Markus Nater
- Department of Clinical and Health Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria; University Research Platform "The Stress of Life - Processes and Mechanisms Underlying Everyday Life Stress", University of Vienna, Liebiggasse 5, 1010 Vienna, Austria
| | - Ulrich Ansorge
- Department of Cognition, Emotion, and Methods in Psychology, University of Vienna, Liebiggasse 5, 1010 Vienna, Austria; Vienna Cognitive Science Hub, University of Vienna, Kolingasse 14-16, 1090 Vienna, Austria; Research Platform Mediatised Lifeworlds, University of Vienna, Spitalgasse 2-4, 1090 Vienna, Austria
| |
Collapse
|
5
|
Best V, Boyd AD, Sen K. An Effect of Gaze Direction in Cocktail Party Listening. Trends Hear 2023; 27:23312165231152356. [PMID: 36691678 PMCID: PMC9896088 DOI: 10.1177/23312165231152356] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/05/2022] [Revised: 11/18/2022] [Accepted: 01/04/2023] [Indexed: 01/25/2023] Open
Abstract
It is well established that gaze direction can influence auditory spatial perception, but the implications of this interaction for performance in complex listening tasks is unclear. In the current study, we investigated whether there is a measurable effect of gaze direction on speech intelligibility in a "cocktail party" listening situation. We presented sequences of digits from five loudspeakers positioned at 0°, ± 15°, and ± 30° azimuth, and asked participants to repeat back the digits presented from a designated target loudspeaker. In different blocks of trials, the participant visually fixated on a cue presented at the target location or at a nontarget location. Eye position was tracked continuously to monitor compliance. Performance was best when fixation was on-target (vs. off-target) and the size of this effect depended on the specific configuration. This result demonstrates an influence of gaze direction in multitalker mixtures, even in the absence of visual speech information.
Collapse
Affiliation(s)
- Virginia Best
- Department of Speech, Language and Hearing Sciences,
Boston
University, Boston, MA, USA
| | - Alex D. Boyd
- Department of Biomedical Engineering,
Boston
University, Boston, MA, USA
| | - Kamal Sen
- Department of Biomedical Engineering,
Boston
University, Boston, MA, USA
| |
Collapse
|
6
|
Skoglund MA, Andersen M, Shiell MM, Keidser G, Rank ML, Rotger-Griful S. Comparing In-ear EOG for Eye-Movement Estimation With Eye-Tracking: Accuracy, Calibration, and Speech Comprehension. Front Neurosci 2022; 16:873201. [PMID: 35844213 PMCID: PMC9279575 DOI: 10.3389/fnins.2022.873201] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2022] [Accepted: 05/25/2022] [Indexed: 12/02/2022] Open
Abstract
This presentation details and evaluates a method for estimating the attended speaker during a two-person conversation by means of in-ear electro-oculography (EOG). Twenty-five hearing-impaired participants were fitted with molds equipped with EOG electrodes (in-ear EOG) and wore eye-tracking glasses while watching a video of two life-size people in a dialog solving a Diapix task. The dialogue was directionally presented and together with background noise in the frontal hemisphere at 60 dB SPL. During three conditions of steering (none, in-ear EOG, conventional eye-tracking), participants' comprehension was periodically measured using multiple-choice questions. Based on eye movement detection by in-ear EOG or conventional eye-tracking, the estimated attended speaker was amplified by 6 dB. In the in-ear EOG condition, the estimate was based on one selected channel pair of electrodes out of 36 possible electrodes. A novel calibration procedure introducing three different metrics was used to select the measurement channel. The in-ear EOG attended speaker estimates were compared to those of the eye-tracker. Across participants, the mean accuracy of in-ear EOG estimation of the attended speaker was 68%, ranging from 50 to 89%. Based on offline simulation, it was established that higher scoring metrics obtained for a channel with the calibration procedure were significantly associated with better data quality. Results showed a statistically significant improvement in comprehension of about 10% in both steering conditions relative to the no-steering condition. Comprehension in the two steering conditions was not significantly different. Further, better comprehension obtained under the in-ear EOG condition was significantly correlated with more accurate estimation of the attended speaker. In conclusion, this study shows promising results in the use of in-ear EOG for visual attention estimation with potential for applicability in hearing assistive devices.
Collapse
Affiliation(s)
- Martin A. Skoglund
- Division of Automatic Control, Department of Electrical Engineering, The Institute of Technology, Linköping University, Linkoping, Sweden
- Eriksholm Research Centre, Part of Oticon A/S, Snekkersten, Denmark
- *Correspondence: Martin A. Skoglund
| | | | - Martha M. Shiell
- Eriksholm Research Centre, Part of Oticon A/S, Snekkersten, Denmark
| | - Gitte Keidser
- Eriksholm Research Centre, Part of Oticon A/S, Snekkersten, Denmark
- Department of Behavioral Sciences and Learning, Linneaus Centre Head, Linköping University, Linkoping, Sweden
| | | | | |
Collapse
|
7
|
Dennis S. Safe and sound: listening to Guns N’ Roses in the car. SOCIAL ANTHROPOLOGY 2021. [DOI: 10.1111/1469-8676.13105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Affiliation(s)
- Simone Dennis
- School of Archaeology and Anthropology Australian National University CanberraACT 0200Australia
| |
Collapse
|
8
|
Saurels BW, Hohaia W, Yarrow K, Johnston A, Arnold DH. Visual predictions, neural oscillations and naïve physics. Sci Rep 2021; 11:16127. [PMID: 34373486 PMCID: PMC8352981 DOI: 10.1038/s41598-021-95295-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/12/2020] [Accepted: 06/29/2021] [Indexed: 11/09/2022] Open
Abstract
Prediction is a core function of the human visual system. Contemporary research suggests the brain builds predictive internal models of the world to facilitate interactions with our dynamic environment. Here, we wanted to examine the behavioural and neurological consequences of disrupting a core property of peoples’ internal models, using naturalistic stimuli. We had people view videos of basketball and asked them to track the moving ball and predict jump shot outcomes, all while we recorded eye movements and brain activity. To disrupt people’s predictive internal models, we inverted footage on half the trials, so dynamics were inconsistent with how movements should be shaped by gravity. When viewing upright videos people were better at predicting shot outcomes, at tracking the ball position, and they had enhanced alpha-band oscillatory activity in occipital brain regions. The advantage for predicting upright shot outcomes scaled with improvements in ball tracking and occipital alpha-band activity. Occipital alpha-band activity has been linked to selective attention and spatially-mapped inhibitions of visual brain activity. We propose that when people have a more accurate predictive model of the environment, they can more easily parse what is relevant, allowing them to better target irrelevant positions for suppression—resulting in both better predictive performance and in neural markers of inhibited information processing.
Collapse
Affiliation(s)
- Blake W Saurels
- School of Psychology, The University of Queensland, Brisbane, Australia.
| | - Wiremu Hohaia
- School of Psychology, The University of Queensland, Brisbane, Australia
| | - Kielan Yarrow
- Department of Psychology, City, University of London, London, UK
| | - Alan Johnston
- School of Psychology, University of Nottingham, Nottingham, UK
| | - Derek H Arnold
- School of Psychology, The University of Queensland, Brisbane, Australia
| |
Collapse
|
9
|
Cheng L, Guo ZY, Qu YL. Cross-modality modulation of auditory midbrain processing of intensity information. Hear Res 2020; 395:108042. [PMID: 32810721 DOI: 10.1016/j.heares.2020.108042] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/03/2020] [Revised: 06/12/2020] [Accepted: 07/08/2020] [Indexed: 02/03/2023]
Abstract
In nature, animals constantly receive a multitude of sensory stimuli, such as visual, auditory, and somatosensory. The integration across sensory modalities is advantageous for the precise processing of sensory inputs which is essential for animals to survival. Although some principles of cross-modality integration have been revealed by many studies, little insight has been gained into its functional potentials. In this study, the functional influence of cross-modality modulation on auditory processing of intensity information was investigated via recording neuronal activity in the auditory midbrain (i.e., inferior colliculus, IC) under the conditions of visual, auditory, and audiovisual stimuli, respectively. Results demonstrated that combined audiovisual stimuli either enhanced or suppressed the responses of IC neurons compared to auditory stimuli alone, even though the same visual stimuli alone induced no response. Audiovisual modulation appeared to be strongest when the combined audiovisual stimuli were located at the best auditory azimuth of neurons as well as when presented with intensity at near-threshold levels. Additionally, the rate-intensity function of IC neurons to auditory stimuli was expanded or compressed by audiovisual modulation, which was highly dependent on the minimal threshold (MT) of neurons. Lowering of the MT and greater audiovisual modulation for the neuron indicated an intensity-specific enhancement of auditory intensity sensitivity by cross-modality modulation. Overall, evidence suggests a potential functional role of cross-modality modulation in IC that serves to instruct adaptive plasticity to enhance the auditory perception of intensity information.
Collapse
Affiliation(s)
- Liang Cheng
- School of Psychology & Key Laboratory of Adolescent Cyberpsycology and Behavior (CCNU) of Ministry of Education, Central China Normal University, Wuhan, 430079, China; School of Life Sciences & Hubei Key Lab of Genetic Regulation and Integrative Biology, Central China Normal University, Wuhan, 430079, China.
| | - Zhao-Yang Guo
- School of Psychology & Key Laboratory of Adolescent Cyberpsycology and Behavior (CCNU) of Ministry of Education, Central China Normal University, Wuhan, 430079, China
| | - Yi-Li Qu
- School of Psychology & Key Laboratory of Adolescent Cyberpsycology and Behavior (CCNU) of Ministry of Education, Central China Normal University, Wuhan, 430079, China
| |
Collapse
|
10
|
Bosker HR, Sjerps MJ, Reinisch E. Temporal contrast effects in human speech perception are immune to selective attention. Sci Rep 2020; 10:5607. [PMID: 32221376 PMCID: PMC7101381 DOI: 10.1038/s41598-020-62613-8] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2018] [Accepted: 03/16/2020] [Indexed: 11/09/2022] Open
Abstract
Two fundamental properties of perception are selective attention and perceptual contrast, but how these two processes interact remains unknown. Does an attended stimulus history exert a larger contrastive influence on the perception of a following target than unattended stimuli? Dutch listeners categorized target sounds with a reduced prefix "ge-" marking tense (e.g., ambiguous between gegaan-gaan "gone-go"). In 'single talker' Experiments 1-2, participants perceived the reduced syllable (reporting gegaan) when the target was heard after a fast sentence, but not after a slow sentence (reporting gaan). In 'selective attention' Experiments 3-5, participants listened to two simultaneous sentences from two different talkers, followed by the same target sounds, with instructions to attend only one of the two talkers. Critically, the speech rates of attended and unattended talkers were found to equally influence target perception - even when participants could watch the attended talker speak. In fact, participants' target perception in 'selective attention' Experiments 3-5 did not differ from participants who were explicitly instructed to divide their attention equally across the two talkers (Experiment 6). This suggests that contrast effects of speech rate are immune to selective attention, largely operating prior to attentional stream segregation in the auditory processing hierarchy.
Collapse
Affiliation(s)
- Hans Rutger Bosker
- Max Planck Institute for Psycholinguistics, PO Box 310, 6500 AH, Nijmegen, The Netherlands.
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands.
| | - Matthias J Sjerps
- Max Planck Institute for Psycholinguistics, PO Box 310, 6500 AH, Nijmegen, The Netherlands
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| | - Eva Reinisch
- Institute of Phonetics and Speech Processing, Ludwig Maximilian University Munich, Munich, Germany
- Acoustics Research Institute, Austrian Academy of Sciences, Vienna, Austria
| |
Collapse
|
11
|
Sakamaki I, Adams K, Tavakoli M, Wiebe SA. Effect of feedback and target size on eye gaze accuracy in an off-screen task. Disabil Rehabil Assist Technol 2020; 16:769-779. [PMID: 32100583 DOI: 10.1080/17483107.2020.1729874] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/24/2022]
Abstract
PURPOSE Eye gaze interfaces have been used by people with severe physical impairment to interact with various assistive technologies. If used to control robots, it would be beneficial if individuals could gaze directly at targets in the physical environment rather than have to switch their gaze between a screen with representations of robot commands and the physical environment to see the response of their selection. By using a homogeneous transformation technique, eye gaze coordinates can be mapped between the reference coordinate frame of eye tracker and the coordinate frame of objects in the physical environment. Feedback about where the eye tracker has determined the eye gaze is fixated is needed so users can select targets more accurately. Screen-based assistive technologies can use visual feedback, but in a physical environment, other forms of feedback need to be examined. MATERIALS AND METHODS In this study, an eye gaze system with different feedback conditions (i.e., visual, auditory, vibrotactile, and no-feedback) was tested when participants received visual feedback on a display (on-screen) and when looking directly at the physical environment (off-screen). Target selection tasks in both screen conditions were performed by ten non-disabled adults, three non-disabled children, and two adults and one child with cerebral palsy. RESULTS Tasks performed with gaze fixation feedback modalities were accomplished faster and with higher success than tasks performed without feedback, and similar results were observed in both screen conditions. No significant difference was observed in performance across the feedback modalities, but participants had personal preferences. CONCLUSION The homogeneous transformation technique enabled the use of a stationary eye tracker to select target objects in the physical environment, and auditory and vibrotactile feedback enabled participants to be more accurate selecting targets than without it.Implications for RehabilitationBeing able to select target objects in the physical environment by eye gaze could make it easier for children with disabilities to control assistive robots, because in this way they do not have to change their focus between a computer screen with commands and the robot.Providing auditory or vibrotactile feedback when using an eye gaze system made it faster and easier to know if a target was being gazed upon.Being able to select targets in the environment using eye gaze could be beneficial for other assistive technology, too, such as destination selection for power wheelchairs.
Collapse
Affiliation(s)
- Isao Sakamaki
- Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB, Canada
| | - Kim Adams
- Faculty of Rehabilitation Medicine, University of Alberta, Edmonton, AB, Canada
| | - Mahdi Tavakoli
- Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada
| | - Sandra A Wiebe
- Department of Psychology, University of Alberta, Edmonton, AB, Canada
| |
Collapse
|
12
|
Šabić E, Henning D, Myüz H, Morrow A, Hout MC, MacDonald JA. Examining the Role of Eye Movements During Conversational Listening in Noise. Front Psychol 2020; 11:200. [PMID: 32116975 PMCID: PMC7033431 DOI: 10.3389/fpsyg.2020.00200] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2019] [Accepted: 01/28/2020] [Indexed: 12/02/2022] Open
Abstract
Speech comprehension is often thought of as an entirely auditory process, but both normal hearing and hearing-impaired individuals sometimes use visual attention to disambiguate speech, particularly when it is difficult to hear. Many studies have investigated how visual attention (or the lack thereof) impacts the perception of simple speech sounds such as isolated consonants, but there is a gap in the literature concerning visual attention during natural speech comprehension. This issue needs to be addressed, as individuals process sounds and words in everyday speech differently than when they are separated into individual elements with no competing sound sources or noise. Moreover, further research is needed to explore patterns of eye movements during speech comprehension – especially in the presence of noise – as such an investigation would allow us to better understand how people strategically use visual information while processing speech. To this end, we conducted an experiment to track eye-gaze behavior during a series of listening tasks as a function of the number of speakers, background noise intensity, and the presence or absence of simulated hearing impairment. Our specific aims were to discover how individuals might adapt their oculomotor behavior to compensate for the difficulty of the listening scenario, such as when listening in noisy environments or experiencing simulated hearing loss. Speech comprehension difficulty was manipulated by simulating hearing loss and varying background noise intensity. Results showed that eye movements were affected by the number of speakers, simulated hearing impairment, and the presence of noise. Further, findings showed that differing levels of signal-to-noise ratio (SNR) led to changes in eye-gaze behavior. Most notably, we found that the addition of visual information (i.e. videos vs. auditory information only) led to enhanced speech comprehension – highlighting the strategic usage of visual information during this process.
Collapse
Affiliation(s)
- Edin Šabić
- Hearing Enhancement and Augmented Reality Lab, Department of Psychology, New Mexico State University, Las Cruces, NM, United States
| | - Daniel Henning
- Hearing Enhancement and Augmented Reality Lab, Department of Psychology, New Mexico State University, Las Cruces, NM, United States
| | - Hunter Myüz
- Hearing Enhancement and Augmented Reality Lab, Department of Psychology, New Mexico State University, Las Cruces, NM, United States
| | - Audrey Morrow
- Hearing Enhancement and Augmented Reality Lab, Department of Psychology, New Mexico State University, Las Cruces, NM, United States
| | - Michael C Hout
- Hearing Enhancement and Augmented Reality Lab, Department of Psychology, New Mexico State University, Las Cruces, NM, United States
| | - Justin A MacDonald
- Hearing Enhancement and Augmented Reality Lab, Department of Psychology, New Mexico State University, Las Cruces, NM, United States
| |
Collapse
|
13
|
Favre-Félix A, Graversen C, Bhuiyan TA, Skoglund MA, Rotger-Griful S, Rank ML, Dau T, Lunner T. Absolute Eye Gaze Estimation With Biosensors in Hearing Aids. Front Neurosci 2019; 13:1294. [PMID: 31920477 PMCID: PMC6915090 DOI: 10.3389/fnins.2019.01294] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/03/2019] [Accepted: 11/15/2019] [Indexed: 11/13/2022] Open
Abstract
People with hearing impairment typically have difficulties following conversations in multi-talker situations. Previous studies have shown that utilizing eye gaze to steer audio through beamformers could be a solution for those situations. Recent studies have shown that in-ear electrodes that capture electrooculography in the ear (EarEOG) can estimate the eye-gaze relative to the head, when the head was fixed. The head movement can be estimated using motion sensors around the ear to create an estimate of the absolute eye-gaze in the room. In this study, an experiment was designed to mimic a multi-talker situation in order to study and model the EarEOG signal when participants attempted to follow a conversation. Eleven hearing impaired participants were presented speech from the DAT speech corpus (Bo Nielsen et al., 2014), with three targets positioned at -30°, 0° and +30° azimuth. The experiment was run in two setups: one where the participants had their head fixed in a chinrest, and the other where they were free to move their head. The participants' task was to focus their visual attention on an LED-indicated target that changed regularly. A model was developed for the relative eye-gaze estimation, taking saccades, fixations, head movement and drift from the electrode-skin half-cell into account. This model explained 90.5% of the variance of the EarEOG when the head was fixed, and 82.6% when the head was free. The absolute eye-gaze was also estimated utilizing that model. When the head was fixed, the estimation of the absolute eye-gaze was reliable. However, due to hardware issues, the estimation of the absolute eye-gaze when the head was free had a variance that was too large to reliably estimate the attended target. Overall, this study demonstrated the potential of estimating absolute eye-gaze using EarEOG and motion sensors around the ear.
Collapse
Affiliation(s)
- Antoine Favre-Félix
- Eriksholm Research Centre, Snekkersten, Denmark.,Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Lyngby, Denmark
| | | | | | - Martin A Skoglund
- Eriksholm Research Centre, Snekkersten, Denmark.,Division of Automatic Control, Linköping University, Linköping, Sweden
| | | | | | - Torsten Dau
- Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Lyngby, Denmark
| | - Thomas Lunner
- Eriksholm Research Centre, Snekkersten, Denmark.,Hearing Systems Section, Department of Health Technology, Technical University of Denmark, Lyngby, Denmark.,Division of Automatic Control, Linköping University, Linköping, Sweden
| |
Collapse
|
14
|
Visual input shapes the auditory frequency responses in the inferior colliculus of mouse. Hear Res 2019; 381:107777. [PMID: 31430633 DOI: 10.1016/j.heares.2019.107777] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/16/2019] [Revised: 07/30/2019] [Accepted: 08/02/2019] [Indexed: 11/23/2022]
Abstract
The integration of visual and auditory information is important for humans or animals to build an accurate and coherent perception of the external world. Although some evidence has shown some principles of the audiovisual integration, little insight has been gained into its functional purpose. In this study, we investigated the functional influence of dynamic visual input on auditory frequency processing by recording single unit activity in the central nucleus of the inferior colliculus (ICc). Results showed that the auditory responses of ICc neurons to sound frequencies could be enhanced or suppressed by visual stimuli even though the same visual stimuli induced no neural responses when presented alone. For each ICc neuron, the most effective visual stimuli were located in the same azimuth as for auditory stimuli and preceded for ∼20 ms. Additionally, visual stimuli could steepen or flatten the frequency tuning curves (FTCs) of ICc neurons by various visual effects at each responsive frequency. The modulation degree of auditory FTCs was dependent on the minimal thresholds (MTs) of ICc neurons, i.e., with MTs increasing, the modulation degree decreased. Due to the non-homogeneous distribution of MTs which was lowest at 10 kHz, visual modulation of auditory FTCs exhibited a frequency-specific manner, the closer it reached the characteristic frequency (CF) of 10 kHz, the greater modulation. Thus, visual modulation of auditory frequency responses in ICc is dependent not only on the visual stimulus but also on the auditory characteristics of ICc neurons. These results suggest a moment-to-moment visual modulation of auditory frequency responses that in real time increase auditory frequency sensitivity to audiovisual stimuli. Furthermore, in the long term such modulation could serve to instruct auditory adaptive plasticity to maintain necessary and accurate auditory detection and perceptual behavior.
Collapse
|
15
|
Pomper U, Ditye T, Ansorge U. Contralateral delay activity during temporal order memory. Neuropsychologia 2019; 129:104-116. [PMID: 30922830 DOI: 10.1016/j.neuropsychologia.2019.03.012] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2018] [Revised: 03/15/2019] [Accepted: 03/22/2019] [Indexed: 11/30/2022]
Abstract
In everyday life, we constantly need to remember the temporal sequence of visual events over short periods of time, for example, when making sense of others' actions or watching a movie. While there is increasing knowledge available on neural mechanisms underlying visual working memory (VWM) regarding the identity and spatial location of objects, less is known about how the brain encodes and retains information on temporal sequences. Here, we investigate whether the contralateral-delay activity (CDA), a well-studied electroencephalographic (EEG) component associated with VWM of object identity, also reflects the encoding and retention of temporal order. In two independent experiments, we presented participants with a sequence of four or six images, followed by a 1 s retention period. Participants judged temporal order by indicating whether a subsequently presented probe image was originally displayed during the first or the second half of the sequence. As a main novel result, we report the emergence of a contralateral negativity already following the presentation of the first item of the sequence, which increases over the course of a trial with every presented item, up to a limit of four items. We further observed no differences in the CDA during the temporal-order task compared to one obtained during a task concerning the spatial location of the presented items. Since the characteristics of the CDA appear to be highly similar between different encoded feature dimensions and increases as additional items are being encoded, we suggest this component might be a general characteristic of various types of VWM.
Collapse
Affiliation(s)
- Ulrich Pomper
- Department of Basic Psychological Research and Research Methods, Faculty of Psychology, University of Vienna, Liebiggasse 5, 1010, Vienna, Austria.
| | - Thomas Ditye
- Department of Basic Psychological Research and Research Methods, Faculty of Psychology, University of Vienna, Liebiggasse 5, 1010, Vienna, Austria; Faculty of Psychology, Sigmund Freud University, Freudplatz 1, 1020, Vienna, Austria
| | - Ulrich Ansorge
- Department of Basic Psychological Research and Research Methods, Faculty of Psychology, University of Vienna, Liebiggasse 5, 1010, Vienna, Austria
| |
Collapse
|
16
|
Ishii D, Takeda K, Yamamoto S, Noguchi A, Ishibashi K, Tanamachi K, Yozu A, Kohno Y. Effect of Visuospatial Attention on the Sensorimotor Gating System. Front Behav Neurosci 2019; 13:1. [PMID: 30697155 PMCID: PMC6341199 DOI: 10.3389/fnbeh.2019.00001] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2018] [Accepted: 01/03/2019] [Indexed: 11/13/2022] Open
Abstract
The integration of multiple sensory modalities allows us to adapt to the environment of the outside world. It is widely known that visual stimuli interfere with the processing of auditory information, which is involved in the ability to pay attention. Additionally, visuospatial attention has the characteristic of laterality. It is unclear whether this laterality of visuospatial attention affects the processing of auditory stimuli. The sensorimotor gating system is a neurological process, which filters out unnecessary stimuli from environmental stimuli in the brain. Prepulse inhibition (PPI) is an operational measure of the sensorimotor gating system, which a weaker prestimulus (prepulse), such as a visual stimulus, inhibits the startle reflex elicited by a subsequent robust startling stimulus (pulse) such as a tone. Therefore, we investigated whether the visual stimulus from the left or right visual space affects the sensorimotor gating system in a "rest" task (low attentional condition) and a "selective attention" task (high attentional condition). In the selective attention task, we found that the target prepulse presented in the left and bilateral visual fields suppressed the startle reflex more than that presented in the right visual field. By contrast, there was no laterality of PPI in the no-target prepulse condition, and there was no laterality of PPI in the rest task. These results suggest that the laterality of visuospatial attention affects the sensorimotor gating system depending on the attentional condition. Moreover, the process of visual information processing may differ between the left and right brain.
Collapse
Affiliation(s)
- Daisuke Ishii
- Center for Medical Sciences, Ibaraki Prefectural University of Health Sciences, Ami, Japan.,Department of Cognitive Behavioral Physiology, Chiba University Graduate School of Medicine, Chiba, Japan
| | - Kotaro Takeda
- Faculty of Rehabilitation, School of Health Sciences, Fujita Health University, Toyoake, Japan
| | - Satoshi Yamamoto
- Department of Physical Therapy, School of Healthcare, Ibaraki Prefectural University of Health Sciences, Ami, Japan
| | | | - Kiyoshige Ishibashi
- Department of Physical Therapy, Ibaraki Prefectural University of Health Sciences Hospital, Ami, Japan
| | - Kenya Tanamachi
- Department of Physical Therapy, Ibaraki Prefectural University of Health Sciences Hospital, Ami, Japan
| | - Arito Yozu
- Center for Medical Sciences, Ibaraki Prefectural University of Health Sciences, Ami, Japan
| | - Yutaka Kohno
- Center for Medical Sciences, Ibaraki Prefectural University of Health Sciences, Ami, Japan
| |
Collapse
|
17
|
Fiedler L, Wöstmann M, Herbst SK, Obleser J. Late cortical tracking of ignored speech facilitates neural selectivity in acoustically challenging conditions. Neuroimage 2018; 186:33-42. [PMID: 30367953 DOI: 10.1016/j.neuroimage.2018.10.057] [Citation(s) in RCA: 68] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2018] [Revised: 09/12/2018] [Accepted: 10/21/2018] [Indexed: 11/25/2022] Open
Abstract
Listening requires selective neural processing of the incoming sound mixture, which in humans is borne out by a surprisingly clean representation of attended-only speech in auditory cortex. How this neural selectivity is achieved even at negative signal-to-noise ratios (SNR) remains unclear. We show that, under such conditions, a late cortical representation (i.e., neural tracking) of the ignored acoustic signal is key to successful separation of attended and distracting talkers (i.e., neural selectivity). We recorded and modeled the electroencephalographic response of 18 participants who attended to one of two simultaneously presented stories, while the SNR between the two talkers varied dynamically between +6 and -6 dB. The neural tracking showed an increasing early-to-late attention-biased selectivity. Importantly, acoustically dominant (i.e., louder) ignored talkers were tracked neurally by late involvement of fronto-parietal regions, which contributed to enhanced neural selectivity. This neural selectivity, by way of representing the ignored talker, poses a mechanistic neural account of attention under real-life acoustic conditions.
Collapse
Affiliation(s)
- Lorenz Fiedler
- Department of Psychology, University of Lübeck, Lübeck, Germany.
| | - Malte Wöstmann
- Department of Psychology, University of Lübeck, Lübeck, Germany
| | - Sophie K Herbst
- Department of Psychology, University of Lübeck, Lübeck, Germany
| | - Jonas Obleser
- Department of Psychology, University of Lübeck, Lübeck, Germany.
| |
Collapse
|