1
|
Foley L, Schlesinger J, Schutz M. More detectable, less annoying: Temporal variation in amplitude envelope and spectral content improves auditory interface efficacy. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2022; 151:3189. [PMID: 35649914 DOI: 10.1121/10.0010447] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/02/2021] [Accepted: 04/22/2022] [Indexed: 06/15/2023]
Abstract
Auditory interfaces, such as auditory alarms, are useful tools for human computer interaction. Unfortunately, poor detectability and annoyance inhibit the efficacy of many interface sounds. Here, it is shown in two ways how moving beyond the traditional simplistic temporal structures of normative interface sounds can significantly improve auditory interface efficacy. First, participants rated tones with percussive amplitude envelopes as significantly less annoying than tones with flat amplitude envelopes. Crucially, this annoyance reduction did not come with a detection cost as percussive tones were detected more often than flat tones-particularly, at relatively low listening levels. Second, it was found that reductions in the duration of a tone's harmonics significantly lowered its annoyance without a commensurate reduction in detection. Together, these findings help inform our theoretical understanding of detection and annoyance of sound. In addition, they offer promising original design considerations for auditory interfaces.
Collapse
Affiliation(s)
- Liam Foley
- Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, Canada
| | - Joseph Schlesinger
- Anesthesiology Critical Care Medicine, Biomedical Engineering, Vanderbilt University Medical Center, Nashville, Tennessee 37212, USA
| | - Michael Schutz
- School of the Arts, McMaster University, Hamilton, Canada
| |
Collapse
|
2
|
Wu Z, Bao X, Liu L, Li L. Looming Effects on Attentional Modulation of Prepulse Inhibition Paradigm. Front Psychol 2021; 12:740363. [PMID: 34867622 PMCID: PMC8634448 DOI: 10.3389/fpsyg.2021.740363] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2021] [Accepted: 10/11/2021] [Indexed: 11/13/2022] Open
Abstract
In a hazardous environment, it is fundamentally important to successfully evaluate the motion of sounds. Previous studies demonstrated "auditory looming bias" in both macaques and humans, as looming sounds that increased in intensity were processed preferentially by the brain. In this study on rats, we used a prepulse inhibition (PPI) of the acoustic startle response paradigm to investigate whether auditory looming sound with intrinsic warning value could draw attention of the animals and dampen the startle reflex caused by the startling noise. We showed looming sound with a duration of 120 ms enhanced PPI compared with receding sound with the same duration; however, when both sound types were at shorter duration/higher change rate (i.e., 30 ms) or longer duration/lower rate (i.e., more than 160 ms), there was no PPI difference. This indicates that looming sound-induced PPI enhancement was duration dependent. We further showed that isolation rearing impaired the abilities of animals to differentiate looming and receding prepulse stimuli, although it did not abolish their discrimination between looming and stationary prepulse stimuli. This suggests that isolation rearing compromised their assessment of potential threats from approaching objects and receding objects.
Collapse
Affiliation(s)
- Zhemeng Wu
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| | | | | | - Liang Li
- School of Psychological and Cognitive Sciences, Beijing Key Laboratory of Behavior and Mental Health, Peking University, Beijing, China
| |
Collapse
|
3
|
Slow Resting State Fluctuations Enhance Neuronal and Behavioral Responses to Looming Sounds. Brain Topogr 2021; 35:121-141. [PMID: 33768383 DOI: 10.1007/s10548-021-00826-4] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/15/2020] [Accepted: 02/17/2021] [Indexed: 01/01/2023]
Abstract
We investigate both experimentally and using a computational model how the power of the electroencephalogram (EEG) recorded in human subjects tracks the presentation of sounds with acoustic intensities that increase exponentially (looming) or remain constant (flat). We focus on the link between this EEG tracking response, behavioral reaction times and the time scale of fluctuations in the resting state, which show considerable inter-subject variability. Looming sounds are shown to generally elicit a sustained power increase in the alpha and beta frequency bands. In contrast, flat sounds only elicit a transient upsurge at frequencies ranging from 7 to 45 Hz. Likewise, reaction times (RTs) in an audio-tactile task at different latencies from sound onset also present significant differences between sound types. RTs decrease with increasing looming intensities, i.e. as the sense of urgency increases, but remain constant with stationary flat intensities. We define the reaction time variation or "gain" during looming sound presentation, and show that higher RT gains are associated with stronger correlations between EEG power responses and sound intensity. Higher RT gain further entails higher relative power differences between loom and flat in the alpha and beta bands. The full-width-at-half-maximum of the autocorrelation function of the eyes-closed resting state EEG also increases with RT gain. The effects are topographically located over the central and frontal electrodes. A computational model reveals that the increase in stimulus-response correlation in subjects with slower resting state fluctuations is expected when EEG power fluctuations at each electrode and in a given band are viewed as simple coupled low-pass filtered noise processes jointly driven by the sound intensity. The model assumes that the strength of stimulus-power coupling is proportional to RT gain in different coupling scenarios, suggesting a mechanism by which slower resting state fluctuations enhance EEG response and shorten reaction times.
Collapse
|
4
|
Schutz M, Gillard J. On the generalization of tones: A detailed exploration of non-speech auditory perception stimuli. Sci Rep 2020; 10:9520. [PMID: 32533008 PMCID: PMC7293323 DOI: 10.1038/s41598-020-63132-2] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2019] [Accepted: 03/13/2020] [Indexed: 11/09/2022] Open
Abstract
The dynamic changes in natural sounds’ temporal structures convey important event-relevant information. However, prominent researchers have previously expressed concern that non-speech auditory perception research disproportionately uses simplistic stimuli lacking the temporal variation found in natural sounds. A growing body of work now demonstrates that some conclusions and models derived from experiments using simplistic tones fail to generalize, raising important questions about the types of stimuli used to assess the auditory system. To explore the issue empirically, we conducted a novel, large-scale survey of non-speech auditory perception research from four prominent journals. A detailed analysis of 1017 experiments from 443 articles reveals that 89% of stimuli employ amplitude envelopes lacking the dynamic variations characteristic of non-speech sounds heard outside the laboratory. Given differences in task outcomes and even the underlying perceptual strategies evoked by dynamic vs. invariant amplitude envelopes, this raises important questions of broad relevance to psychologists and neuroscientists alike. This lack of exploration of a property increasingly recognized as playing a crucial role in perception suggests future research using stimuli with time-varying amplitude envelopes holds significant potential for furthering our understanding of the auditory system’s basic processing capabilities.
Collapse
Affiliation(s)
- Michael Schutz
- School of the Arts, McMaster University, Hamilton, Canada. .,Department of Psychology, Neuroscience & Behaviour, McMaster University, Hamilton, Canada.
| | - Jessica Gillard
- Department of Psychology, Neuroscience & Behaviour, McMaster University, Hamilton, Canada
| |
Collapse
|
5
|
Abstract
Hearing is the most accurate sense for perceiving duration. However, rarely it produces inaccurate estimates of duration, for example when it compares the subjective duration of tones that are increasing in intensity over time (i.e., ramped) with that of tones that are decreasing in intensity over time (i.e., damped). The literature reports that the damped tones are perceived as much being shorter than the ramped tones of the same length. The short subjective duration of damped tones may originate from a decay suppression mechanism that parses the source-informative part of many natural sounds (i.e., the beginning) from the less informative part of them (the decay): listeners may interpret the tail of damped tones like an echo or like the decay portion of an impact sound and exclude it from the account of the duration of the tone. In the natural soundscape, the tail of sounds produced in reverberant environments and the tail of impact sounds have a frequency content that is constant throughout the sound's duration. Here, the carriers used for ramped and damped sounds were a tone constant in frequency and a tone modulated in frequency. The frequency modulation was introduced to prevent the listener from interpreting the tail of these tones as the result of reverberation or the decay portion of an impact sound. Frequency constant damped tones were largely underestimated in duration whereas frequency modulated ones were not (or were only slightly), demonstrating that the decay suppression mechanism is a worthy explanation for the short subjective duration of damped tones.
Collapse
|
6
|
Judging Relative Onsets and Offsets of Audiovisual Events. Vision (Basel) 2020; 4:vision4010017. [PMID: 32138261 PMCID: PMC7157228 DOI: 10.3390/vision4010017] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2019] [Revised: 02/15/2020] [Accepted: 02/23/2020] [Indexed: 01/29/2023] Open
Abstract
This study assesses the fidelity with which people can make temporal order judgments (TOJ) between auditory and visual onsets and offsets. Using an adaptive staircase task administered to a large sample of young adults, we find that the ability to judge temporal order varies widely among people, with notable difficulty created when auditory events closely follow visual events. Those findings are interpretable within the context of an independent channels model. Visual onsets and offsets can be difficult to localize in time when they occur within the temporal neighborhood of sound onsets or offsets.
Collapse
|
7
|
Glatz C, Chuang LL. The time course of auditory looming cues in redirecting visuo-spatial attention. Sci Rep 2019; 9:743. [PMID: 30679468 PMCID: PMC6345893 DOI: 10.1038/s41598-018-36033-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2018] [Accepted: 11/14/2018] [Indexed: 11/09/2022] Open
Abstract
By orienting attention, auditory cues can improve the discrimination of spatially congruent visual targets. Looming sounds that increase in intensity are processed preferentially by the brain. Thus, we investigated whether auditory looming cues can orient visuo-spatial attention more effectively than static and receding sounds. Specifically, different auditory cues could redirect attention away from a continuous central visuo-motor tracking task to peripheral visual targets that appeared occasionally. To investigate the time course of crossmodal cuing, Experiment 1 presented visual targets at different time-points across a 500 ms auditory cue's presentation. No benefits were found for simultaneous audio-visual cue-target presentation. The largest crossmodal benefit occurred at early cue-target asynchrony onsets (i.e., CTOA = 250 ms), regardless of auditory cue type, which diminished at CTOA = 500 ms for static and receding cues. However, auditory looming cues showed a late crossmodal cuing benefit at CTOA = 500 ms. Experiment 2 showed that this late auditory looming cue benefit was independent of the cue's intensity when the visual target appeared. Thus, we conclude that the late crossmodal benefit throughout an auditory looming cue's presentation is due to its increasing intensity profile. The neural basis for this benefit and its ecological implications are discussed.
Collapse
Affiliation(s)
- Christiane Glatz
- Max Planck Institute for Biological Cybernetics, Department Human Perception, Cognition, and Action, Tübingen, 72076, Germany.,Graduate Training Centre of Neuroscience, University of Tübingen, Tübingen, 72074, Germany
| | - Lewis L Chuang
- Max Planck Institute for Biological Cybernetics, Department Human Perception, Cognition, and Action, Tübingen, 72076, Germany. .,Institute for Informatics, Ludwig-Maximilian-Universiät, Munich, 80337, Germany.
| |
Collapse
|
8
|
Improving Human–Computer Interface Design through Application of Basic Research on Audiovisual Integration and Amplitude Envelope. MULTIMODAL TECHNOLOGIES AND INTERACTION 2019. [DOI: 10.3390/mti3010004] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
Abstract
Quality care for patients requires effective communication amongst medical teams. Increasingly, communication is required not only between team members themselves, but between members and the medical devices monitoring and managing patient well-being. Most human–computer interfaces use either auditory or visual displays, and despite significant experimentation, they still elicit well-documented concerns. Curiously, few interfaces explore the benefits of multimodal communication, despite extensive documentation of the brain’s sensitivity to multimodal signals. New approaches built on insights from basic audiovisual integration research hold the potential to improve future human–computer interfaces. In particular, recent discoveries regarding the acoustic property of amplitude envelope illustrate that it can enhance audiovisual integration while also lowering annoyance. Here, we share key insights from recent research with the potential to inform applications related to human–computer interface design. Ultimately, this could lead to a cost-effective way to improve communication in medical contexts—with signification implications for both human health and the burgeoning medical device industry.
Collapse
|
9
|
Abstract
Persistence of excitation and neural adaptation are competing theories proposed to explain the mechanisms underlying psychophysical forward masking. Previous research has been directed towards finding models that accurately describe the phenomenon but cannot account for the underlying explanation. The current study was designed to determine which theory best accounts for results obtained from behavioral gap duration adjustment tasks. Thirteen adults adjusted the gap within asymmetrical noise markers to be subjectively equal to the gap within equal-intensity-noise markers. The duration of the perceived gap between the asymmetrical markers is expected to vary depending on which theory dominates perception. The persistence of excitation mechanism would lead to longer duration gaps when the second noise marker is lower in intensity than the preceding. Neural adaptation would result in matched gaps that are shorter in duration when the second noise marker was lower in level. The outcome of our data analysis is consistent with the persistence of excitation as a dominant mechanism in forward masking.
Collapse
|
10
|
Yamasaki D, Miyoshi K, Altmann CF, Ashida H. Front-Presented Looming Sound Selectively Alters the Perceived Size of a Visual Looming Object. Perception 2018; 47:751-771. [PMID: 29783921 DOI: 10.1177/0301006618777708] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
In spite of accumulating evidence for the spatial rule governing cross-modal interaction according to the spatial consistency of stimuli, it is still unclear whether 3D spatial consistency (i.e., front/rear of the body) of stimuli also regulates audiovisual interaction. We investigated how sounds with increasing/decreasing intensity (looming/receding sound) presented from the front and rear space of the body impact the size perception of a dynamic visual object. Participants performed a size-matching task (Experiments 1 and 2) and a size adjustment task (Experiment 3) of visual stimuli with increasing/decreasing diameter, while being exposed to a front- or rear-presented sound with increasing/decreasing intensity. Throughout these experiments, we demonstrated that only the front-presented looming sound caused overestimation of the spatially consistent looming visual stimulus in size, but not of the spatially inconsistent and the receding visual stimulus. The receding sound had no significant effect on vision. Our results revealed that looming sound alters dynamic visual size perception depending on the consistency in the approaching quality and the front-rear spatial location of audiovisual stimuli, suggesting that the human brain differently processes audiovisual inputs based on their 3D spatial consistency. This selective interaction between looming signals should contribute to faster detection of approaching threats. Our findings extend the spatial rule governing audiovisual interaction into 3D space.
Collapse
Affiliation(s)
| | | | - Christian F Altmann
- Human Brain Research Center, Graduate School of Medicine, Kyoto University, Japan
| | | |
Collapse
|
11
|
Schutz M, Stefanucci JK, Sarah HB, Roth A. Name that Percussive Tune: Associative Memory and Amplitude Envelope. Q J Exp Psychol (Hove) 2017; 70:1323-1343. [DOI: 10.1080/17470218.2016.1182562] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
A series of experiments demonstrated novel effects of amplitude envelope on associative memory, with tones exhibiting naturally decaying amplitude envelopes (e.g., those made by two wine glasses clinking) better associated with target objects than amplitude-invariant tones. In Experiment 1 participants learned associations between household objects and 4-note tone sequences constructed of spectrally matched pure tones with either “flat” or “percussive” amplitude envelopes. Those hearing percussive tones correctly recalled significantly more sequence–object associations. Experiment 2 demonstrated that participants hearing percussive tones learned the associations more quickly. Experiment 3 used “reverse percussive” tones (percussive tones played backwards) to test whether differences in overall energy might account for this effect, finding they did not lead to the same level of performance as percussive tones. Experiment 4 varied the envelope at encoding and retrieval to determine which stage of the task was most affected by the envelope manipulation. Participants hearing percussive tones at both encoding and retrieval performed significantly better than the other three groups (i.e., flat at encoding/percussive at retrieval, etc.). We conclude that amplitude envelope plays an important role in learning and memory, a finding with relevance to psychological research on audition and associative memory, as well as practical relevance for improving human–computer interface design.
Collapse
Affiliation(s)
- Michael Schutz
- School of the Arts, McMaster University, Hamilton, ON, Canada
| | | | - H. Baum Sarah
- Department of Psychology, The College of William & Mary, Williamsburg, VA, USA
| | - Amber Roth
- Department of Psychology, The College of William & Mary, Williamsburg, VA, USA
| |
Collapse
|
12
|
Statistics of natural reverberation enable perceptual separation of sound and space. Proc Natl Acad Sci U S A 2016; 113:E7856-E7865. [PMID: 27834730 DOI: 10.1073/pnas.1612524113] [Citation(s) in RCA: 33] [Impact Index Per Article: 4.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022] Open
Abstract
In everyday listening, sound reaches our ears directly from a source as well as indirectly via reflections known as reverberation. Reverberation profoundly distorts the sound from a source, yet humans can both identify sound sources and distinguish environments from the resulting sound, via mechanisms that remain unclear. The core computational challenge is that the acoustic signatures of the source and environment are combined in a single signal received by the ear. Here we ask whether our recognition of sound sources and spaces reflects an ability to separate their effects and whether any such separation is enabled by statistical regularities of real-world reverberation. To first determine whether such statistical regularities exist, we measured impulse responses (IRs) of 271 spaces sampled from the distribution encountered by humans during daily life. The sampled spaces were diverse, but their IRs were tightly constrained, exhibiting exponential decay at frequency-dependent rates: Mid frequencies reverberated longest whereas higher and lower frequencies decayed more rapidly, presumably due to absorptive properties of materials and air. To test whether humans leverage these regularities, we manipulated IR decay characteristics in simulated reverberant audio. Listeners could discriminate sound sources and environments from these signals, but their abilities degraded when reverberation characteristics deviated from those of real-world environments. Subjectively, atypical IRs were mistaken for sound sources. The results suggest the brain separates sound into contributions from the source and the environment, constrained by a prior on natural reverberation. This separation process may contribute to robust recognition while providing information about spaces around us.
Collapse
|
13
|
A “looming bias” in spatial hearing? Effects of acoustic intensity and spectrum on categorical sound source localization. Atten Percept Psychophys 2016; 79:352-362. [DOI: 10.3758/s13414-016-1201-9] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
14
|
Perceptual Temporal Asymmetry Associated with Distinct ON and OFF Responses to Time-Varying Sounds with Rising versus Falling Intensity: A Magnetoencephalography Study. Brain Sci 2016; 6:brainsci6030027. [PMID: 27527227 PMCID: PMC5039456 DOI: 10.3390/brainsci6030027] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2016] [Revised: 07/26/2016] [Accepted: 07/29/2016] [Indexed: 11/29/2022] Open
Abstract
This magnetoencephalography (MEG) study investigated evoked ON and OFF responses to ramped and damped sounds in normal-hearing human adults. Two pairs of stimuli that differed in spectral complexity were used in a passive listening task; each pair contained identical acoustical properties except for the intensity envelope. Behavioral duration judgment was conducted in separate sessions, which replicated the perceptual bias in favour of the ramped sounds and the effect of spectral complexity on perceived duration asymmetry. MEG results showed similar cortical sites for the ON and OFF responses. There was a dominant ON response with stronger phase-locking factor (PLF) in the alpha (8–14 Hz) and theta (4–8 Hz) bands for the damped sounds. In contrast, the OFF response for sounds with rising intensity was associated with stronger PLF in the gamma band (30–70 Hz). Exploratory correlation analysis showed that the OFF response in the left auditory cortex was a good predictor of the perceived temporal asymmetry for the spectrally simpler pair. The results indicate distinct asymmetry in ON and OFF responses and neural oscillation patterns associated with the dynamic intensity changes, which provides important preliminary data for future studies to examine how the auditory system develops such an asymmetry as a function of age and learning experience and whether the absence of asymmetry or abnormal ON and OFF responses can be taken as a biomarker for certain neurological conditions associated with auditory processing deficits.
Collapse
|
15
|
The unity assumption facilitates cross-modal binding of musical, non-speech stimuli: The role of spectral and amplitude envelope cues. Atten Percept Psychophys 2016; 78:1512-28. [DOI: 10.3758/s13414-016-1088-5] [Citation(s) in RCA: 15] [Impact Index Per Article: 1.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
16
|
Alards-Tomalin D, Walker AC, Kravetz A, Leboe-McGowan LC. Numerical Context and Time Perception: Contrast Effects and the Perceived Duration of Numbers. Perception 2015; 45:222-45. [PMID: 26562847 DOI: 10.1177/0301006615594905] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
In the current study, we examined how the contextual repetition of magnitude information presented in either symbolic (Arabic digits) or nonsymbolic (numerosities) formats impacted on the perceived duration of a later occurring target number. The results of the current study demonstrated a time-magnitude bias in which, on average, large magnitude target numbers were judged to last for longer durations relative to small magnitude target numbers, regardless of notation (symbolic number and numerosity). Furthermore, context effects were found, in which a greater discrepancy in the target's magnitude from the initial context led to longer perceived duration ratings. However, this was found to be asymmetrical, occurring only for large magnitude targets. Additionally, the type of context effect was shown to be determined by whether the context was presented in the same notation as the target or a different notation.
Collapse
Affiliation(s)
| | | | - Alexa Kravetz
- Department of Psychology, University of Manitoba, Winnipeg, Canada
| | | |
Collapse
|
17
|
A robust asymmetry in loudness between rising- and falling-intensity tones. Atten Percept Psychophys 2015; 77:907-20. [DOI: 10.3758/s13414-014-0824-y] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
18
|
Audition dominates vision in duration perception irrespective of salience, attention, and temporal discriminability. Atten Percept Psychophys 2014; 76:1485-502. [PMID: 24806403 DOI: 10.3758/s13414-014-0663-x] [Citation(s) in RCA: 22] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Whereas the visual modality tends to dominate over the auditory modality in bimodal spatial perception, the auditory modality tends to dominate over the visual modality in bimodal temporal perception. Recent results suggest that the visual modality dominates bimodal spatial perception because spatial discriminability is typically greater for the visual than for the auditory modality; accordingly, visual dominance is eliminated or reversed when visual-spatial discriminability is reduced by degrading visual stimuli to be equivalent or inferior to auditory spatial discriminability. Thus, for spatial perception, the modality that provides greater discriminability dominates. Here, we ask whether auditory dominance in duration perception is similarly explained by factors that influence the relative quality of auditory and visual signals. In contrast to the spatial results, the auditory modality dominated over the visual modality in bimodal duration perception even when the auditory signal was clearly weaker, when the auditory signal was ignored (i.e., the visual signal was selectively attended), and when the temporal discriminability was equivalent for the auditory and visual signals. Thus, unlike spatial perception, where the modality carrying more discriminable signals dominates, duration perception seems to be mandatorily linked to auditory processing under most circumstances.
Collapse
|
19
|
Meunier S, Vannier M, Chatron J, Susini P. Asymmetry in perceived duration between up-ramp and down-ramp sounds as a function of duration. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2014; 136:EL166-EL172. [PMID: 25096142 DOI: 10.1121/1.4890175] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
The perceived duration of 1-kHz pure tones with increasing or decreasing intensity profiles was measured. The ratio between the down- and up-ramp durations at equal subjective durations was examined as a function of the sound duration (50, 100, 200, 500, 1000, 2000 ms). At 50 and 100 ms, the ratio was constant and equaled about 1.7, then it logarithmically decreased from 100 to 1000 ms to reach a constant value of 1 at 1 and 2 s. The different mechanisms proposed in the literature to explain the perceived duration asymmetry between up-ramp and down-ramp were discussed in the light of the dependence of this ratio on duration.
Collapse
Affiliation(s)
- Sabine Meunier
- Laboratoire de Mécanique et d'Acoustique, Centre National de la Recherche Scientifique, Unité Propre de Recherche 7051, Aix-Marseille Université, Centrale Marseille, F-13402 Marseille Cedex 20, France
| | - Michaël Vannier
- Laboratoire Vibrations Acoustique, INSA-Lyon, 25 bis Avenue Jean Capelle, F-69621 Villeurbanne, Cedex, France
| | - Jacques Chatron
- Laboratoire de Mécanique et d'Acoustique, Centre National de la Recherche Scientifique, Unité Propre de Recherche 7051, Aix-Marseille Université, Centrale Marseille, F-13402 Marseille Cedex 20, France
| | - Patrick Susini
- STMS Lab (IRCAM, CNRS, UPMC), 1 Place Igor Stravinsky, 75004 Paris, France
| |
Collapse
|
20
|
Olsen KN, Stevens CJ, Dean RT, Bailes F. Continuous loudness response to acoustic intensity dynamics in melodies: effects of melodic contour, tempo, and tonality. Acta Psychol (Amst) 2014; 149:117-28. [PMID: 24809252 DOI: 10.1016/j.actpsy.2014.03.007] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2013] [Revised: 03/06/2014] [Accepted: 03/24/2014] [Indexed: 10/25/2022] Open
Abstract
The aim of this work was to investigate perceived loudness change in response to melodies that increase (up-ramp) or decrease (down-ramp) in acoustic intensity, and the interaction with other musical factors such as melodic contour, tempo, and tonality (tonal/atonal). A within-subjects design manipulated direction of linear intensity change (up-ramp, down-ramp), melodic contour (ascending, descending), tempo, and tonality, using single ramp trials and paired ramp trials, where single up-ramps and down-ramps were assembled to create continuous up-ramp/down-ramp or down-ramp/up-ramp pairs. Twenty-nine (Exp 1) and thirty-six (Exp 2) participants rated loudness continuously in response to trials with monophonic 13-note piano melodies lasting either 6.4s or 12s. Linear correlation coefficients >.89 between loudness and time show that time-series loudness responses to dynamic up-ramp and down-ramp melodies are essentially linear across all melodies. Therefore, 'indirect' loudness change derived from the difference in loudness at the beginning and end points of the continuous response was calculated. Down-ramps were perceived to change significantly more in loudness than up-ramps in both tonalities and at a relatively slow tempo. Loudness change was also greater for down-ramps presented with a congruent descending melodic contour, relative to an incongruent pairing (down-ramp and ascending melodic contour). No differential effect of intensity ramp/melodic contour congruency was observed for up-ramps. In paired ramp trials assessing the possible impact of ramp context, loudness change in response to up-ramps was significantly greater when preceded by down-ramps, than when not preceded by another ramp. Ramp context did not affect down-ramp perception. The contribution to the fields of music perception and psychoacoustics are discussed in the context of real-time perception of music, principles of music composition, and performance of musical dynamics.
Collapse
|
21
|
Vallet GT, Shore DI, Schutz M. Exploring the Role of the Amplitude Envelope in Duration Estimation. Perception 2014; 43:616-30. [DOI: 10.1068/p7656] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
A sound's duration provides important information about the event producing it. Although many of the sounds we hear every day are ‘percussive’ in nature (ie resulting from two objects impacting) and therefore exhibit decaying/damped amplitude envelopes, perceptual experiments frequently use tones synthesized with ‘flat’ or abruptly ending envelopes. Such sounds afford an estimation strategy involving calculating the elapsed time between tone onset and offset—a strategy that would be problematic for ecologically pervasive decaying sounds. Here we compare duration judgments for tones with percussive (ie gradually decaying) and flat (ie abruptly ending) amplitude envelopes, finding evidence for the use of different strategies. This result is discussed in terms of its implications for dominant theories and models of sensory perception that are often assessed using artificial sounds (ie ‘flat tones’) affording strategies that may not be optimal or even available for everyday listening.
Collapse
Affiliation(s)
- Guillaume T Vallet
- Multisensory Perception Laboratory, Department of Psychology, Neuroscience and Behaviour, McMaster University, Canada; also Centre de Recherche de l'Institut Universitaire de Gériatrie de Montréal (CRIUGM), University of Montreal, Canada
| | - David I Shore
- Multisensory Perception Laboratory, Department of Psychology, Neuroscience and Behaviour, McMaster University, Canada
| | - Michael Schutz
- Music, Acoustics, Perception and LEarning (MAPLE) Laboratory, School of the Arts, McMaster University, 280 Main Street West, Hamilton, ON L8S 4K1, Canada
| |
Collapse
|
22
|
Perceptual distortions in pitch and time reveal active prediction and support for an auditory pitch-motion hypothesis. PLoS One 2013; 8:e70646. [PMID: 23936462 PMCID: PMC3735601 DOI: 10.1371/journal.pone.0070646] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/24/2013] [Accepted: 06/20/2013] [Indexed: 11/19/2022] Open
Abstract
A number of accounts of human auditory perception assume that listeners use prior stimulus context to generate predictions about future stimulation. Here, we tested an auditory pitch-motion hypothesis that was developed from this perspective. Listeners judged either the time change (i.e., duration) or pitch change of a comparison frequency glide relative to a standard (referent) glide. Under a constant-velocity assumption, listeners were hypothesized to use the pitch velocity (Δf/Δt) of the standard glide to generate predictions about the pitch velocity of the comparison glide, leading to perceptual distortions along the to-be-judged dimension when the velocities of the two glides differed. These predictions were borne out in the pattern of relative points of subjective equality by a significant three-way interaction between the velocities of the two glides and task. In general, listeners' judgments along the task-relevant dimension (pitch or time) were affected by expectations generated by the constant-velocity standard, but in an opposite manner for the two stimulus dimensions. When the comparison glide velocity was faster than the standard, listeners overestimated time change, but underestimated pitch change, whereas when the comparison glide velocity was slower than the standard, listeners underestimated time change, but overestimated pitch change. Perceptual distortions were least evident when the velocities of the standard and comparison glides were matched. Fits of an imputed velocity model further revealed increasingly larger distortions at faster velocities. The present findings provide support for the auditory pitch-motion hypothesis and add to a larger body of work revealing a role for active prediction in human auditory perception.
Collapse
|
23
|
Evidence for a basic level in a taxonomy of everyday action sounds. Exp Brain Res 2013; 226:253-64. [PMID: 23411674 DOI: 10.1007/s00221-013-3430-7] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/28/2012] [Accepted: 01/23/2013] [Indexed: 10/27/2022]
Abstract
We searched for evidence that the auditory organization of categories of sounds produced by actions includes a privileged or "basic" level of description. The sound events consisted of single objects (or substances) undergoing simple actions. Performance on sound events was measured in two ways: sounds were directly verified as belonging to a category, or sounds were used to create lexical priming. The category verification experiment measured the accuracy and reaction time to brief excerpts of these sounds. The lexical priming experiment measured reaction time benefits and costs caused by the presentation of these sounds prior to a lexical decision. The level of description of a sound varied in how specifically it described the physical properties of the action producing the sound. Both identification and priming effects were superior when a label described the specific interaction causing the sound (e.g. trickling) in comparison to the following: (1) more general descriptions (e.g. pour, liquid: trickling is a specific manner of pouring liquid), (2) more detailed descriptions using adverbs to provide detail regarding the manner of the action (e.g. trickling evenly). These results are consistent with neuroimaging studies showing that auditory representations of sounds produced by actions familiar to the listener activate motor representations of the gestures involved in sound production.
Collapse
|
24
|
Tyll S, Bonath B, Schoenfeld MA, Heinze HJ, Ohl FW, Noesselt T. Neural basis of multisensory looming signals. Neuroimage 2013; 65:13-22. [DOI: 10.1016/j.neuroimage.2012.09.056] [Citation(s) in RCA: 42] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/21/2012] [Revised: 09/03/2012] [Accepted: 09/20/2012] [Indexed: 10/27/2022] Open
|
25
|
Olsen KN, Stevens CJ. Psychophysiological Response to Acoustic Intensity Change in a Musical Chord. J PSYCHOPHYSIOL 2013. [DOI: 10.1027/0269-8803/a000082] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022]
Abstract
This paper investigates psychological and psychophysiological components of arousal and emotional response to a violin chord stimulus comprised of continuous increases (up-ramp) or decreases (down-ramp) of intensity. A factorial experiment manipulated direction of intensity change (60–90 dB SPL up-ramp, 90–60 dB SPL down-ramp) and duration (1.8 s, 3.6 s) within-subjects (N = 45). Dependent variables were ratings of emotional arousal, valence, and loudness change, and a fine-grained analysis of event-related skin conductance response (SCR). As hypothesized, relative to down-ramps, musical up-ramps elicited significantly higher ratings of emotional arousal and loudness change, with marginally longer SCR rise times. However, SCR magnitude was greater in response to musical down-ramps. The implications of acoustic intensity change for music-induced emotion and auditory warning perception are discussed.
Collapse
Affiliation(s)
- Kirk N. Olsen
- MARCS Institute and School of Social Sciences and Psychology, University of Western Sydney, Penrith, NSW, Australia
| | - Catherine J. Stevens
- MARCS Institute and School of Social Sciences and Psychology, University of Western Sydney, Penrith, NSW, Australia
| |
Collapse
|
26
|
Abstract
Looming visual stimuli (log-increasing in proximal size over time) and auditory stimuli (of increasing sound intensity over time) have been shown to be perceived as longer than receding visual and auditory stimuli (i.e., looming stimuli reversed in time). Here, we investigated whether such asymmetry in subjective duration also occurs for audiovisual looming and receding stimuli, as well as for stationary stimuli (i.e., stimuli that do not change in size and/or intensity over time). Our results showed a great temporal asymmetry in audition but a null asymmetry in vision. In contrast, the asymmetry in audiovision was moderate, suggesting that multisensory percepts arise from the integration of unimodal percepts in a maximum-likelihood fashion.
Collapse
|
27
|
Olsen KN, Stevens CJ. Forward masking of dynamic acoustic intensity: effects of intensity region and end-level. Perception 2012; 41:594-605. [PMID: 23025162 DOI: 10.1068/p7128] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Abstract
Overestimation of loudness change typically occurs in response to up-ramp auditory stimuli (increasing intensity) relative to down-ramps (decreasing intensity) matched on frequency, duration, and end-level. In the experiment reported, forward masking is used to investigate a sensory component of up-ramp overestimation: persistence of excitation after stimulus presentation. White-noise and synthetic vowel 3.6 s up-ramp and down-ramp maskers were presented over two regions of intensity change (40-60 dB SPL, 60-80 dB SPL). Three participants detected 10 ms 1.5 kHz pure tone signals presented at masker-offset to signal-offset delays of 10, 20, 30, 50, 90, 170 ms. Masking magnitude was significantly greater in response to up-ramps compared with down-ramps for masker-signal delays up to and including 50 ms. When controlling for an end-level recency bias (40-60 dB SPL up-ramp vs 80-60 dB SPL down-ramp), the difference in masking magnitude between up-ramps and down-ramps was not significant at each masker-signal delay. Greater sensory persistence in response to up-ramps is argued to have minimal effect on perceptual overestimation of loudness change when response biases are controlled. An explanation based on sensory adaptation is discussed.
Collapse
Affiliation(s)
- Kirk N Olsen
- MARCS Institute, University of Western Sydney, Locked Bag 1797, Penrith, NSW 2751, Australia.
| | | |
Collapse
|
28
|
Alards-Tomalin D, Leboe-McGowan LC, Mondor TA. Examining auditory kappa effects through manipulating intensity differences between sequential tones. PSYCHOLOGICAL RESEARCH 2012; 77:480-91. [DOI: 10.1007/s00426-012-0438-8] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2012] [Accepted: 04/07/2012] [Indexed: 10/28/2022]
|
29
|
Lentz JJ, Shen Y. Investigating temporal asymmetry using masking period patterns and models of peripheral auditory processing. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2011; 129:3194-3205. [PMID: 21568421 DOI: 10.1121/1.3573979] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/30/2023]
Abstract
Two experiments were conducted in conjunction with modeling to evaluate the role of peripheral nonlinearity and neural adaptation in the perception of temporally asymmetric sounds. In both experiments, maskers were broadband noises amplitude modulated with ramped and damped exponential modulators that repeated at 40 Hz. Masking period patterns (MPPs) were constructed by measuring detection threshold of a 5-ms, 1000-Hz tone burst as function of the signal's onset delay. Experiment I showed that varying modulator half-life from 1 to 16 ms led to differences in the damped and the ramped MPPs that were largest at the short half-lives and diminished at the longer half-lives. When masker level was varied (experiment II), the largest difference between ramped and damped MPPs occurred at moderate stimulus levels. Two peripheral auditory models were evaluated, one a simple auditory filter followed by a power-law nonlinearity and another, a model of auditory nerve processing [J. Acoust. Soc. Am. 126, 2390-2412 (2009)] that includes neural adaptation. Neither models predicted differences between the ramped and damped MPPs, providing indirect support that the central auditory system has a role in perceptual temporal asymmetry.
Collapse
Affiliation(s)
- Jennifer J Lentz
- Department of Speech and Hearing Sciences, Indiana University, Bloomington, Indiana 47405-7000, USA.
| | | |
Collapse
|
30
|
Grassi M. Sex difference in subjective duration of looming and receding sounds. Perception 2011; 39:1424-6. [PMID: 21180365 DOI: 10.1068/p6810] [Citation(s) in RCA: 19] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Looming sounds (sounds increasing in intensity over time) are more salient than receding sounds (a looming sound reversed in time). For example, they are estimated as being longer, louder, and more changing in loudness than receding sounds. Some authors interpret the looming salience as evolutionarily adaptive, because it increases the margins of safety of the perceiver in the case of preparatory behaviours (e.g., a motor reaction to an approaching sound source). Recently, Neuhoff et al (2009, Journal of Experimental Psychology: Human Perception and Performance 35 225-234) found that females more than males show overestimation of the spatiotemporal properties of virtually simulated looming sound sources. Here, I investigated whether the sex difference could be observed for the subjective duration of looming and receding sounds, and found that females more than males overestimate the duration of looming sounds in comparison to receding sounds.
Collapse
Affiliation(s)
- Massimo Grassi
- Dipartimento di Psicologia Generale, Università di Padova, via Venezia 8, I 35131 Padua, Italy.
| |
Collapse
|
31
|
Magnitude judgments of loudness change for discrete, dynamic, and hybrid stimuli. Atten Percept Psychophys 2010; 73:886-907. [PMID: 21264709 DOI: 10.3758/s13414-010-0056-8] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
|
32
|
Olsen KN, Stevens CJ. Perceptual Overestimation of Rising Intensity: Is Stimulus Continuity Necessary? Perception 2010; 39:695-704. [DOI: 10.1068/p6592] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2022]
Abstract
A “perceptual bias for rising intensity” (Neuhoff 1998, Nature395 123–124) is not dependent on the continuous change of a dynamic, looming sound source. Thirty participants were presented with pairs of 500 ms steady-state sounds corresponding to onset and offset levels of previously used dynamic increasing- and decreasing-intensity stimuli. Independent variables, intensity-change direction (increasing, decreasing), intensity region (high: 70–90 dB SPL, low: 50–70 dB SPL), interstimulus interval (ISI) (0 s, 1.8 s, 3.6 s), and timbre (vowel, violin) were manipulated as a fully within-subjects design. The dependent variable was perceived loudness change between each stimulus item in a pair. It was hypothesised that (i) noncontinuous increases of intensity are overestimated in loudness change, relative to decreases, in both low-intensity and high-intensity regions; and (ii) perceptual overestimation does not occur when end-levels are balanced. The hypotheses were partially supported. At the high-intensity region, increasing stimuli were perceived to change more in loudness than decreasing-intensity stimuli. At the low-intensity region and under balanced end-level conditions, decreasing-intensity stimuli were perceived to change more in loudness than increasing-intensity stimuli. A significant direction × region interaction varied as a function of ISI. Methodological, sensory, and cognitive explanations for overestimation in certain circumstances are discussed.
Collapse
Affiliation(s)
- Kirk N Olsen
- MARCS Auditory Laboratories, University of Western Sydney, South Penrith DC, NSW 1797, Australia
| | - Catherine J Stevens
- MARCS Auditory Laboratories, University of Western Sydney, South Penrith DC, NSW 1797, Australia
| |
Collapse
|
33
|
Ries DT, Schlauch RS, DiGiovanni JJ. The role of temporal-masking patterns in the determination of subjective duration and loudness for ramped and damped sounds. THE JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA 2008; 124:3772-3783. [PMID: 19206803 PMCID: PMC2676627 DOI: 10.1121/1.2999342] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/15/2007] [Revised: 09/17/2008] [Accepted: 09/22/2008] [Indexed: 05/27/2023]
Abstract
The association between temporal-masking patterns, duration, and loudness for broadband noises with ramped and damped envelopes was examined. Duration and loudness matches between the ramped and damped sounds differed significantly. Listeners perceived the ramped stimuli to be longer and louder than the damped stimuli, but the outcome was biased by the stimulus context. Next, temporal-masking patterns were measured for ramped- and damped-broadband noises using three (0.5, 1.5, and 4.0 kHz) 10 ms probe tones presented individually at various temporal delays. Predictions of subjective duration derived from masking results underpredicted the matching results. Loudness estimates derived from models that assume persistence of neural activity after stimulus offset [Glasberg B. R., and Moore, B. C. J. (2002). "A model of loudness applicable to time-varying sounds," J. Audio. Eng. Soc. 50, 331-341; Chalupper, J., and Fastl, H. (2002) "Dynamic loudness model (DLM) for normal and hearing-impaired listeners," Acust. Acta Acust. 88, 378-386] were greater for ramped sounds than for damped sounds and were close to the average results obtained via the matching task. Differences in simultaneous-masked thresholds for these stimuli could not account for the loudness-matching results. Decay suppression of the later occurring portion of the damped stimulus may account for the differences in perception due to the stimulus context; however, a parsimonious implementation of this process that accounts for both subjective duration and loudness judgments remains unclear.
Collapse
Affiliation(s)
- Dennis T Ries
- School of Hearing, Speech and Language Sciences, Ohio University, Athens, Ohio 45701, USA.
| | | | | |
Collapse
|