Psychological and physiological effects of soundscapes: A systematic review of 25 experiments in the English and Chinese literature

The aim of this systematic review was to conduct a comprehensive and rigorous investigation of both psychological and physiological responses to, and audio-visual interactions with, soundscapes to present an overview of the current status and to provide suggestions for future research. Our literature search focused on empirical and quantitative studies of journal articles and gray literature in English and Chinese. This systematic review excluded literature related to pure music, religious sounds, humanistic sounds, historical sounds, medical research, and differences in materials used. The Joanna Briggs Institute's Checklist for Randomized Controlled Trials was used to assess the risk of bias in the included studies. Twenty-five studies were included, involving 1950 participants. The major findings of this systematic review were that: (1) there were significant associations between the psychological and physiological responses; (2) the audio-visual interaction affected the psychological and physiological responses; and (3) because of the high risk of bias of the included studies, interpretation of their findings should be cautious. Nevertheless, given that this systematic review has a higher level of evidence than a single study and the synthesized evidence identified in this review is aligned with the results of other studies, the studies reviewed herein together provide consistent evidence. Replications are important in empirical research to build trustworthy results. Future research should focus on the psychological responses of pleasantness, preference, tranquility, the eight semantic dimensions (ISO 12913-2:2018), and the 11 pairs of adjectives describing the soundscape (Ba et al., 2023) and the physiological responses of heart rate variability and salivary, and follow the CONSORT guidelines to improve the research quality. An integration of sensory modalities, environmental factors, contextual indicators, temporal data, demographic variables, socio-cultural factors, and psychological and physiological responses may provide deeper insights into how people experience and understand the acoustic environment in context.

Sensorimotor processing is dependent on observed speed during the observation of hand-hand and hand-object interactions

Observing a physical interaction between individuals (e.g., observing friends shaking hands) or between an object and an individual (e.g., observing a teammate striking or being struck with a ball) can lead to somatosensory activation in the observer. However, it is not known whether the speed of the observed interaction modulates such somatosensory activation (e.g., observing a teammate being struck with a slow vs. a fast-moving ball). In three experiments, participants observed a hand or object interact with another hand or object, all presented with a slow- or fast-moving effector. To probe sensorimotor processes during observation, participants were asked to react to an auditory beep (i.e., response time [RT] task) at the moment of observed contact. If observed contact led to increased somatosensory activation, RTs would decrease due to statistical and/ or intersensory facilitation. In all three experiments, RTs were lower when observing fast compared to slow motion stimuli, regardless of the moving (i.e., hand or ball) and target stimulus (i.e., hand or leaf). Further, when only an object (i.e., leaf) was the target, RTs did not differ between the moving hand and moving ball condition. In contrast, when an object (i.e., ball) was used as the moving stimulus, the magnitude of the speed effect (i.e., fast - slow RT difference) was significantly larger when the ball contacted a hand as compared to a leaf. Overall, these results provide novel evidence for a relationship between the observed kinematics of an object-human interaction and the sensorimotor processing in the observer.

Auditory enhancement of visual searches for event scenes

Increasing research has revealed that uninformative spatial sounds facilitate the early processing of visual stimuli. This study examined the crossmodal interactions of semantically congruent stimuli by assessing whether the presentation of event-related characteristic sounds facilitated or interfered with the visual search for corresponding event scenes in pictures. The search array consisted of four images: one target and three non-target pictures. Auditory stimuli were presented to participants in synchronization with picture onset using three types of sounds: a sound congruent with a target, a sound congruent with a distractor, or a control sound. The control sound varied across six experiments, alternating between a sound unrelated to the search stimuli, white noise, and no sound. Participants were required to swiftly localize a target position while ignoring the sound presentation. Visual localization resulted in rapid responses when a sound that was semantically related to the target was played. Furthermore, when a sound was semantically related to a distractor picture, the response times were longer. When the distractor-congruent sound was used, participants incorrectly localized the distractor position more often than at the chance level. These findings were replicated when the experiments ruled out the possibility that participants would learn picture-sound pairs during the visual tasks (i.e., the possibility of brief training during the experiments). Overall, event-related crossmodal interactions occur based on semantic representations, and audiovisual associations may develop as a result of long-term experiences rather than brief training in a laboratory.

Perceptual influence of auditory pitch on motion speed

There is a cross-modal mapping between auditory pitch and many visual properties, but the relationship between auditory pitch and motion speed is unexplored. In this article, the ball and baffle are used as the research objects, and an object collision experiment is used to explore the perceptual influence of auditory pitch on motion speed. Since cross-modal mapping can influence perceptual experience, this article also explores the influence of auditory pitch on action measures. In Experiment 1, 12 participants attempted to release a baffle to block a falling ball on the basis of speed judgment, and after each trial, they were asked to rate the speed of the ball. The speed score and baffle release time were recorded and used for analysis of variance. Since making explicit judgments about speed can alter the processing of visual paths, another group of participants in Experiment 2 completed the experiment without making explicit judgments about speed. Our results show that there is a cross-modal mapping between auditory pitch and motion speed, and high or low tones cause perception shift to faster or slower speeds.

A possible key role of vision in the development of schizophrenia

Based on a brief overview of the various aspects of schizophrenia reported by numerous studies, here we hypothesize that schizophrenia may originate (and in part be performed) from visual areas. In other words, it seems that a normal visual system or at least an evanescent visual perception may be an essential prerequisite for the development of schizophrenia as well as of various types of hallucinations. Our study focuses on auditory and visual hallucinations, as they are the most prominent features of schizophrenic hallucinations (and also the most studied types of hallucinations). Here, we evaluate the possible key role of the visual system in the development of schizophrenia.

Auditory capture of visual apparent motion, both laterally and looming

Traditional tests of multisensory stimuli typically support that vision dominates spatial judgments and audition dominates temporal ones. Here, we examine if unambiguous auditory spatial cues can capture ambiguous visual ones in judgments of direction of apparent motion. The visual motion judgments include both lateral movement and movement in depth, each when coupled with auditory stimuli moving at one of four rates. Experiment 1 tested lateral visual movement judgments (leftward vs rightward) coupled with auditory stimuli that moved laterally. Experiment 2 tested depth visual movement judgments (approaching vs receding) coupled with auditory stimuli that got louder or quieter. Results of Experiment 1 revealed and replicated an overall leftward motion bias, but with additional acoustic capture to experience visual movement away from the side on which sound initially occurred, and no effect of auditory motion speed. Results of Experiment 2 revealed and replicated an approaching motion bias, but with no effect of initial sound intensity, and an additional systematic capture effect of auditory motion speed. Faster changes in acoustic intensity produced larger visual motion capture consistent with the direction of acoustic intensity change. Findings of both experiments generalized over conditions of listening device (head phones vs speakers) and test-setting (Laboratory vs Web-based data-collection). The leftward and approaching motion bias results replicate previous research. Our principal new findings, the auditory motion capture effects, confirm the multisensory nature of dynamic spatial perception and support that extent of inter-sensory capture is a function of the relative reliability of spatial information acquired by each sensory modality.

Rewarding objects appear larger but not brighter

Whether reward can accentuate the perception of visual objects, that is, makes them appear larger than they really are, is a long-standing and controversial question. Here, we revisit this issue with a novel two-alternative forced-choice paradigm combining asymmetric reward schedule and task reversal. In a first experiment, participants (n = 27) choose the larger of two unequally rewarded objects in some sessions and the smaller one in other sessions. Response biases toward the most rewarding object differ significantly between the reversed tasks, revealing an influence of reward on perceived sizes. In a second experiment, participants (n = 27) indicate either the brighter or darker object. In contrast with the first experiment, response biases are similar between those reversed tasks, indicating that the perceived luminance is immune to reward manipulation. Together, these results reveal that if two objects are associated with different amounts of reward, participants will perceive the more rewarded object to be slightly larger, but not brighter, than the less rewarded one.

Electrophysiological correlates and psychoacoustic characteristics of hearing-motion synaesthesia

People with hearing-motion synaesthesia experience sounds from moving or changing (e.g. flickering) visual stimuli. This phenomenon may be one of the most common forms of synaesthesia but it has rarely been studied and there are no studies of its neural basis. We screened for this in a sample of 200+ individuals, and estimated a prevalence of 4.2%. We also document its characteristics: it tends to be induced by physically moving stimuli (more so than static stimuli which imply motion or trigger illusory motion); and the psychoacoustic features are simple (e.g. "whooshing") with some systematic correspondences to vision (e.g. faster movement is higher pitch). We demonstrate using event-related potentials that it emerges from early perceptual processing of vision. The synaesthetes have a higher amplitude motion-evoked N2 (165-185ms), with some evidence of group differences as early as 55-75ms. We discuss similarities between hearing-motion synaesthesia and previous observations that visual motion triggers auditory activity in the congenitally deaf. It is possible that both conditions reflect the maintenance of multisensory pathways found in early development that most people lose but can be retained in certain people in response to sensory deprivation (in the deaf) or, in people with normal hearing, as a result of other differences (e.g. genes predisposing to synaesthesia).