Multisensory perception constrains the formation of object categories: a review of evidence from sensory-driven and predictive processes on categorical decisions

Although object categorization is a fundamental cognitive ability, it is also a complex process going beyond the perception and organization of sensory stimulation. Here we review existing evidence about how the human brain acquires and organizes multisensory inputs into object representations that may lead to conceptual knowledge in memory. We first focus on evidence for two processes on object perception, multisensory integration of redundant information (e.g. seeing and feeling a shape) and crossmodal, statistical learning of complementary information (e.g. the 'moo' sound of a cow and its visual shape). For both processes, the importance attributed to each sensory input in constructing a multisensory representation of an object depends on the working range of the specific sensory modality, the relative reliability or distinctiveness of the encoded information and top-down predictions. Moreover, apart from sensory-driven influences on perception, the acquisition of featural information across modalities can affect semantic memory and, in turn, influence category decisions. In sum, we argue that both multisensory processes independently constrain the formation of object categories across the lifespan, possibly through early and late integration mechanisms, respectively, to allow us to efficiently achieve the everyday, but remarkable, ability of recognizing objects. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

Older adults with slow sit to stand times show reduced temporal precision of audio-visual integration

Sustained integration of sensory inputs over increased temporal delays is associated with reduced cognitive and physical functioning in older adults and adverse outcomes such as falls. Here, we explored the relationship between multisensory integration and a clinically relevant measure of balance/postural control; Sit-to-Stand Time, the efficiency with which an older adult can transition between a seated and a standing posture. We investigated whether temporal multisensory integration was associated with performance on the Five-Times Sit-to-Stand Test (FTSST) in a large sample of 2556 older adults (mean age = 63.62 years, SD = 7.50; 55% female) drawn from The Irish Longitudinal Study on Ageing (TILDA). K-means clustering was applied to FTSST data, yielding three clusters characterised by fast (mean = 10.88 s; n = 1122), medium (mean = 14.34 s; n = 1133) and slow (mean = 18.97 s; n = 301) sit-to-stand times. At wave 3 of TILDA, older adults participated in the Sound Induced Flash Illusion (SIFI), a measure of the precision of temporal audio-visual integration, which included three audio-visual stimulus onset asynchronies (SOAs): 70, 150 and 230 ms. Older adults with the slowest sit-to-stand times were more susceptible to the SIFI at the longest SOA (230 ms) compared to the shortest SOA (70 ms) relative to those with the fastest times (p = 0.02). Older adults who take longer to repeatedly transition from a seated to a standing posture exhibit an expanded temporal binding window for audio-visual events, supporting a link between multisensory perception and balance/postural control in ageing.

246 THE ROLE OF AGEING ON SELF-REPORTED ABILITY ACROSS MULTIPLE SENSES: A LONGITUDINAL ANALYSIS OF DATA FROM THE TILDA STUDY

Background

During the healthy ageing process, sensory degeneration typically occurs and often in more than one sensory modality. This can present significant challenges to the well-being and quality of life of older adults. However, it is unclear whether senses decline into impaired functioning in a manner that is independent from each other or that is suggestive of a common, amodal mechanism related to the ageing process.

Methods

We used discrete survival analysis to explore the longitudinal (10-year) associations between a number of measures (including age, lifestyle factors and measures of sensory, physical, mental and cognitive health) and the probability of transitioning from normal to impaired sensory function for vision, audition, olfaction and gustation. Sensory impairment was captured as self-report measures for a large sample of older adults (N = 5,066) from The Irish Longitudinal Study on Ageing.

Results

Auditory impairment was most prevalent, followed by impaired vision, olfaction and gustation. Multivariable regression models revealed an increased probability of impairment in one modality for those reporting impairments in other modalities, although this pattern was not consistent for all modality combinations. Visual impairment was associated with socioeconomic status, education, eye disease, smoking and depression. Auditory impairment was associated with age, the sex of participant, hearing aid use and difficulty following group conversations. Olfactory impairment was associated with age, the sex of participant, smoking and difficulty following group conversations. Gustatory impairment was associated with age, smoking and depression. Importantly, self-rated decline in general health was associated with an increased probability of reporting an impairment across all sensory modalities.

Conclusion

Our results suggest that older adults report a decline across multiple senses over time, but the rates are of decline are not consistent. Sensory decline appears associated with both global and modality-specific factors, providing some insights into the impact of sensory health in ageing.

The rubber hand illusion is influenced by self-recognition

Susceptibility to the Rubber Hand Illusion (RHI) demonstrates that body ownership can be modulated by visuotactile inputs. In contrast to body-like images, other objects cannot be embodied suggesting that crossmodal interactions on body ownership are based on a 'goodness-of-fit' mechanism relative to one's own body. However, it is not clear whether visual self-recognition influences susceptibility to the RHI, although evidence for individual differences in the perceptual body image on the RHI suggests that this may be the case. We investigated the role of self-recognition on the subjective experience of the RHI and measured proprioceptive drift and onset time of the RHI between two groups, one with the ability to identify an image of their own hand and the other without this ability. A typical RHI response was found overall with no group difference in the subjective experience of the RHI. However, a larger proprioceptive drift and an earlier onset time for the RHI was found for the non-recognisers than the self-recognition group. Our findings provide evidence for a link between a visual representation of one's own body in long-term memory and plasticity of the body representation.