Optimal multisensory integration leads to optimal time estimation

Higher synchronization stability with piano experience: relationship with finger and presentation modality

BACKGROUND

Synchronous finger tapping to external sensory stimuli is more stable for audiovisual combined stimuli than sole auditory or visual stimuli. In addition, piano players are superior in synchronous tapping and manipulating the ring and little fingers as compared to inexperienced individuals. However, it is currently unknown whether the ability to synchronize to external sensory stimuli with the ring finger is at the level of the index finger in piano players. The aim of this study was to compare the effect of piano experience on synchronization stability between the index and ring fingers using auditory, visual, and audiovisual combined stimuli.

METHODS

Thirteen piano players and thirteen novices participated in this study. They were instructed to tap with their index or ring finger synchronously to auditory, visual, and audiovisual combined stimuli. The stimuli were presented from an electronic metronome at 1 Hz, and the tapping was performed 30 times in each condition. We analyzed standard deviation of intervals between the stimulus onset and the tap onset as synchronization stability.

RESULTS

Synchronization stability for visual stimuli was lower during ring than index finger tapping in novices; however, this decline was absent in piano players. Also, piano players showed the higher synchronization stability for audiovisual combined stimuli than sole visual and auditory stimuli when tapping with the index finger. On the other hand, in novices, synchronization stability was higher for audiovisual combined stimuli than only visual stimuli.

CONCLUSIONS

These findings suggest that improvements of both sensorimotor processing and finger motor control by piano practice would contribute to superior synchronization stability.

Memory decay enhances central bias in time perception

Temporal expectations are essential for appropriately interacting with the environment, but they can be biased. This tendency, called central bias, places higher weights on expected rather than actual duration distributions when perceiving incoming sensory stimuli. In particular, the central bias is strengthened in order to decrease total response error when incoming sensory stimuli are unclear. In the present study, we investigated whether the central bias was enhanced via memory decay. For this, we used a delayed reproduction task, manipulating retention periods by introducing delays between the sample interval and the reproduction phase (0.4, 2, 4 s in Experiment 1; 0.4, 2, 8 s in Experiments 2 and 3). Through three experiments, we found the gradual strengthening of the central bias as a function of the retention period (i.e., short-term memory decay). This suggests that the integration of temporal expectation, generated from past trials and stored sensory stimuli, in a current trial occurs in the reproduction phase in the delayed reproduction task.

Perceptual inference, accuracy, and precision in temporal reproduction in schizophrenia

Accumulating evidence suggests that deficits in perceptual inference account for symptoms of schizophrenia. One manifestation of perceptual inference is the central bias, i.e., the tendency to put emphasis on prior experiences over actual events in perceiving incoming sensory stimuli. Using an interval reproduction task, this study aimed to determine whether patients with schizophrenia show a stronger central bias than participants without schizophrenia. In the interval reproduction task, participants were shown a cross on a screen. The cross was replaced with a Gaussian patch for a predetermined time interval, and participants were required to reproduce the interval duration by pressing and releasing the space key. We manipulated the uncertainty of prior information using different interval distributions. We found no difference in the influence of prior information on interval reproduction between patients and controls. However, patients with SZ showed a stronger central bias than healthy participants in the intermediate interval range (approximately 450 ms to 900 ms). It is possible that the patients in SZ have non-uniform deficits associated with interval range or uncertainty of prior information in perceptual inference. Further, the severity of avolition and alogia was correlated with the strength of central bias in SZ. This study provides some insights into the mechanisms underlying the association between schizophrenic symptoms and perceptual inference.

The role of neural tuning in quantity perception

Perception of quantities, such as numerosity, timing, and size, is essential for behavior and cognition. Accumulating evidence demonstrates neurons processing quantities are tuned, that is, have a preferred quantity amount, not only for numerosity, but also other quantity dimensions and sensory modalities. We argue that quantity-tuned neurons are fundamental to understanding quantity perception. We illustrate how the properties of quantity-tuned neurons can underlie a range of perceptual phenomena. Furthermore, quantity-tuned neurons are organized in distinct but overlapping topographic maps. We suggest that this overlap in tuning provides the neural basis for perceptual interactions between different quantities, without the need for a common neural representational code.

Weighted Integration of Duration Information Across Visual and Auditory Modality Is Influenced by Modality-Specific Attention

We constantly integrate multiple types of information from different sensory modalities. Generally, such integration is influenced by the modality that we attend to. However, for duration perception, it has been shown that when duration information from visual and auditory modalities is integrated, the perceived duration of the visual stimulus leaned toward the duration of the auditory stimulus, irrespective of which modality was attended. In these studies, auditory dominance was assessed using visual and auditory stimuli with different durations whose timing of onset and offset would affect perception. In the present study, we aimed to investigate the effect of attention on duration integration using visual and auditory stimuli of the same duration. Since the duration of a visual flicker and auditory flutter tends to be perceived as longer than and shorter than its physical duration, respectively, we used the 10 Hz visual flicker and auditory flutter with the same onset and offset timings but different perceived durations. The participants were asked to attend either visual, auditory, or both modalities. Contrary to the attention-independent auditory dominance reported in previous studies, we found that the perceived duration of the simultaneous flicker and flutter presentation depended on which modality the participants attended. To further investigate the process of duration integration of the two modalities, we applied Bayesian hierarchical modeling, which enabled us to define a flexible model in which the multisensory duration is represented by the weighted average of each sensory modality. In addition, to examine whether auditory dominance results from the higher reliability of auditory stimuli, we applied another models to consider the stimulus reliability. These behavioral and modeling results suggest the following: (1) the perceived duration of visual and auditory stimuli is influenced by which modality the participants attended to when we control for the confounding effect of onset-offset timing of stimuli, and (2) the increase of the weight by attention affects the duration integration, even when the effect of stimulus reliability is controlled. Our models can be extended to investigate the neural basis and effects of other sensory modalities in duration integration.

Central tendency biases must be accounted for to consistently capture Bayesian cue combination in continuous response data

Observers in perceptual tasks are often reported to combine multiple sensory cues in a weighted average that improves precision—in some studies, approaching statistically optimal (Bayesian) weighting, but in others departing from optimality, or not benefitting from combined cues at all. To correctly conclude which combination rules observers use, it is crucial to have accurate measures of their sensory precision and cue weighting. Here, we present a new approach for accurately recovering these parameters in perceptual tasks with continuous responses. Continuous responses have many advantages, but are susceptible to a central tendency bias, where responses are biased towards the central stimulus value. We show that such biases lead to inaccuracies in estimating both precision gains and cue weightings, two key measures used to assess sensory cue combination. We introduce a method that estimates sensory precision by regressing continuous responses on targets and dividing the variance of the residuals by the squared slope of the regression line, “correcting-out” the error introduced by the central bias and increasing statistical power. We also suggest a complementary analysis that recovers the sensory cue weights. Using both simulations and empirical data, we show that the proposed methods can accurately estimate sensory precision and cue weightings in the presence of central tendency biases. We conclude that central tendency biases should be (and can easily be) accounted for to consistently capture Bayesian cue combination in continuous response data.

Cross-Modal Conflict Increases With Time-on-Task in a Temporal Discrimination Task

The modality appropriateness hypothesis argues that the auditory modality is preferred over the visual modality in tasks demanding temporal operations; hence, we predicted that responses to visual stimuli would be more sensitive to the detrimental effect of Time-on-Task. We used a bimodal temporal discrimination task. The factors were durational congruency between the modalities and the direction of modality-transmission. Participants needed to decide the duration of the cued stimulus (visual or auditory). The first five blocks of the task lasted about 1.5 h without rest [Time-on-Task (ToT) period]. The participants then had a 12-min break followed by an additional block of trials. Subjective fatigue, reaction time, error rates, and electrocardiographic data were recorded. In the visual modality, we found an enhanced congruency effect as a function of ToT. The cost of attentional shifting was higher in the auditory modality, but remained constant, suggesting that processing of auditory stimuli is robust against the effects of fatigue. Performance did not improve after the break, indicating that the effects of fatigue could not be overcome by taking a brief break. The heart rate variability (HRV) data showed that vagal inhibition increased with ToT, but this increase was not associated with the changes in performance.

1-s Productions: A Validation of an Efficient Measure of Clock Variability

Objective: Clock variance is an important statistic in many clinical and developmental studies. Existing methods require a large number of trials for accurate clock variability assessment, which is problematic in studies using clinical or either young or aged participants. Furthermore, these existing methods often implicitly convolute clock and memory processes, making it difficult to disentangle whether the clock or memory system are driving the observed deviations. Here we assessed whether 20 repeated productions of a well-engrained interval (1 s), a task that does not incorporate memory updating nor the processing of feedback, could provide an accurate assessment of clock variability.

Method: Sixty-eight undergraduate students completed two tasks: a 1-s production task in which they were asked to produce a 1-s duration by ending a tone by a keypress, and a multi-duration reproduction task. Durations presented in the reproduction task were tones lasting 1.17, 1.4 and 1.68 s. No feedback was presented in either task, and the order of presentation was counterbalanced between participants.

Results: The observed central tendency in the reproduction task was better explained by models including the measures of clock variability derived from the 1-s production task than by models without it. Three clock variability measures were calculated for each participant [standard deviation, root mean squared residuals (RMSRs) from an estimated linear slope, and RMSR scaled by mean production duration]. The model including the scaled RMSR was preferred over the alternative models, and no notable effects of the order of task presentation were observed. These results suggest that: (1) measures of variability should account for drift; (2) the presentation of another timing task before a 1-s production task did not influence the assessment of the clock variability; and (3) the observed variability adheres to the scalar property and predicts temporal performance, and is thus a usable index of clock variability.

Conclusion: This study shows that just 20 repeated productions of 1 s provide a reliable index of clock variability. As administering this task is fast and easy, it could prove to be useful in a large variety of developmental and clinical populations.