Computational Methods for Continuous Eye-Tracking Perimetry Based on Spatio-Temporal Integration and a Deep Recurrent Neural Network

Continuous psychophysics for two-variable experiments; A new "Bayesian participant" approach

Interest in continuous psychophysical approaches as a means of collecting data quickly under natural conditions is growing. Such approaches require stimuli to be changed randomly on a continuous basis so that participants can not guess future stimulus states. Participants are generally tasked with responding continuously using a continuum of response options. These features introduce variability in the data that is not present in traditional trial-based experiments. Given the unique weaknesses and strengths of continuous psychophysical approaches, we propose that they are well suited to quickly mapping out relationships between above-threshold stimulus variables such as the perceived direction of a moving target as a function of the direction of the background against which the target is moving. We show that modelling the participant in such a two-variable experiment using a novel "Bayesian Participant" model facilitates the conversion of the noisy continuous data into a less-noisy form that resembles data from an equivalent trial-based experiment. We also show that adaptation can result from longer-than-usual stimulus exposure times during continuous experiments, even to features that the participant is not aware of. Methods for mitigating the effects of adaptation are discussed.

Contrastive self-supervised representation learning without negative samples for multimodal human action recognition

Action recognition is an important component of human-computer interaction, and multimodal feature representation and learning methods can be used to improve recognition performance due to the interrelation and complementarity between different modalities. However, due to the lack of large-scale labeled samples, the performance of existing ConvNets-based methods are severely constrained. In this paper, a novel and effective multi-modal feature representation and contrastive self-supervised learning framework is proposed to improve the action recognition performance of models and the generalization ability of application scenarios. The proposed recognition framework employs weight sharing between two branches and does not require negative samples, which could effectively learn useful feature representations by using multimodal unlabeled data, e.g., skeleton sequence and inertial measurement unit signal (IMU). The extensive experiments are conducted on two benchmarks: UTD-MHAD and MMAct, and the results show that our proposed recognition framework outperforms both unimodal and multimodal baselines in action retrieval, semi-supervised learning, and zero-shot learning scenarios.

Stimulus contrast, pursuit mode, and age strongly influence tracking performance on a continuous visual tracking task

Human observers tend to naturally track moving stimuli. This tendency may be exploited towards an intuitive means of screening visual function as an impairment induced reduction in stimulus visibility will decrease tracking performance. Yet, to be able to detect subtle impairments, stimulus contrast is critical. If too high, the decrease in performance may remain undetected. Therefore, for this approach to become reliable and sensitive, we need a detailed understanding of how age, stimulus contrast, and the type of stimulus movement affect continuous tracking performance. To do so, we evaluated how well twenty younger and twenty older participants tracked a semi-randomly moving stimulus (Goldmann size III, 0.43 degrees of visual angle), presented at five contrast levels (5%-10%-20%-40%-80%). The stimulus could move smoothly only (smooth pursuit mode) or in alternation with displacements (saccadic pursuit mode). Additionally, we assessed static foveal and peripheral contrast thresholds. For all participants, tracking performance improved with increasing contrast in both pursuit modes. To reach threshold performance levels, older participants required about twice as much contrast (20% vs. 10% and 40% vs. 20% in smooth and saccadic modes respectively). Saccadic pursuit detection thresholds correlated significantly with static peripheral contrast thresholds (rho = 0.64). Smooth pursuit detection thresholds were uncorrelated with static foveal contrast thresholds (rho = 0.29). We conclude that continuous visual stimulus tracking is strongly affected by stimulus contrast, pursuit mode, and age. This provides essential insights that can be applied towards new and intuitive approaches of screening visual function.

Putting perception into action with inverse optimal control for continuous psychophysics

Psychophysical methods are a cornerstone of psychology, cognitive science, and neuroscience where they have been used to quantify behavior and its neural correlates for a vast range of mental phenomena. Their power derives from the combination of controlled experiments and rigorous analysis through signal detection theory. Unfortunately, they require many tedious trials and preferably highly trained participants. A recently developed approach, continuous psychophysics, promises to transform the field by abandoning the rigid trial structure involving binary responses and replacing it with continuous behavioral adjustments to dynamic stimuli. However, what has precluded wide adoption of this approach is that current analysis methods do not account for the additional variability introduced by the motor component of the task and therefore recover perceptual thresholds that are larger compared to equivalent traditional psychophysical experiments. Here, we introduce a computational analysis framework for continuous psychophysics based on Bayesian inverse optimal control. We show via simulations and previously published data that this not only recovers the perceptual thresholds but additionally estimates subjects' action variability, internal behavioral costs, and subjective beliefs about the experimental stimulus dynamics. Taken together, we provide further evidence for the importance of including acting uncertainties, subjective beliefs, and, crucially, the intrinsic costs of behavior, even in experiments seemingly only investigating perception.