Bayes Optimal Integration of Social and Endogenous Uncertainty in Numerosity Estimation

One of the most prominent social influences on human decision making is conformity, which is even more prominent when the perceptual information is ambiguous. The Bayes optimal solution to this problem entails weighting the relative reliability of cognitive information and perceptual signals in constructing the percept from self-sourced/endogenous and social sources, respectively. The current study investigated whether humans integrate the statistics (i.e., mean and variance) of endogenous perceptual and social information in a Bayes optimal way while estimating numerosities. Our results demonstrated adjustment of initial estimations toward group means only when group estimations were more reliable (or "certain"), compared to participants' endogenous metric uncertainty. Our results support Bayes optimal social conformity while also pointing to an implicit form of metacognition.

Bayesian approaches to smooth pursuit of random dot kinematograms: effects of varying RDK noise and the predictability of RDK direction

Smooth pursuit eye movements respond on the basis of both immediate and anticipated target motion, where anticipations may be derived from either memory or perceptual cues. To study the combined influence of both immediate sensory motion and anticipation, subjects pursued clear or noisy random dot kinematograms (RDKs) whose mean directions were chosen from Gaussian distributions with SDs = 10° (narrow prior) or 45° (wide prior). Pursuit directions were consistent with Bayesian theory in that transitions over time from dependence on the prior to near total dependence on immediate sensory motion (likelihood) took longer with the noisier RDKs and with the narrower, more reliable, prior. Results were fit to Bayesian models in which parameters representing the variability of the likelihood either were or were not constrained to be the same for both priors. The unconstrained model provided a statistically better fit, with the influence of the prior in the constrained model smaller than predicted from strict reliability-based weighting of prior and likelihood. Factors that may have contributed to this outcome include prior variability different from nominal values, low-level sensorimotor learning with the narrow prior, or departures of pursuit from strict adherence to reliability-based weighting. Although modifications of, or alternatives to, the normative Bayesian model will be required, these results, along with previous studies, suggest that Bayesian approaches are a promising framework to understand how pursuit combines immediate sensory motion, past history, and informative perceptual cues to accurately track the target motion that is most likely to occur in the immediate future.NEW & NOTEWORTHY Smooth pursuit eye movements respond on the basis of anticipated, as well as immediate, target motions. Bayesian models using reliability-based weighting of previous (prior) and immediate target motions (likelihood) accounted for many, but not all, aspects of pursuit of clear and noisy random dot kinematograms with different levels of predictability. Bayesian approaches may solve the long-standing problem of how pursuit combines immediate sensory motion and anticipation of future motion to configure an effective response.

Working memory for time intervals: Another manifestation of the central tendency effect

The relationship between working memory and time perception has been typically investigated using dual-task paradigms (e.g., testing timing performance during a concurrent task). To our knowledge, none of these studies used time intervals as the target stimulus to be remembered. The current study investigated the working memory for time intervals by asking participants to reproduce durations they experienced at different orders in a series of experienced intervals (n-back task). One of the experiments was conducted online and the other one in the lab setting. Results showed a central tendency bias and additive elongation of time reproductions with increasing working memory load. Our results also showed that participants assigned different weights to experienced intervals based on their order of presentation (higher weight to the target interval). We conclude that the recall of intervals from working memory under high cognitive load leads to a central tendency effect, which is known to be induced by the temporal context and present particularly in aging and in those with Parkinson's disease.

Bayesian integration during sensorimotor estimation in elite athletes

An experiment was designed to determine the effects of sensory uncertainty on sensorimotor estimation in elite athletes compared to non-athletes. Nineteen elite athletes and 16 non-athletes were required to estimate when and where a cursor arrived at a target location. The cursor position was displayed through its entire trajectory in the certain condition while only briefly in the uncertain condition. Accuracy and variability in time and spatial domains were calculated. A Bayesian analysis using subsets of subjects' total spatial variance was also performed. The results indicated that athletes and non-athletes used estimation strategies consistent with Bayesian integration. The results also showed a decrease in variability for spatial performance for both groups during the uncertain condition compared to the certain condition, especially when the cursor location was further away from the prior mean. This decrease in variability was significantly greater for non-athletes. By concentrating performance around the end-point mean location, an increase in spatial error occurred. More spatial and timing errors were observed in non-athletes than athletes, indicating athletes were more certain about likelihood information or their interpretation of likelihood information than non-athletes. These results suggest that athletic experience may facilitate the use of probabilistic information for optimal sensorimotor estimations.

Validating model-based Bayesian integration using prior-cost metamers

There are two competing views on how humans make decisions under uncertainty. Bayesian decision theory posits that humans optimize their behavior by establishing and integrating internal models of past sensory experiences (priors) and decision outcomes (cost functions). An alternative hypothesis posits that decisions are optimized through trial and error without explicit internal models for priors and cost functions. To distinguish between these possibilities, we introduce a paradigm that probes the sensitivity of humans to transitions between prior-cost pairs that demand the same optimal policy (metamers) but distinct internal models. We demonstrate the utility of our approach in two experiments that were classically explained by Bayesian theory. Our approach validates the Bayesian learning strategy in an interval timing task but not in a visuomotor rotation task. More generally, our work provides a domain-general approach for testing the circumstances under which humans explicitly implement model-based Bayesian computations.

Deficits in multi-scale top-down processes distorting auditory perception in schizophrenia

Cognitive models postulate that impaired source monitoring incorrectly weights the top-down prediction and bottom-up sensory processes and causes hallucinations. However, the underlying mechanisms of the interaction, such as whether the incorrectly weighting is ubiquitously on all levels of sensory features and whether different top-down processes have distinct effects in subgroups of schizophrenia are still unclear. This study investigates how multi-scale predictions influence perception of basic tonal features in schizophrenia. Sixty-three schizophrenia patients with and without symptoms of auditory verbal hallucinations (AVHs), and thirty healthy controls identified target tones in noise at the end of tone sequences. Predictions of different timescales were manipulated by either an alternating pattern in the preceding tone sequences (long-term regularity) or a repetition between the target tone and the tone immediately before (short-term repetition). The sensitivity index, d prime (d'), was obtained to assess the modulation of predictions on tone identification. Patients with AVHs showed higher d' when the target tones conformed to the long-term regularity of alternating pattern in the preceding tone sequence than when the target tones were inconsistent with the pattern. Whereas, the short-term repetition modulated the tone identification in patients without AVHs. Predictions did not influence tone identification in healthy controls. Our results suggest that impaired source monitoring in schizophrenia patients with AVHs heavily weights top-down predictions over bottom-up perceptual processes to form incorrect perception. The weighting function in source monitoring can extend to the processes of basic tonal features, and predictions at multiple timescales could differentially modulate perception in different clinical populations. The impaired interaction between top-down and bottom-up processes might underlie the development of hallucination symptoms in schizophrenia.

FunCoup 5: Functional Association Networks in All Domains of Life, Supporting Directed Links and Tissue-Specificity

FunCoup (https://funcoup.sbc.su.se) is one of the most comprehensive functional association networks of genes/proteins available. Functional associations are inferred by integrating different types of evidence using a redundancy-weighted naïve Bayesian approach, combined with orthology transfer. FunCoup's high coverage comes from using eleven different types of evidence, and extensive transfer of information between species. Since the latest update of the database, the availability of source data has improved drastically, and user expectations on a tool for functional associations have grown. To meet these requirements, we have made a new release of FunCoup with updated source data and improved functionality. FunCoup 5 now includes 22 species from all domains of life, and the source data for evidences, gold standards, and genomes have been updated to the latest available versions. In this new release, directed regulatory links inferred from transcription factor binding can be visualized in the network viewer for the human interactome. Another new feature is the possibility to filter by genes expressed in a certain tissue in the network viewer. FunCoup 5 further includes the SARS-CoV-2 proteome, allowing users to visualize and analyze interactions between SARS-CoV-2 and human proteins in order to better understand COVID-19. This new release of FunCoup constitutes a major advance for the users, with updated sources, new species and improved functionality for analysis of the networks.

Optimality and Limitations of Audio-Visual Integration for Cognitive Systems

Multimodal integration is an important process in perceptual decision-making. In humans, this process has often been shown to be statistically optimal, or near optimal: sensory information is combined in a fashion that minimizes the average error in perceptual representation of stimuli. However, sometimes there are costs that come with the optimization, manifesting as illusory percepts. We review audio-visual facilitations and illusions that are products of multisensory integration, and the computational models that account for these phenomena. In particular, the same optimal computational model can lead to illusory percepts, and we suggest that more studies should be needed to detect and mitigate these illusions, as artifacts in artificial cognitive systems. We provide cautionary considerations when designing artificial cognitive systems with the view of avoiding such artifacts. Finally, we suggest avenues of research toward solutions to potential pitfalls in system design. We conclude that detailed understanding of multisensory integration and the mechanisms behind audio-visual illusions can benefit the design of artificial cognitive systems.

Bayesian Computation through Cortical Latent Dynamics

Statistical regularities in the environment create prior beliefs that we rely on to optimize our behavior when sensory information is uncertain. Bayesian theory formalizes how prior beliefs can be leveraged and has had a major impact on models of perception, sensorimotor function, and cognition. However, it is not known how recurrent interactions among neurons mediate Bayesian integration. By using a time-interval reproduction task in monkeys, we found that prior statistics warp neural representations in the frontal cortex, allowing the mapping of sensory inputs to motor outputs to incorporate prior statistics in accordance with Bayesian inference. Analysis of recurrent neural network models performing the task revealed that this warping was enabled by a low-dimensional curved manifold and allowed us to further probe the potential causal underpinnings of this computational strategy. These results uncover a simple and general principle whereby prior beliefs exert their influence on behavior by sculpting cortical latent dynamics.

The material-weight illusion disappears or inverts in objects made of two materials

The material-weight illusion (MWI) occurs when an object that looks heavy (e.g., stone) and one that looks light (e.g., Styrofoam) have the same mass. When such stimuli are lifted, the heavier-looking object feels lighter than the lighter-looking object, presumably because well-learned priors about the density of different materials are violated. We examined whether a similar illusion occurs when a certain weight distribution is expected (such as the metal end of a hammer being heavier), but weight is uniformly distributed. In experiment 1, participants lifted bipartite objects that appeared to be made of two materials (combinations of stone, Styrofoam, and wood) but were manipulated to have a uniform weight distribution. Most participants experienced an inverted MWI (i.e., the heavier-looking side felt heavier), suggesting an integration of incoming sensory information with density priors. However, a replication of the classic MWI was found when the objects appeared to be uniformly made of just one of the materials (experiment 2). Both illusions seemed to be independent of the forces used when the objects were lifted. When lifting bipartite objects but asked to judge the weight of the whole object, participants experienced no illusion (experiment 3). In experiment 4, we investigated weight perception in objects with a nonuniform weight distribution and again found evidence for an integration of prior and sensory information. Taken together, our seemingly contradictory results challenge most theories about the MWI. However, Bayesian integration of competing density priors with the likelihood of incoming sensory information may explain the opposing illusions.

NEW & NOTEWORTHY We report a novel weight illusion that contradicts all current explanations of the material-weight illusion: When lifting an object composed of two materials, the heavier-looking side feels heavier, even when the true weight distribution is uniform. The opposite (classic) illusion is found when the same materials are lifted in two separate objects. Identifying the common mechanism underlying both illusions will have implications for perception more generally. A potential candidate is Bayesian inference with competing priors.

Interlimb Generalization of Learned Bayesian Visuomotor Prior Occurs in Extrinsic Coordinates

Recent work suggests that the brain represents probability distributions and performs Bayesian integration during sensorimotor learning. However, our understanding of the neural representation of this learning remains limited. To begin to address this, we performed two experiments. In the first experiment, we replicated the key behavioral findings of Körding and Wolpert (2004), demonstrating that humans can perform in a Bayes-optimal manner by combining information about their own sensory uncertainty and a statistical distribution of lateral shifts encountered in a visuomotor adaptation task. In the second experiment, we extended these findings by testing whether visuomotor learning occurring during the same task generalizes from one limb to the other, and relatedly, whether this learning is represented in an extrinsic or intrinsic reference frame. We found that the learned mean of the distribution of visuomotor shifts generalizes to the opposite limb only when the perturbation is congruent in extrinsic coordinates, indicating that the underlying representation of learning acquired during training is available to the untrained limb and is coded in an extrinsic reference frame.

Psychophysical Evaluation of Sensory Reweighting in Bilateral Vestibulopathy

Perception of spatial orientation is thought to rely on the brain's integration of visual, vestibular, proprioceptive, and somatosensory signals, as well as internal beliefs. When one of these signals breaks down, such as the vestibular signal in bilateral vestibulopathy, patients start compensating by relying more on the remaining cues. How these signals are reweighted in this integration process is difficult to establish, since they cannot be measured in isolation during natural tasks, are inherently noisy, and can be ambiguous or in conflict. Here, we review our recent work, combining experimental psychophysics with a reverse engineering approach, based on Bayesian inference principles, to quantify sensory noise levels and optimal (re)weighting at the individual subject level, in both patients with bilateral vestibular deficits and healthy controls. We show that these patients reweight the remaining sensory information, relying more on visual and other nonvestibular information than healthy controls in the perception of spatial orientation. This quantification approach could improve diagnostics and prognostics of multisensory integration deficits in vestibular patients, and contribute to an evaluation of rehabilitation therapies directed toward specific training programs.

Both Intrinsic Substrate Preference and Network Context Contribute to Substrate Selection of Classical Tyrosine Phosphatases

Reversible tyrosine phosphorylation is a widespread post-translational modification mechanism underlying cell physiology. Thus, understanding the mechanisms responsible for substrate selection by kinases and phosphatases is central to our ability to model signal transduction at a system level. Classical protein-tyrosine phosphatases can exhibit substrate specificity in vivo by combining intrinsic enzymatic specificity with the network of protein-protein interactions, which positions the enzymes in close proximity to their substrates. Here we use a high throughput approach, based on high density phosphopeptide chips, to determine the in vitro substrate preference of 16 members of the protein-tyrosine phosphatase family. This approach helped identify one residue in the substrate binding pocket of the phosphatase domain that confers specificity for phosphopeptides in a specific sequence context. We also present a Bayesian model that combines intrinsic enzymatic specificity and interaction information in the context of the human protein interaction network to infer new phosphatase substrates at the proteome level.

Perceptual attraction in tool use: evidence for a reliability-based weighting mechanism

Humans are well able to operate tools whereby their hand movement is linked, via a kinematic transformation, to a spatially distant object moving in a separate plane of motion. An everyday example is controlling a cursor on a computer monitor. Despite these separate reference frames, the perceived positions of the hand and the object were found to be biased toward each other. We propose that this perceptual attraction is based on the principles by which the brain integrates redundant sensory information of single objects or events, known as optimal multisensory integration. That is, 1) sensory information about the hand and the tool are weighted according to their relative reliability (i.e., inverse variances), and 2) the unisensory reliabilities sum up in the integrated estimate. We assessed whether perceptual attraction is consistent with optimal multisensory integration model predictions. We used a cursor-control tool-use task in which we manipulated the relative reliability of the unisensory hand and cursor position estimates. The perceptual biases shifted according to these relative reliabilities, with an additional bias due to contextual factors that were present in experiment 1 but not in experiment 2 The biased position judgments' variances were, however, systematically larger than the predicted optimal variances. Our findings suggest that the perceptual attraction in tool use results from a reliability-based weighting mechanism similar to optimal multisensory integration, but that certain boundary conditions for optimality might not be satisfied.NEW & NOTEWORTHY Kinematic tool use is associated with a perceptual attraction between the spatially separated hand and the effective part of the tool. We provide a formal account for this phenomenon, thereby showing that the process behind it is similar to optimal integration of sensory information relating to single objects.

Post-Movement Beta Activity in Sensorimotor Cortex Indexes Confidence in the Estimations from Internal Models

Beta oscillations are a dominant feature of the sensorimotor system. A transient and prominent increase in beta oscillations is consistently observed across the sensorimotor cortical-basal ganglia network after cessation of voluntary movement: the post-movement beta synchronization (PMBS). Current theories about the function of the PMBS have been focused on either the closure of motor response or the processing of sensory afferance. Computational models of sensorimotor control have emphasized the importance of the integration between feedforward estimation and sensory feedback, and therefore the putative motor and sensory functions of beta oscillations may reciprocally interact with each other and in fact be indissociable. Here we show that the amplitude of sensorimotor PMBS is modulated by the history of visual feedback of task-relevant errors, and negatively correlated with the trial-to-trial exploratory adjustment in a sensorimotor adaptation task in young healthy human subjects. The PMBS also negatively correlated with the uncertainty associated with the feedforward estimation, which was recursively updated in light of new sensory feedback, as identified by a Bayesian learning model. These results reconcile the two opposing motor and sensory views of the function of PMBS, and suggest a unifying theory in which PMBS indexes the confidence in internal feedforward estimation in Bayesian sensorimotor integration. Its amplitude simultaneously reflects cortical sensory processing and signals the need for maintenance or adaptation of the motor output, and if necessary, exploration to identify an altered sensorimotor transformation.

SIGNIFICANCE STATEMENT

For optimal sensorimotor control, sensory feedback and feedforward estimation of a movement's sensory consequences should be weighted by the inverse of their corresponding uncertainties, which require recursive updating in a dynamic environment. We show that post-movement beta activity (13-30 Hz) over sensorimotor cortex in young healthy subjects indexes the evaluation of uncertainty in feedforward estimation. Our work contributes to the understanding of the function of beta oscillations in sensorimotor control, and provides further insight into how aberrant beta activity can contribute to the pathophysiology of movement disorders.

Neural mechanisms for integrating prior knowledge and likelihood in value-based probabilistic inference

In Bayesian decision theory, knowledge about the probabilities of possible outcomes is captured by a prior distribution and a likelihood function. The prior reflects past knowledge and the likelihood summarizes current sensory information. The two combined (integrated) form a posterior distribution that allows estimation of the probability of different possible outcomes. In this study, we investigated the neural mechanisms underlying Bayesian integration using a novel lottery decision task in which both prior knowledge and likelihood information about reward probability were systematically manipulated on a trial-by-trial basis. Consistent with Bayesian integration, as sample size increased, subjects tended to weigh likelihood information more compared with prior information. Using fMRI in humans, we found that the medial prefrontal cortex (mPFC) correlated with the mean of the posterior distribution, a statistic that reflects the integration of prior knowledge and likelihood of reward probability. Subsequent analysis revealed that both prior and likelihood information were represented in mPFC and that the neural representations of prior and likelihood in mPFC reflected changes in the behaviorally estimated weights assigned to these different sources of information in response to changes in the environment. Together, these results establish the role of mPFC in prior-likelihood integration and highlight its involvement in representing and integrating these distinct sources of information.

Multi-sensory weights depend on contextual noise in reference frame transformations

During reach planning, we integrate multiple senses to estimate the location of the hand and the target, which is used to generate a movement. Visual and proprioceptive information are combined to determine the location of the hand. The goal of this study was to investigate whether multi-sensory integration is affected by extraretinal signals, such as head roll. It is believed that a coordinate matching transformation is required before vision and proprioception can be combined because proprioceptive and visual sensory reference frames do not generally align. This transformation utilizes extraretinal signals about current head roll position, i.e., to rotate proprioceptive signals into visual coordinates. Since head roll is an estimated sensory signal with noise, this head roll dependency of the reference frame transformation should introduce additional noise to the transformed signal, reducing its reliability and thus its weight in the multi-sensory integration. To investigate the role of noisy reference frame transformations on multi-sensory weighting, we developed a novel probabilistic (Bayesian) multi-sensory integration model (based on Sober and Sabes, 2003) that included explicit (noisy) reference frame transformations. We then performed a reaching experiment to test the model's predictions. To test for head roll dependent multi-sensory integration, we introduced conflicts between viewed and actual hand position and measured reach errors. Reach analysis revealed that eccentric head roll orientations led to an increase of movement variability, consistent with our model. We further found that the weighting of vision and proprioception depended on head roll, which we interpret as being a result of signal dependant noise. Thus, the brain has online knowledge of the statistics of its internal sensory representations. In summary, we show that sensory reliability is used in a context-dependent way to adjust multi-sensory integration weights for reaching.