Bayesian and "anti-Bayesian" biases in sensory integration for action and perception in the size-weight illusion

Investigating how prior knowledge influences perception and action in developmental coordination disorder

Developmental coordination disorder (DCD) is characterised by a broad spectrum of difficulties in performing motor tasks. It has recently been proposed that a specific deficit in sensorimotor prediction and feedforward planning might underpin these motoric impairments. The purpose of this study was to use a naturalistic object lifting paradigm to examine whether deficits in sensorimotor prediction might underpin the broad spectrum of difficulties individuals with DCD face when interacting with objects in their environment. We recruited 60 children with probable DCD and 61 children without DCD and measured perceptions of heaviness and fingertip force rate application when interacting with objects which varied in their apparent weight. If deficits in sensorimotor prediction do underpin the broad-ranging motor difficulties seen in DCD, we would expect to see a reduced effect of visual size cues on fingertip force rates and illusory misperceptions of object heaviness. We found no evidence of differences in any metrics of sensorimotor prediction between children with (n = 46) and without DCD (n = 61). Furthermore, there was no correlation between any metrics of sensorimotor prediction and motor performance (as assessed by the standard diagnostic movement assessment battery). Illusory misperceptions of object weight also did not appear to differ between groups. These findings suggest that issues with sensorimotor prediction are unlikely to affect the performance of simple real-world movements in those with DCD.

Visual experience shapes the Bouba-Kiki effect and the size-weight illusion upon sight restoration from congenital blindness

The Bouba-Kiki effect is the systematic mapping between round/spiky shapes and speech sounds ("Bouba"/"Kiki"). In the size-weight illusion, participants judge the smaller of two equally-weighted objects as being heavier. Here we investigated the contribution of visual experience to the development of these phenomena. We compared three groups: early blind individuals (no visual experience), individuals treated for congenital cataracts years after birth (late visual experience), and typically sighted controls (visual experience from birth). We found that, in cataract-treated participants (tested visually/visuo-haptically), both phenomena are absent shortly after sight onset, just like in blind individuals (tested haptically). However, they emerge within months following surgery, becoming statistically indistinguishable from the sighted controls. This suggests a pivotal role of visual experience and refutes the existence of an early sensitive period: A short period of experience, even when gained only years after birth, is sufficient for participants to visually pick-up regularities in the environment, contributing to the development of these phenomena.

F = ma. Is the macaque brain Newtonian?

Intuitive Physics, the ability to anticipate how the physical events involving mass objects unfold in time and space, is a central component of intelligent systems. Intuitive physics is a promising tool for gaining insight into mechanisms that generalize across species because both humans and non-human primates are subject to the same physical constraints when engaging with the environment. Physical reasoning abilities are widely present within the animal kingdom, but monkeys, with acute 3D vision and a high level of dexterity, appreciate and manipulate the physical world in much the same way humans do.

Effect on Perceived Weight of Object Shapes

The perceived weight of an object is an important research topic in terms of sensation and perception, and it is known that it has size-weight, color-weight, and material-weight illusions due to the influence of size, color, and material, as well as the weight of the object. Although the physical size of an object is measured by volume, the size of an object that we subjectively feel depends on the shape of the object, even if it has the same volume. Therefore, the shape of the object may determine the perceived size of the object, thereby changing its perceived weight accordingly. These cognitive factors play an important role in the period of rehabilitation therapy after an exacerbation or attack of neurological diseases, such as stroke or Parkinson's disease, regarding the motor functions of the patient. Moreover, the study of these sensation and perception factors is important for the period of the early development of children, for example, for tracking and correcting fine motor skills. Existing related studies analyzed the perceived weight according to three shapes (tetrahedron, cube, and sphere), but only some shapes showed a difference in the perceived weight. This study attempted to demonstrate the difference in perceived weight according to the shape that has yet to be clearly identified. To this end, this study investigated objects with the same physical size (volume) as in previous studies, but in the shapes of tetrahedron, cube, and sphere. In addition, the volumes of these objects were set to 64,000 cm³, 125,000 cm³, and 216,000 cm³, and their weights were set to be 100 g, 150 g, and 200 g, in proportion to the size of the small, medium, and large volumes, respectively. Thirty-eight college students (21 males, 17 females) participated and the perceived weight of a given object compared to a reference object was evaluated according to the modulus method used for sensory size measurement. The analysis of the experimental data found that both weight (volume) and shape had significant effects on the perceived weight. The results support that the shape of objects also led to the size-weight illusion phenomenon. At the same weight (volume), the perceived weight of an object according to shape decreased significantly in the order of sphere, cube, and tetrahedron. At the same volume level, subjective size according to shape is small in the order of tetrahedron, cube, and sphere. The results of weight perception according to shape in this study showed that the subjective size of an object according to shape had an effect on perceived weight.

Bayesian causal inference: A unifying neuroscience theory

Understanding of the brain and the principles governing neural processing requires theories that are parsimonious, can account for a diverse set of phenomena, and can make testable predictions. Here, we review the theory of Bayesian causal inference, which has been tested, refined, and extended in a variety of tasks in humans and other primates by several research groups. Bayesian causal inference is normative and has explained human behavior in a vast number of tasks including unisensory and multisensory perceptual tasks, sensorimotor, and motor tasks, and has accounted for counter-intuitive findings. The theory has made novel predictions that have been tested and confirmed empirically, and recent studies have started to map its algorithms and neural implementation in the human brain. The parsimony, the diversity of the phenomena that the theory has explained, and its illuminating brain function at all three of Marr's levels of analysis make Bayesian causal inference a strong neuroscience theory. This also highlights the importance of collaborative and multi-disciplinary research for the development of new theories in neuroscience.

Size, weight, and expectations

The size-weight illusion is well-known: if two equally heavy objects differ in size, the large one feels lighter than the small one. Most explanations for this illusion assume that because the information about the relevant attribute (weight itself) is unreliable, information about an irrelevant but correlated attribute (size) is used as well. If such reasoning is correct, one would expect that the illusion can be inverted: if size information is unreliable, weight information will be used to judge size. We explored whether such a weight-size illusion exists by asking participants to lift Styrofoam balls that were coated with glow in the dark paint. The balls (2 sizes, 3 weights) were lifted using a pulley system in complete darkness at 2 distances. Participants reported the size using free magnitude estimation. The visual size information was indeed unreliable: balls that were presented at a 20% larger distance were judged 15% smaller. Nevertheless, the judgments of size were not systematically affected by the 20% weight change (differences < 0.5%). We conclude that because the weight-size illusion does not exist, the mechanism behind the size-weight illusion is specific for judging heaviness.

Familiarity with an Object’s Size Influences the Perceived Size of Its Image

It is known that judgments about objects’ distances are influenced by familiar size: a soccer ball looks farther away than a tennis ball if their images are equally large on the retina. We here investigate whether familiar size also influences judgments about the size of images of objects that are presented side-by-side on a computer screen. Sixty-three participants indicated which of two images appeared larger on the screen in a 2-alternative forced-choice discrimination task. The objects were either two different types of balls, two different types of coins, or a ball and a grey disk. We found that the type of ball biased the comparison between their image sizes: the size of the image of the soccer ball was over-estimated by about 5% (assimilation). The bias in the comparison between the two balls was equal to the sum of the biases in the comparisons with the grey disk. The bias for the coins was smaller and in the opposite direction (contrast). The average precision of the size comparison was 3.5%, irrespective of the type of object. We conclude that knowing a depicted object’s real size can influence the perceived size of its image, but the perceived size is not always attracted towards the familiar size.

"Impossible" Somatosensation and the (Ir)rationality of Perception

Impossible figures represent the world in ways it cannot be. From the work of M. C. Escher to any popular perception textbook, such experiences show how some principles of mental processing can be so entrenched and inflexible as to produce absurd and even incoherent outcomes that could not occur in reality. However, impossible experiences of this sort are mostly limited to visual perception; are there “impossible figures” for other sensory modalities? Here, we import a known magic trick into the laboratory to report and investigate an impossible experience for somatosensation—one that can be physically felt. We show that, even under full-cue conditions with objects that can be freely inspected, subjects can be made to experience a single object alone as feeling heavier than a group of objects that includes the single object as a member—an impossible and phenomenologically striking experience of weight. Moreover, we suggest that this phenomenon—a special case of the size-weight illusion—reflects a kind of “anti-Bayesian” perceptual updating that amplifies a challenge to rational models of perception and cognition.

Force Anticipation and Its Potential Implications on Feedforward and Feedback Human Motor Control

OBJECTIVE

To investigate the effects of human force anticipation, we conducted an experimental load-pushing task with diverse combinations of informed and actual loading weights.

BACKGROUND

Human motor control tends to rely upon the anticipated workload to plan the force to exert, particularly in fast tasks such as pushing objects in less than 1 s. The motion and force responses in such tasks may depend on the anticipated resistive forces, based on a learning process.

METHOD

Pushing performances of 135 trials were obtained from 9 participants. We varied the workload by changing the masses from 0.2 to 5 kg. To influence anticipation, participants were shown a display of the workload that was either correct or incorrect. We collected the motion and force data, as well as electromyography (EMG) signals from the actively used muscle groups.

RESULTS

Overanticipation produced overshoot performances in more than 80% of trials. Lighter actual workloads were also associated with overshoot. Pushing behaviors with heavier workloads could be classified into feedforward-dominant and feedback-dominant responses based on the timing of force, motion, and EMG responses. In addition, we found that the preceding trial condition affected the performance of the subsequent trial.

CONCLUSION

Our results show that the first peak of the pushing force increases consistently with anticipatory workload.

APPLICATION

This study improves our understanding of human motion control and can be applied to situations such as simulating interactions between drivers and assistive systems in intelligent vehicles.

Intrinsic motivation in virtual assistant interaction for fostering spontaneous interactions

With the growing utility of today’s conversational virtual assistants, the importance of user motivation in human–artificial intelligence interactions is becoming more obvious. However, previous studies in this and related fields, such as human–computer interaction, scarcely discussed intrinsic motivation (the motivation to interact with the assistants for fun). Previous studies either treated motivation as an inseparable concept or focused on non-intrinsic motivation (the motivation to interact with the assistant for utilitarian purposes). The current study aims to cover intrinsic motivation by taking an affective engineering approach. A novel motivation model is proposed, in which intrinsic motivation is affected by two factors that derive from user interactions with virtual assistants: expectation of capability and uncertainty. Experiments in which these two factors are manipulated by making participants believe they are interacting with the smart speaker “Amazon Echo” are conducted. Intrinsic motivation is measured both by using questionnaires and by covertly monitoring a five-minute free-choice period in the experimenter’s absence, during which the participants could decide for themselves whether to interact with the virtual assistants. Results of the first experiment showed that high expectation engenders more intrinsically motivated interaction compared with low expectation. However, the results did not support our hypothesis that expectation and uncertainty have an interaction effect on intrinsic motivation. We then revised our hypothetical model of action selection accordingly and conducted a verification experiment of the effects of uncertainty. Results of the verification experiment showed that reducing uncertainty encourages more interactions and causes the motivation behind these interactions to shift from non-intrinsic to intrinsic.

Predictive sensorimotor control in autism

Autism spectrum disorder has been characterized by atypicalities in how predictions and sensory information are processed in the brain. To shed light on this relationship in the context of sensorimotor control, we assessed prediction-related measures of cognition, perception, gaze and motor functioning in a large general population (n = 92; Experiment 1) and in clinically diagnosed autistic participants (n = 29; Experiment 2). In both experiments perception and action were strongly driven by prior expectations of object weight, with large items typically predicted to weigh more than equally-weighted smaller ones. Interestingly, these predictive action models were used comparably at a sensorimotor level in both autistic and neurotypical individuals with varying levels of autistic-like traits. Specifically, initial fingertip force profiles and resulting action kinematics were both scaled according to participants' pre-lift heaviness estimates, and generic visual sampling behaviours were notably consistent across groups. These results suggest that the weighting of prior information is not chronically underweighted in autism, as proposed by simple Bayesian accounts of the disorder. Instead, our results cautiously implicate context-sensitive processing mechanisms, such as precision modulation and hierarchical volatility inference. Together, these findings present novel implications for both future scientific investigations and the autism community.

Probing the effect of the expected-speed violation illusion

Motion perception is complex for the brain to process, involving interacting computations of distance, time, and speed. These computations can be biased by the context and the features of the perceived moving object, giving rise to several types of motion illusions. Recent research has shown that, in addition to object features and context, lifelong priors can bias attributes of perception. In the present work, we investigated if such long acquired expectations can bias speed perception. Using a two-interval forced-choice (2-IFC) task, we asked 160 participants in different experiments to judge which of two vehicles, one archetypically fast (e.g. a motorbike), and one comparatively slower (e.g. a bike), was faster. By varying the objective speeds of the two-vehicle types, and measuring the participants' point of subjective equality, we observed a consistent bias in participants' speed perception. Counterintuitively, in the first three experiments the speed of an archetypically slow vehicle had to be decreased relative to that of an archetypically fast vehicle, for the two to be judged as the same. Similarly, in the next three experiments, an archetypically fast vehicle's speed had to be increased relative to an archetypically slow vehicle's speed, for the two to be perceived as equal. Four additional control experiments replicated our results. We define this newly found bias as the expected-speed violation illusion (ESVI). We believe the ESVI as conceptually very similar to the size-weight illusion, and discuss it within the Bayesian framework of human perception.

The size-weight illusion comes along with improved weight discrimination

When people judge the weight of two objects of equal mass but different size, they perceive the smaller one as being heavier. Up to date, there is no consensus about the mechanisms which give rise to this size-weight illusion. We recently suggested a model that describes heaviness perception as a weighted average of two sensory heaviness estimates with correlated noise: one estimate derived from mass, the other one derived from density. The density estimate is first derived from mass and size, but at the final perceptual level, perceived heaviness is biased by an object's density, not by its size. Here, we tested the models' prediction that weight discrimination of equal-size objects is better in lifting conditions which are prone to the size-weight illusion as compared to conditions lacking (the essentially uninformative) size information. This is predicted because in these objects density covaries with mass, and according to the model density serves as an additional sensory cue. Participants performed a two-interval forced-choice weight discrimination task. We manipulated the quality of either haptic (Experiment 1) or visual (Experiment 2) size information and measured just-noticeable differences (JNDs). Both for the haptic and the visual illusion, JNDs were lower in lifting conditions in which size information was available. Thus, when heaviness perception can be influenced by an object's density, it is more reliable. This discrimination benefit under conditions that provide the additional information that objects are of equal size is further support for the role of density and the integration of sensory estimates in the size-weight illusion.

Can resources save rationality? "Anti-Bayesian" updating in cognition and perception

Resource rationality may explain suboptimal patterns of reasoning; but what of "anti-Bayesian" effects where the mind updates in a direction opposite the one it should? We present two phenomena - belief polarization and the size-weight illusion - that are not obviously explained by performance- or resource-based constraints, nor by the authors' brief discussion of reference repulsion. Can resource rationality accommodate them?

Dynamic size-weight changes after object lifting reduce the size-weight illusion

In the size-weight illusion, the smaller object from two equally weighted objects is typically judged as being heavier. One explanation is that the mismatch between the weight expectation based on object size and actual sensory feedback influences heaviness perception. In most studies, the size of an object is perceived before its weight. We investigated whether size changes would influence weight judgement if both would be perceived simultaneously. We used virtual reality to change the size and weight of an object after lifting and asked participants to judge whether the object became lighter or heavier. We found that simultaneous size-weight changes greatly reduced the size-weight illusion to perceptual biases below discrimination thresholds. In a control experiment in which we used a standard size-weight illusion protocol with sequential lifts of small and large objects in the same virtual reality setup, we found a larger, typical perceptual bias. These results show that the size-weight illusion is smaller when size and weight information is perceived simultaneously. This provides support for the prediction mismatch theory explaining the size-weight illusion. The comparison between perceived and expected weight during the lifting phase could be a critical brain mechanism for mediating the size-weight illusion.

Thermal-Tactile Integration in Object Temperature Perception

The brain consistently faces a challenge of whether and how to combine the available information sources to estimate the properties of an object explored by hand. While object perception is an inference process involving multisensory inputs, thermal referral (TR) is an illusion demonstrating how the interaction between thermal and tactile systems can lead to deviations from physical reality-when observers touch three stimulators simultaneously with the middle three fingers of one hand but only the outer two stimulators are heated (or cooled), thermal uniformity is perceived across three fingers. Here, we used TR of warmth to examine the thermal-tactile interaction in object temperature perception. We show that TR is consistent with precision-weighted averaging of thermal sensation across tactile locations. Furthermore, we show that prolonged contact with TR stimulation results in adaptation to the local variations of veridical temperatures instead of the thermal uniformity perceived across three fingers. Our results illuminate the flexibility of processing that underlies thermal-tactile interactions and serve as a basis for thermal display design.

Paradoxical impact of memory on color appearance of faces

What is color vision for? Here we compared the extent to which memory modulates color appearance of objects and faces. Participants matched the colors of stimuli illuminated by low-pressure sodium light, which renders scenes monochromatic. Matches for fruit were not predicted by stimulus identity. In contrast, matches for faces were predictable, but surprising: faces appeared green and looked sick. The paradoxical face-color percept could be explained by a Bayesian observer model constrained by efficient coding. The color-matching data suggest that the face-color prior is established by visual signals arising from the recently evolved L-M cone system, not the older S-cone channel. Taken together, the results show that when retinal mechanisms of color vision are impaired, the impact of memory on color perception is greatest for face color, supporting the idea that trichromatic color plays an important role in social communication.

What is the function of color vision? Here, the authors show that when retinal mechanisms of color are impaired, memory has a paradoxical impact on color appearance that is selective for faces, providing evidence that color contributes to face encoding and social communication.

The development of the size-weight illusion in children coincides with the development of nonverbal cognition rather than motor skills

We examined how the strength of the size-weight illusion develops with age in typically developing children. To this end, we recruited children aged 5-12 years and quantified the degree to which they experienced the illusion. We hypothesized that the strength of the illusion would increase with age. The results supported this hypothesis. We also measured abilities in manual dexterity, receptive language, and abstract reasoning to determine whether changes in illusion strength were associated with these factors. Manual dexterity and receptive language did not correlate with illusion strength. Conversely, illusion strength and abstract reasoning were tightly coupled with each other. Multiple regression further revealed that age, manual dexterity, and receptive language did not contribute more to the variance in illusion strength beyond children's abilities in abstract reasoning. Taken together, the effects of age on the size-weight illusion appear to be explained by the development of nonverbal cognition. These findings not only inform the literature on child development but also have implications for theoretical explanations on the size-weight illusion. We suggest that the illusion has a strong acquired component to it and that it is strengthened by children's reasoning skills and perhaps an understanding of the world that develops with age.

Move on up: Fingertip forces and felt heaviness are modulated by the goal of the lift

When we interact with objects, we usually do so for a purpose. It is well known that the specific goal of an action can have a substantial effect on initial reach kinematics. No research, however, has examined the effect that the goal of a lift can have on the fingertip forces and perception of object weight when picking up an object to move it. Here, we report a study in which participants were asked to move objects laterally to a higher platform, to a lower platform, or to a platform of the same height. The objects were rated, on average, as feeling heavier after they were moved to a higher platform than after they were moved to a lower platform or to a platform of the same height. Furthermore, participants gripped and lifted with more force, and used higher rates of force, when moving objects to a higher platform compared with moving it to a platform of the same height. These findings suggest that the goal of movement in the context of object interaction may affect how heavy an object feels and the way in which it is lifted.

Examining the size-weight illusion with visuo-haptic conflict in immersive virtual reality

When we experience our environment, we do so by combining sensory inputs with expectations derived from our prior knowledge, which can lead to surprising perceptual effects such as small objects feeling heavier than equally weighted large objects (the size-weight illusion (SWI)). Interestingly, there is evidence that the way in which the volume of an object is experienced can affect the strength of the illusion, with a SWI induced by exclusively haptic volume cues feeling stronger than a SWI induced with only visual volume cues. Furthermore, visual cues appear to add nothing over and above haptic size cues in terms of the strength of the induced weight illusion-findings which are difficult to reconcile with work using cue-conflict paradigms where visual cues usually dominate haptic cues. Here, virtual reality was used to place these senses in conflict with one another. Participants (N = 22) judged the heaviness of identically weighted cylinders across three conditions: (1) objects appeared different sizes but were physically the same size, (2) objects were physically different sizes but appeared to be the same size, or (3) objects which looked and felt different sizes from one another. Consistent with prior work, haptic size cues induced a larger SWI than that induced by visual size differences. In contrast to prior work, however, congruent vision and haptic size cues yielded a larger still SWI. These findings not only add to our understanding of how different modalities combine to influence our hedonic perception but also showcase how virtual reality can develop novel cue-conflict paradigms.

When Does One Decide How Heavy an Object Feels While Picking It Up?

When lifting an object, it takes time to decide how heavy it is. How does this weight judgment develop? To answer this question, we examined when visual size information has to be present to induce a size-weight illusion. We found that a short glimpse (200 ms) of size information is sufficient to induce a size-weight illusion. The illusion occurred not only when the glimpse was before the onset of lifting but also when the object's weight could already be felt. Only glimpses more than 300 ms after the onset of lifting did not influence the judged weight. This suggests that it takes about 300 ms to reach a perceptual decision about the weight.

The influence of size in weight illusions is unique relative to other object features

Research into weight illusions has provided valuable insight into the functioning of the human perceptual system. Associations between the weight of an object and its other features, such as its size, material, density, conceptual information, or identity, influence our expectations and perceptions of weight. Earlier accounts of weight illusions underscored the importance of previous interactions with objects in the formation of these associations. In this review, we propose a theory that the influence of size on weight perception could be driven by innate and phylogenetically older mechanisms, and that it is therefore more deep-seated than the effects of other features that influence our perception of an object's weight. To do so, we first consider the different associations that exist between the weight of an object and its other features and discuss how different object features influence weight perception in different weight illusions. After this, we consider the cognitive, neurological, and developmental evidence, highlighting the uniqueness of size-weight associations and how they might be reinforced rather than driven by experience alone. In the process, we propose a novel neuroanatomical account of how size might influence weight perception differently than other object features do.

The material-weight illusion disappears or inverts in objects made of two materials

The material-weight illusion (MWI) occurs when an object that looks heavy (e.g., stone) and one that looks light (e.g., Styrofoam) have the same mass. When such stimuli are lifted, the heavier-looking object feels lighter than the lighter-looking object, presumably because well-learned priors about the density of different materials are violated. We examined whether a similar illusion occurs when a certain weight distribution is expected (such as the metal end of a hammer being heavier), but weight is uniformly distributed. In experiment 1, participants lifted bipartite objects that appeared to be made of two materials (combinations of stone, Styrofoam, and wood) but were manipulated to have a uniform weight distribution. Most participants experienced an inverted MWI (i.e., the heavier-looking side felt heavier), suggesting an integration of incoming sensory information with density priors. However, a replication of the classic MWI was found when the objects appeared to be uniformly made of just one of the materials (experiment 2). Both illusions seemed to be independent of the forces used when the objects were lifted. When lifting bipartite objects but asked to judge the weight of the whole object, participants experienced no illusion (experiment 3). In experiment 4, we investigated weight perception in objects with a nonuniform weight distribution and again found evidence for an integration of prior and sensory information. Taken together, our seemingly contradictory results challenge most theories about the MWI. However, Bayesian integration of competing density priors with the likelihood of incoming sensory information may explain the opposing illusions.

NEW & NOTEWORTHY We report a novel weight illusion that contradicts all current explanations of the material-weight illusion: When lifting an object composed of two materials, the heavier-looking side feels heavier, even when the true weight distribution is uniform. The opposite (classic) illusion is found when the same materials are lifted in two separate objects. Identifying the common mechanism underlying both illusions will have implications for perception more generally. A potential candidate is Bayesian inference with competing priors.

The material-weight illusion is a Bayes-optimal percept under competing density priors

The material-weight illusion (MWI) is one example in a class of weight perception illusions that seem to defy principled explanation. In this illusion, when an observer lifts two objects of the same size and mass, but that appear to be made of different materials, the denser-looking (e.g., metal-look) object is perceived as lighter than the less-dense-looking (e.g., polystyrene-look) object. Like the size-weight illusion (SWI), this perceptual illusion occurs in the opposite direction of predictions from an optimal Bayesian inference process, which predicts that the denser-looking object should be perceived as heavier, not lighter. The presence of this class of illusions challenges the often-tacit assumption that Bayesian inference holds universal explanatory power to describe human perception across (nearly) all domains: If an entire class of perceptual illusions cannot be captured by the Bayesian framework, how could it be argued that human perception truly follows optimal inference? However, we recently showed that the SWI can be explained by an optimal hierarchical Bayesian causal inference process (Peters, Ma & Shams, 2016) in which the observer uses haptic information to arbitrate among competing hypotheses about objects’ possible density relationship. Here we extend the model to demonstrate that it can readily explain the MWI as well. That hierarchical Bayesian inference can explain both illusions strongly suggests that even puzzling percepts arise from optimal inference processes.

Force feedback delay affects perception of stiffness but not action, and the effect depends on the hand used but not on the handedness

Interaction with an object often requires the estimation of its mechanical properties. We examined whether the hand that is used to interact with the object and their handedness affected people's estimation of these properties using stiffness estimation as a test case. We recorded participants' responses on a stiffness discrimination of a virtual elastic force field and the grip force applied on the robotic device during the interaction. In half of the trials, the robotic device delayed the participants' force feedback. Consistent with previous studies, delayed force feedback biased the perceived stiffness of the force field. Interestingly, in both left-handed and right-handed participants, for the delayed force field, there was even less perceived stiffness when participants used their left hand than their right hand. This result supports the idea that haptic processing is affected by laterality in the brain, not by handedness. Consistent with previous studies, participants adjusted their applied grip force according to the correct size and timing of the load force regardless of the hand that was used, the handedness, or the delay. This suggests that in all of these conditions, participants were able to form an accurate internal representation of the anticipated trajectory of the load force (size and timing) and that this representation was used for accurate control of grip force independently of the perceptual bias. Thus these results provide additional evidence for the dissociation between action and perception in the processing of delayed information. NEW & NOTEWORTHY Introducing delay to force feedback during interaction with an elastic force field biases the perceived stiffness of the force field. We show that this bias depends on the hand that was used for probing but not on handedness. At the same time, both left-handed and right-handed participants adjusted their applied grip force while using either their left or right hands in anticipation of the correct magnitude and timing despite the delay in load force.

Suboptimality in perceptual decision making

Human perceptual decisions are often described as optimal. Critics of this view have argued that claims of optimality are overly flexible and lack explanatory power. Meanwhile, advocates for optimality have countered that such criticisms single out a few selected papers. To elucidate the issue of optimality in perceptual decision making, we review the extensive literature on suboptimal performance in perceptual tasks. We discuss eight different classes of suboptimal perceptual decisions, including improper placement, maintenance, and adjustment of perceptual criteria; inadequate tradeoff between speed and accuracy; inappropriate confidence ratings; misweightings in cue combination; and findings related to various perceptual illusions and biases. In addition, we discuss conceptual shortcomings of a focus on optimality, such as definitional difficulties and the limited value of optimality claims in and of themselves. We therefore advocate that the field drop its emphasis on whether observed behavior is optimal and instead concentrate on building and testing detailed observer models that explain behavior across a wide range of tasks. To facilitate this transition, we compile the proposed hypotheses regarding the origins of suboptimal perceptual decisions reviewed here. We argue that verifying, rejecting, and expanding these explanations for suboptimal behavior - rather than assessing optimality per se - should be among the major goals of the science of perceptual decision making.

A mass-density model can account for the size-weight illusion

When judging the heaviness of two objects with equal mass, people perceive the smaller and denser of the two as being heavier. Despite the large number of theories, covering bottom-up and top-down approaches, none of them can fully account for all aspects of this size-weight illusion and thus for human heaviness perception. Here we propose a new maximum-likelihood estimation model which describes the illusion as the weighted average of two heaviness estimates with correlated noise: One estimate derived from the object's mass, and the other from the object's density, with estimates' weights based on their relative reliabilities. While information about mass can directly be perceived, information about density will in some cases first have to be derived from mass and volume. However, according to our model at the crucial perceptual level, heaviness judgments will be biased by the objects' density, not by its size. In two magnitude estimation experiments, we tested model predictions for the visual and the haptic size-weight illusion. Participants lifted objects which varied in mass and density. We additionally varied the reliability of the density estimate by varying the quality of either visual (Experiment 1) or haptic (Experiment 2) volume information. As predicted, with increasing quality of volume information, heaviness judgments were increasingly biased towards the object's density: Objects of the same density were perceived as more similar and big objects were perceived as increasingly lighter than small (denser) objects of the same mass. This perceived difference increased with an increasing difference in density. In an additional two-alternative forced choice heaviness experiment, we replicated that the illusion strength increased with the quality of volume information (Experiment 3). Overall, the results highly corroborate our model, which seems promising as a starting point for a unifying framework for the size-weight illusion and human heaviness perception.

Contribution of surface material and size to the expected versus the perceived weight of objects

Because the perceived weight of objects may be affected by various nonweight properties, such as their size and the density of their surface material, relative weight is sometimes misperceived (the size-weight illusion and the material-weight illusion, respectively). A widely accepted explanation for weight illusions is provided by the so-called expectation model, according to which the perceived weight stems from the contrast between the actual and expected weights. In the present study, we varied both the surface material and the size of stimuli, while keeping constant their physical weights. In Experiment 1, the participants lifted the stimuli by grasping them on opposite sides, whereas in Experiment 2 they lifted them by using a string that was attached to their top surface. We used a variant of the random conjoint measurement paradigm to obtain subjective interval scales of the contributions of surface material and size to the expected and the perceived weight of the stimuli. Inconsistently with the predictions from the expectation model, we found, in both experiments, that the surface material contributed more than the size to the expected weight, whereas the size contributed more than the surface material to the perceived weight. The results support the hypothesis that perceived weight may depend on implicit, rather than explicit, weight expectations.

The absence or temporal offset of visual feedback does not influence adaptation to novel movement dynamics

Delays in transmitting and processing sensory information require correctly associating delayed feedback to issued motor commands for accurate error compensation. The flexibility of this alignment between motor signals and feedback has been demonstrated for movement recalibration to visual manipulations, but the alignment dependence for adapting movement dynamics is largely unknown. Here we examined the effect of visual feedback manipulations on force-field adaptation. Three subject groups used a manipulandum while experiencing a lag in the corresponding cursor motion (0, 75, or 150 ms). When the offset was applied at the start of the session (continuous condition), adaptation was not significantly different between groups. However, these similarities may be due to acclimation to the offset before motor adaptation. We tested additional subjects who experienced the same delays concurrent with the introduction of the perturbation (abrupt condition). In this case adaptation was statistically indistinguishable from the continuous condition, indicating that acclimation to feedback delay was not a factor. In addition, end-point errors were not significantly different across the delay or onset conditions, but end-point correction (e.g., deceleration duration) was influenced by the temporal offset. As an additional control, we tested a group of subjects who performed without visual feedback and found comparable movement adaptation results. These results suggest that visual feedback manipulation (absence or temporal misalignment) does not affect adaptation to novel dynamics, independent of both acclimation and perceptual awareness. These findings could have implications for modeling how the motor system adjusts to errors despite concurrent delays in sensory feedback information.NEW & NOTEWORTHY A temporal offset between movement and distorted visual feedback (e.g., visuomotor rotation) influences the subsequent motor recalibration, but the effects of this offset for altered movement dynamics are largely unknown. Here we examined the influence of 1) delayed and 2) removed visual feedback on the adaptation to novel movement dynamics. These results contribute to understanding of the control strategies that compensate for movement errors when there is a temporal separation between motion state and sensory information.

Cross-Sensory Correspondences: Heaviness is Dark and Low-Pitched

Everyday language reveals how stimuli encoded in one sensory feature domain can possess qualities normally associated with a different domain (e.g., higher pitch sounds are bright, light in weight, sharp, and thin). Such cross-sensory associations appear to reflect crosstalk among aligned (corresponding) feature dimensions, including brightness, heaviness, and sharpness. Evidence for heaviness being one such dimension is very limited, with heaviness appearing primarily as a verbal associate of other feature contrasts (e.g., darker objects and lower pitch sounds are heavier than their opposites). Given the presumed bidirectionality of the crosstalk between corresponding dimensions, heaviness should itself induce the cross-sensory associations observed elsewhere, including with brightness and pitch. Taking care to dissociate effects arising from the size and mass of an object, this is confirmed. When hidden objects varying independently in size and mass are lifted, objects that feel heavier are judged to be darker and to make lower pitch sounds than objects feeling less heavy. These judgements track the changes in perceived heaviness induced by the size-weight illusion. The potential involvement of language, natural scene statistics, and Bayesian processes in correspondences, and the effects they induce, is considered.

The Influence of Prior Knowledge on Perception and Action: Relationships to Autistic Traits

Autism is characterised by a range of perceptual and sensorimotor deficits, which might be related to abnormalities in how autistic individuals use prior knowledge. We investigated this proposition in a large non-clinical population in the context of the size-weight illusion, where individual’s expectations about object weight influence their perceptions of heaviness and fingertip forces. Although there was no relationship between autistic traits and the magnitude of the illusion, we observed an inverse relationship between AQ scores and how expectations influenced initial fingertip force application. These findings provide a novel dissociation between how perceptual and sensorimotor processes are related to autistic traits, and suggest that, autistic traits might explain some of the variance surrounding how individuals grip and lift objects.

Does the sensorimotor system minimize prediction error or select the most likely prediction during object lifting?

The human sensorimotor system is routinely capable of making accurate predictions about an object's weight, which allows for energetically efficient lifts and prevents objects from being dropped. Often, however, poor predictions arise when the weight of an object can vary and sensory cues about object weight are sparse (e.g., picking up an opaque water bottle). The question arises, what strategies does the sensorimotor system use to make weight predictions when one is dealing with an object whose weight may vary? For example, does the sensorimotor system use a strategy that minimizes prediction error (minimal squared error) or one that selects the weight that is most likely to be correct (maximum a posteriori)? In this study we dissociated the predictions of these two strategies by having participants lift an object whose weight varied according to a skewed probability distribution. We found, using a small range of weight uncertainty, that four indexes of sensorimotor prediction (grip force rate, grip force, load force rate, and load force) were consistent with a feedforward strategy that minimizes the square of prediction errors. These findings match research in the visuomotor system, suggesting parallels in underlying processes. We interpret our findings within a Bayesian framework and discuss the potential benefits of using a minimal squared error strategy.

NEW & NOTEWORTHY

Using a novel experimental model of object lifting, we tested whether the sensorimotor system models the weight of objects by minimizing lifting errors or by selecting the statistically most likely weight. We found that the sensorimotor system minimizes the square of prediction errors for object lifting. This parallels the results of studies that investigated visually guided reaching, suggesting an overlap in the underlying mechanisms between tasks that involve different sensory systems.

Inflections of the Bayesian Paradigm in Perceptual Psychology

Bayesian modeling has gained a conspicuous position in contemporary perceptual psychology. It can be examined from two viewpoints: a formal one, concerning the logical attributes of and the algebraic operations on the components of the models, and a substantive one, concerning the empirical meaning of those components. We maintain that, while there is homogeneity between Bayesian models of visual perception in their formal setup, remarkable differences can be found in their substantive aspect, that is, how the question "Where do probabilities come from?" is answered when designing the models. In particular, we focus on an inflection that we call "congenial" because it consistently embodies the inversion idea of the Bayes' rule in terms of optical inversion and highlight delicate issues that face this inflection for a consistent realization of the scientific program it represents. We also suggest ideas concerning the organization of the Bayesian area within perceptual psychology, which appears variegated, with the congenial inflection in a central position, and a fringe of disputable classification along the border.

Representing multiple object weights: competing priors and sensorimotor memories

When lifting an object, individuals scale lifting forces based on long-term priors relating external object properties (such as material and size) to object weight. When experiencing objects that are poorly predicted by priors, people rapidly form and update sensorimotor memories that can be used to predict an object's atypical size-weight relation in support of predictively scaling lift forces. With extensive experience in lifting such objects, long-term priors, assessed with weight judgments, are gradually updated. The aim of the present study was to understand the formation and updating of these memory processes. Participants lifted, over multiple days, a set of black cubes with a normal size-weight mapping and green cubes with an inverse size-weight mapping. Sensorimotor memory was assessed with lifting forces, and priors associated with the black and green cubes were assessed with the size-weight illusion (SWI). Interference was observed in terms of adaptation of the SWI, indicating that priors were not independently adjusted. Half of the participants rapidly learned to scale lift forces appropriately, whereas reduced learning was observed in the others, suggesting that individual differences may be affecting sensorimotor memory abilities. A follow-up experiment showed that lifting forces are not accurately scaled to objects when concurrently performing a visuomotor association task, suggesting that sensorimotor memory formation involves cognitive resources to instantiate the mapping between object identity and weight, potentially explaining the results of experiment 1 These results provide novel insight into the formation and updating of sensorimotor memories and provide support for the independent adjustment of sensorimotor memory and priors.

The Size-Weight Illusion is not anti-Bayesian after all: a unifying Bayesian account

When we lift two differently-sized but equally-weighted objects, we expect the larger to be heavier, but the smaller feels heavier. However, traditional Bayesian approaches with “larger is heavier” priors predict the smaller object should feel lighter; this Size-Weight Illusion (SWI) has thus been labeled “anti-Bayesian” and has stymied psychologists for generations. We propose that previous Bayesian approaches neglect the brain’s inference process about density. In our Bayesian model, objects’ perceived heaviness relationship is based on both their size and inferred density relationship: observers evaluate competing, categorical hypotheses about objects’ relative densities, the inference about which is then used to produce the final estimate of weight. The model can qualitatively and quantitatively reproduce the SWI and explain other researchers’ findings, and also makes a novel prediction, which we confirmed. This same computational mechanism accounts for other multisensory phenomena and illusions; that the SWI follows the same process suggests that competitive-prior Bayesian inference can explain human perception across many domains.

The role of expectancies in the size-weight illusion: a review of theoretical and empirical arguments and a new explanation

The size-weight illusion (SWI) refers to the phenomenon that objects that are objectively equal in weight but different in size or volume are perceived to differ in weight, such that smaller objects feel heavier than larger ones. This article reviews studies trying to support three different viewpoints with respect to the role of expectancies in causing the SWI. The first viewpoint argues for a crucial role; the second admits a role, yet without seeing consequences for sensorimotor processes; and the third denies any causal role for expectancies at all. A new explanation of the SWI is proposed that can integrate the different arguments. A distinctive feature of the new explanation is that it recognizes the causal influence of expectancies, yet combines this with certain reactive and direct behavioral consequences of perceiving size differences that are independent of experience-based expectancies, and that normally result in the adaptive application of forces to lift or handle differently sized objects. The new account explains why the illusion is associated with the repeated generation of inappropriate lifting forces (which can, however, be modified through extensive training), as well as why it depends on continuous visual exposure to size cues, appears at an early age, and is cognitively impenetrable.

A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts

Bayesian observer models provide a principled account of the fact that our perception of the world rarely matches physical reality. The standard explanation is that our percepts are biased toward our prior beliefs. However, reported psychophysical data suggest that this view may be simplistic. We propose a new model formulation based on efficient coding that is fully specified for any given natural stimulus distribution. The model makes two new and seemingly anti-Bayesian predictions. First, it predicts that perception is often biased away from an observer's prior beliefs. Second, it predicts that stimulus uncertainty differentially affects perceptual bias depending on whether the uncertainty is induced by internal or external noise. We found that both model predictions match reported perceptual biases in perceived visual orientation and spatial frequency, and were able to explain data that have not been explained before. The model is general and should prove applicable to other perceptual variables and tasks.

Smaller = denser, and the brain knows it: natural statistics of object density shape weight expectations

If one nondescript object's volume is twice that of another, is it necessarily twice as heavy? As larger objects are typically heavier than smaller ones, one might assume humans use such heuristics in preparing to lift novel objects if other informative cues (e.g., material, previous lifts) are unavailable. However, it is also known that humans are sensitive to statistical properties of our environments, and that such sensitivity can bias perception. Here we asked whether statistical regularities in properties of liftable, everyday objects would bias human observers' predictions about objects' weight relationships. We developed state-of-the-art computer vision techniques to precisely measure the volume of everyday objects, and also measured their weight. We discovered that for liftable man-made objects, "twice as large" doesn't mean "twice as heavy": Smaller objects are typically denser, following a power function of volume. Interestingly, this "smaller is denser" relationship does not hold for natural or unliftable objects, suggesting some ideal density range for objects designed to be lifted. We then asked human observers to predict weight relationships between novel objects without lifting them; crucially, these weight predictions quantitatively match typical weight relationships shown by similarly-sized objects in everyday environments. These results indicate that the human brain represents the statistics of everyday objects and that this representation can be quantitatively abstracted and applied to novel objects. Finally, that the brain possesses and can use precise knowledge of the nonlinear association between size and weight carries important implications for implementation of forward models of motor control in artificial systems.

Combining colour and temperature: A blue object is more likely to be judged as warm than a red object

It is commonly believed that reddish colour induces warm feelings while bluish colour induces cold feelings. We, however, demonstrate an opposite effect when the temperature information is acquired by direct touch. Experiment 1 found that a red object, relative to a blue object, raises the lowest temperature required for an object to feel warm, indicating that a blue object is more likely to be judged as warm than a red object of the same physical temperature. Experiment 2 showed that hand colour also affects temperature judgment, with the direction of the effect opposite to object colours. This study provides the first demonstration that colour can modulate temperature judgments when the temperature information is acquired by direct touch. The effects apparently oppose the common conception of red-hot/blue-cold association. We interpret this phenomenon in terms of “Anti-Bayesian” integration, which suggests that the brain integrates direct temperature input with prior expectations about temperature relationship between object and hand in a way that emphasizes the contrast between the two.

Properties of the size-weight illusion as shown by lines of subjective equality

We studied the size-weight illusion through comparative judgments. The experiment had two direct aims: to verify whether the relative contribution of size to apparent heaviness can differ for different stimulus sets, and to verify whether that contribution can differ for different methods of comparing two objects (consecutive vs. simultaneous weighing). Thirty university students participated. Results show that the relative contribution of size depends on stimulus set, but is independent of the method used for comparing objects. The first finding implies that a linear model cannot describe the integration of size and weight information in the illusion; the second finding is evidence for the low-level character of the integration process.

Getting a grip on heaviness perception: a review of weight illusions and their probable causes

Weight illusions--where one object feels heavier than an identically weighted counterpart--have been the focus of many recent scientific investigations. The most famous of these illusions is the 'size-weight illusion', where a small object feels heavier than an identically weighted, but otherwise similar-looking, larger object. There are, however, a variety of similar illusions which can be induced by varying other stimulus properties, such as surface material, temperature, colour, and even shape. Despite well over 100 years of research, there is little consensus about the mechanisms underpinning these illusions. In this review, I will first provide an overview of the weight illusions that have been described. I will then outline the dominant theories that have emerged over the past decade for why we consistently misperceive the weights of objects which vary in size, with a particular focus on the role of lifters' expectations of heaviness. Finally, I will discuss the magnitude of the various weight illusions and suggest how this largely overlooked facet of the topic might resolve some of the debates surrounding the cause of these misperceptions of heaviness.

Occam's Razor in sensorimotor learning

A large number of recent studies suggest that the sensorimotor system uses probabilistic models to predict its environment and makes inferences about unobserved variables in line with Bayesian statistics. One of the important features of Bayesian statistics is Occam's Razor--an inbuilt preference for simpler models when comparing competing models that explain some observed data equally well. Here, we test directly for Occam's Razor in sensorimotor control. We designed a sensorimotor task in which participants had to draw lines through clouds of noisy samples of an unobserved curve generated by one of two possible probabilistic models-a simple model with a large length scale, leading to smooth curves, and a complex model with a short length scale, leading to more wiggly curves. In training trials, participants were informed about the model that generated the stimulus so that they could learn the statistics of each model. In probe trials, participants were then exposed to ambiguous stimuli. In probe trials where the ambiguous stimulus could be fitted equally well by both models, we found that participants showed a clear preference for the simpler model. Moreover, we found that participants' choice behaviour was quantitatively consistent with Bayesian Occam's Razor. We also show that participants' drawn trajectories were similar to samples from the Bayesian predictive distribution over trajectories and significantly different from two non-probabilistic heuristics. In two control experiments, we show that the preference of the simpler model cannot be simply explained by a difference in physical effort or by a preference for curve smoothness. Our results suggest that Occam's Razor is a general behavioural principle already present during sensorimotor processing.

Are we predictive engines? Perils, prospects, and the puzzle of the porous perceiver

The target article sketched and explored a mechanism (action-oriented predictive processing) most plausibly associated with core forms of cortical processing. In assessing the attractions and pitfalls of the proposal we should keep that element distinct from larger, though interlocking, issues concerning the nature of adaptive organization in general.

When the predictive brain gets it really wrong

Clark examines the notion of the "predictive brain" as a unifying model for cognitive neuroscience, from the level of basic neural processes to sensorimotor control. Although we are in general agreement with this notion, we feel that there are many details that still need to be fleshed out from the standpoint of perception and action.

The effect of temporal perception on weight perception

A successful catch of a falling ball requires an accurate estimation of the timing for when the ball hits the hand. In a previous experiment in which participants performed ball-catching task in virtual reality environment, we accidentally found that the weight of a falling ball was perceived differently when the timing of ball load force to the hand was shifted from the timing expected from visual information. Although it is well known that spatial information of an object, such as size, can easily deceive our perception of its heaviness, the relationship between temporal information and perceived heaviness is still not clear. In this study, we investigated the effect of temporal factors on weight perception. We conducted ball-catching experiments in a virtual environment where the timing of load force exertion was shifted away from the visual contact timing (i.e., time when the ball hit the hand in the display). We found that the ball was perceived heavier when force was applied earlier than visual contact and lighter when force was applied after visual contact. We also conducted additional experiments in which participants were conditioned to one of two constant time offsets prior to testing weight perception. After performing ball-catching trials with 60 ms advanced or delayed load force exertion, participants’ subjective judgment on the simultaneity of visual contact and force exertion changed, reflecting a shift in perception of time offset. In addition, timing of catching motion initiation relative to visual contact changed, reflecting a shift in estimation of force timing. We also found that participants began to perceive the ball as lighter after conditioning to 60 ms advanced offset and heavier after the 60 ms delayed offset. These results suggest that perceived heaviness depends not on the actual time offset between force exertion and visual contact but on the subjectively perceived time offset between them and/or estimation error in force timing.