Spatial stimulus-response compatibility and affordance effects are not ruled by the same mechanisms

A Trained Humanoid Robot can Perform Human-Like Crossmodal Social Attention and Conflict Resolution

To enhance human-robot social interaction, it is essential for robots to process multiple social cues in a complex real-world environment. However, incongruency of input information across modalities is inevitable and could be challenging for robots to process. To tackle this challenge, our study adopted the neurorobotic paradigm of crossmodal conflict resolution to make a robot express human-like social attention. A behavioural experiment was conducted on 37 participants for the human study. We designed a round-table meeting scenario with three animated avatars to improve ecological validity. Each avatar wore a medical mask to obscure the facial cues of the nose, mouth, and jaw. The central avatar shifted its eye gaze while the peripheral avatars generated sound. Gaze direction and sound locations were either spatially congruent or incongruent. We observed that the central avatar's dynamic gaze could trigger crossmodal social attention responses. In particular, human performance was better under the congruent audio-visual condition than the incongruent condition. Our saliency prediction model was trained to detect social cues, predict audio-visual saliency, and attend selectively for the robot study. After mounting the trained model on the iCub, the robot was exposed to laboratory conditions similar to the human experiment. While the human performance was overall superior, our trained model demonstrated that it could replicate attention responses similar to humans.

Seeing through the cat's eyes: evidence of a spontaneous perspective taking process using a non-human avatar

In many daily face-to-face interactions, people are able to take the perspective of others, for example, coding right and left based on point-of-view of others. In the present study, we investigated whether observers are able to take the perspective of a non-human figure such as a cat, observing the same effects obtained with human or robot avatars. In both experiments, we used a centrally presented stimulus (i.e. a cat), with its tail lateralized to the left or to the right. Participants had to respond to the side of the tail with a lateralized keypress. In Experiment 1 (spatial perspective taking task), participants were required to explicitly adopt the cat's perspective to respond, whereas in Experiment 2 (SR compatibility task), this was not explicitly required. In both experiments, faster RTs are obtained when the cat is presented back, with a greater difference between front and back views when the tail is on the right; furthermore, there is no temporal modulation of the back-front effect. These common results between the two experiments are interpreted on the basis of the spatial perspective taking processes, elicited voluntarily (Experiment 1) or spontaneously (Experiment 2).

Practice and transfer with mappings of spoon tip and handle to keypress responses

When orientation of a horizontal spoon image varies to the left or right, instructions can map left and right keypresses to the tip or handle location. We conducted Experiment 1 to determine whether practice with an incompatible mapping of the salient tip transfers to a test session in which the relevant part and/or mapping are changed. Participants performed 80 practice trials with tip-incompatible mapping, followed by 80 test trials with tip-compatible, tip-incompatible, handle-compatible, or handle-incompatible mapping. Performance improved across 20-trial blocks in the practice session. In the test session, responses were 65 ms faster with tip-compatible than tip-incompatible mapping but 31 ms faster with handle-incompatible than handle-compatible mapping. This latter result, and verbal reports, indicate that some participants adopted a strategy of responding compatibly to the salient tip even though instructed to respond to the handle. Experiment 2 focused on whether participants with handle-incompatible mapping instructions would adopt the tip-compatible strategy spontaneously or after receiving a hint: 77% of participants reported adopting the tip-compatible strategy in Session 1, showing that prior experience responding to the tip is not necessary and 9% of participants did not report using that strategy in Session 1 but reported changing to it in Session 2 after receiving the hint. Their responses in Session 2 were slower than those who used the strategy throughout, but this difference was minimal in the last two trial blocks. Compatible mapping of the salient spoon tip to keypresses dominated performance over prior practice with incompatible tip mapping and instructions with incompatible handle mapping.

Pixel asymmetry predicts between-object differences in the object-based compatibility effect

When participants make left/right responses to unimanually graspable objects, response times (RTs) are faster when the responding hand is aligned with the viewed object's handle. This object-based compatibility effect (CE) is often attributed to motor activation elicited by the object's afforded grasp. However, some evidence suggests that the object-based CE is an example of spatial CEs, or Simon effects, elicited by the protruding nature of objects' handles. Moreover, recent work shows that the way in which objects are centred on-screen might attenuate or reverse CEs, perhaps due to differences in pixel asymmetry (the proportion of pixels either side of fixation) between centralities. In this study, we tested whether pixel asymmetry also contributes to between-object variation in object-based CEs. In experiment 1 (N = 34), we found that between-object differences in asymmetry predicted object-based CEs, such that objects with a greater proportion of pixels to the handle-congruent side of fixation produced larger CEs. In experiment 2 (N = 35), we presented participants with mug (low asymmetry) and frying pan (high asymmetry) images and found that between-object and within-object (due to stimulus centrality) differences in pixel asymmetry interact to moderate CEs. Base-centred stimuli (centred according to the width of the object's base) produced conventional CEs, whereas object-centred (centred according to the object's total width) stimuli produced negative CEs (NCEs). Furthermore, the effect of centrality was smaller for mugs than pans, indicating an interaction between within-object and between-object differences in pixel asymmetry.

On the Evolution of a Radical Concept: Affordances According to Gibson and Their Subsequent Use and Development

James J. Gibson, the founder of ecological psychology, introduced a radical empiricist approach to perception and action centered on direct perception in naturalistic environments that was counter to popular representational views of his time. This direct perception approach and the associated introduction of the affordance concept have been extremely influential in several fields of study. However, since its inception, the affordance concept has evolved in a manner such that it now deviates significantly from Gibson's original intention. This review follows use of the affordance concept by four sets of influential experimental psychologists: Gibson, Donald Norman, Mike Tucker and Rob Ellis, and Daniel Bub and Michael Masson. Particular attention is paid to the manner in which they applied the concept and the contributions provided by each set of researchers. The primary goal of this review is to determine what cognitive psychologists can take away from developments within the field and what considerations should be taken into account when using the term affordance. Having a more thorough understanding of the factors that led to the concept of affordance and its recent reformulations will better equip cognitive psychologists and, by extension, human factors researchers to further advance the study of perception-action relations.

Sidestepping spatial confounds in object-based correspondence effects: The Bimanual Affordance Task (BMAT)

Tucker and Ellis found that when participants made left/right button-presses to indicate whether objects were upright or inverted, responses were faster when the response hand aligned with the task-irrelevant handle orientation of the object. The effect of handle orientation on response times has been interpreted as evidence that individuals perceive grasp affordances when viewing briefly presented objects, which in turn activate grasp-related motor systems. Although the effect of handle alignment has since been replicated, there remains doubt regarding the extent to which the effect is indeed driven by affordance perception. Objects that feature in affordance-compatibility paradigms are asymmetrical and have laterally protruding handles (e.g., mugs) and thus confound spatial and affordance properties. Research has attempted to disentangle spatial compatibility and affordance effects with varying results. In this study, we present a novel paradigm with which to study affordance perception while sidestepping spatial confounds. We use the Bimanual Affordance Task (BMAT) to test whether object affordances in symmetrical objects facilitate response times. Participants (N = 36) used one of three (left unimanual/right unimanual/bimanual) responses to indicate the colour of presented objects. Objects afforded either a unimanual (e.g., handbag) or a bimanual (e.g., laundry hamper) grasp. Responses were faster when the afforded grasp corresponded with the response type (unimanual vs. bimanual), suggesting that affordance effects exist independent of spatial compatibility.

Effects of broken affordance on visual extinction

Previous studies have shown that visual extinction can be reduced if two objects are positioned to “afford” an action. Here we tested if this affordance effect was disrupted by “breaking” the affordance, i.e., if one of the objects actively used in the action had a broken handle. We assessed the effects of broken affordance on recovery from extinction in eight patients with right hemisphere lesions and left-sided extinction. Patients viewed object pairs that were or were not commonly used together and that were positioned for left- or right-hand actions. In the unrelated pair conditions, either two tools or two objects were presented. In line with previous research (e.g., Riddoch et al., 2006), extinction was reduced when action-related object pairs and when unrelated tool pairs were presented compared to unrelated object pairs. There was no significant difference in recovery rate between action-related (object-tool) and unrelated tool pairs. In addition, performance with action-related objects decreased when the tool appeared on the ipsilesional side compared to when it was on the contralesional side, but only when the tool handle was intact. There were minimal effects of breaking the handle of an object rather than a tool, and there was no effect of breaking the handle on either tools or objects on single item trials. The data suggest that breaking the handle of a tool lessens the degree to which it captures attention, with this attentional capture being strongest when the tool appears on the ipsilesional side. The capture of attention by the ipsilesional item then reduces the chance of detecting the contralesional stimulus. This attentional capture effect is mediated by the affordance to the intact tool.

Influence of short incompatible practice on the Simon effect: transfer along the vertical dimension and across vertical and horizontal dimensions

In spatial compatibility and Simon tasks, the response is faster when stimulus and response locations are on the same side than when they are on opposite sides. It has been shown that a spatial incompatible practice leads to a subsequent modulation of the Simon effect along the horizontal dimension. It has also been reported that this modulation occurs both along and across vertical and horizontal dimensions, but only after intensive incompatible training (600 trials). In this work, we show that this modulatory effect can be obtained with a smaller number of incompatible trials, changing the spatial arrangement of the vertical response keys to obtain a stronger dimensional overlap between the spatial codes of stimuli and response keys. The results of Experiment 1 showed that 80 incompatible vertical trials abolished the Simon effect in the same dimension. Experiment 2 showed that a modulation of the vertical Simon effect could be obtained after 80 horizontal incompatible trials. Experiment 3 explored whether the transfer effect can also occur in a horizontal Simon task after a brief vertical spatial incompatibility task, and results were similar to the previous experiments. In conclusion, we suggest that the spatial arrangement between response key and stimulus locations may be critical to establish the short-term memory links that enable the transfer of learning between brief incompatible practices and the Simon effects, both along the vertical dimension and across vertical and horizontal dimensions.