The cognitive nature of action - functional links between cognitive psychology, movement science, and robotics

Mapping relational links between motor imagery, action observation, action-related language, and action execution

Actions can be physically executed, observed, imagined, or simply thought about. Unifying mental processes, such as simulation, emulation, or predictive processing, are thought to underlie different action types, whether they are mental states, as in the case of motor imagery and action observation, or involve physical execution. While overlapping brain activity is typically observed across different actions which indicates commonalities, research interest is also concerned with investigating the distinct functional components of these action types. Unfortunately, untangling subtleties associated with the neurocognitive bases of different action types is a complex endeavour due to the high dimensional nature of their neural substrate (e.g., any action process is likely to activate multiple brain regions thereby having multiple dimensions to consider when comparing across them). This has impeded progress in action-related theorising and application. The present study addresses this challenge by using the novel approach of multidimensional modeling to reduce the high-dimensional neural substrate of four action-related behaviours (motor imagery, action observation, action-related language, and action execution), find the least number of dimensions that distinguish or relate these action types, and characterise their neurocognitive relational links. Data for the model comprised brain activations for action types from whole-brain analyses reported in 53 published articles. Eighty-two dimensions (i.e., 82 brain regions) for the action types were reduced to a three-dimensional model, that mapped action types in ordination space where the greater the distance between the action types, the more dissimilar they are. A series of one-way ANOVAs and post-hoc comparisons performed on the mean coordinates for each action type in the model showed that across all action types, action execution and concurrent action observation (AO)-motor imagery (MI) were most neurocognitively similar, while action execution and AO were most dissimilar. Most action types were similar on at least one neurocognitive dimension, the exception to this being action-related language. The import of the findings are discussed in terms of future research and implications for application.

Variations of Sensorimotor Representation (Structure): The Functional Interplay between Object Features and Goal-Directed Grasping Actions

This study investigated the structure of sensorimotor representations during goal-directed grasping actions and explored their relationship with object features. Sixteen 3D-printed spheres that varied in size (i.e., a diameter of 20 mm, 40 mm, 60 mm, 80 mm) and weight (i.e., 40 g, 52 g, 76 g, 91 g) were used as experimental stimuli. The Structural Dimensional Analysis of Mental Representation (SDA-M) method was used to assess the sensorimotor representation structure during grasping. Participants were instructed in each trial to weigh, lift, or transport sets of two different spheres and to judge the similarity of the objects’ features, taking into account the executed grasping movement. Each participant performed a total of 240 trials, and object presentation was randomized. The results suggest that the functional interplay between object features and goal-directed actions accounts for the significant variations in the structure of sensorimotor representations after grasping. Specifically, the relevance of the perceived objects’ size and weight is closely interrelated to the grasping task demands and movement dynamics of the executed action. Our results suggest that distinct sensorimotor representations support individual grasping actions according to top-down influences modulated by motor intentions, functional task demands, and task-relevant object features.

Motor Imagery Combined With Physical Training Improves Response Inhibition in the Stop Signal Task

Background

Motor imagery training has a similar effect to that of physical training on motor performance. The objective of this study was to investigate the short-term effectiveness of motor imagery training on response inhibition using the stop signal task (SST).

Methods

Participants were divided into a physical training group (PT, n = 17), a motor imagery training group (MIT, n = 17), and a motor imagery combined with physical training group (MIPT, n = 17). All participants performed 10 SST training sessions over 5 days. Both stop signal reaction time (SSRT) and non-signal reaction time (NSRT) were measured before and after SST training.

Results

There were significant interaction (time × group) and time effects, although the group effect was not statistically significant. Bonferroni post hoc analysis showed that MIPT group revealed a significantly greater change in SSRT than PT and MIT groups, while there was no significant difference between PT and MIT groups. SSRT significantly decreased after training in all groups. In NSRT, there was a significant effect of time, but there was no significant interaction effect (time × group) or group effect.

Conclusion

Response inhibition could be enhanced via training, and it was most effective when motor imagery and physical training were combined. We demonstrate that motor imagery training significantly improves response inhibition and should be accompanied by physical training when performing SST.

The Relationship Between the Perceived Movement Quality and the Kinematic Pattern of Complex Skills in Gymnastics

The study aimed to investigate the relationship between the perceived movement quality of a gymnastics skill and its kinematic pattern, as well as the influence of expertise. Thirty participants with different levels of gymnastics expertise (n = 10 visual experts, n = 10 motor experts and n = 10 novices) were recruited for the study. They were instructed to compare the movement quality of eleven video sequences, showing different handstand - back handspring performances. To extract the kinematics, the performances were digitized. By means of an ongoing cluster analysis, the kinematic pattern as well as the pattern of the perceived movement quality of the skills were determined for each experimental group. The results of the cluster analysis of the different experimental groups were analyzed and compared. Expertise differences were found regarding the pattern of the perceived movement quality. There was a significant correlation between the dendrograms of the visual experts and the motor experts (p = .021), as well as between the dendrograms of the visual experts and the novices (p = .011). There was no significant correlation between the dendrograms of the motor experts and the novices (p = .173). The pattern of the perceived movement quality was not correlated with the holistic kinematic pattern of judged skills (p > .143). These results suggest perceptual and cognitive differences of the participants due to their different previous visual and motor experience.

Empirical relationships between algorithmic SDA-M-based memory assessments and human errors in manual assembly tasks

The majority of manufacturing tasks are still performed by human workers, and this will probably continue to be the case in many industry 4.0 settings that aim at highly customized products and small lot sizes. Technical systems could assist on-the-job training and execution of these manual assembly processes, using augmented reality and other means, by properly treating and supporting workers' cognitive resources. Recent algorithmic advancements automatized the assessment of task-related mental representation structures based on SDA-M, which enables technical systems to anticipate mistakes and provide corresponding user-specific assistance. Two studies have empirically investigated the relations between algorithmic assessments of individual memory structures and the occurrences of human errors in different assembly tasks. Hereby theoretical assumptions of the automatized SDA-M assessment approaches were deliberately violated in realistic ways to evaluate the practical applicability of these approaches. Substantial but imperfect correspondences were found between task-related mental representation structures and actual performances with sensitivity and specificity values ranging from 0.63 to 0.72, accompanied by prediction accuracies that were highly significant above chance level.

The Effects of Insole-Based Visual Feedback on Weight-Bearing in Patients Undergoing Total Hip Replacement

This study aimed to investigate the visual biofeedback effect of a sensorized system for plantar pressure dynamic evaluation of in patients with a total hip replacement. Experimental group followed the rehabilitation training wearing sensorized insoles that provided images on three monitors. The control group followed the verbal instructions of physiotherapists during training. Weight bearing percentage healthy limb (WBPH), weight bearing percentage surgical limb (WBPS), swing healthy limb (SWH) and swing surgical limb (SWS) improved significantly more in the experimental group. The results underline the effectiveness of visual biofeedback based on sensorized system with dynamic evaluation of the plantar pressure.

Cognitive control processes associated with successful gait performance in dual-task walking in healthy young adults

Growing evidence suggests that the reliance on cognitive control processes during normal walking increases as the locomotor task gets more complex and challenging. The aims of the present study were to explore the (negative) effects of smartphone gaming on gait performance in healthy young adults, and to identify cognitive resources that might help to maintain high gait performance during dual-task walking. Gait speed and gait variability during walking at a self-selected comfortable speed were assessed in 40 healthy, young adults, and compared between single-task and dual-task walking (i.e., concurrent smartphone gaming) in undisturbed, simple and more challenging walking environments (i.e., stepping over an obstacle while walking). Based on single-task performance, dual-tasking costs were computed and linked to higher-level cognitive control processes, which were assessed for each individual. Cognitive function testing encompassed tests on the mental representation of the gait, working memory capacity, inhibitory control and cognitive flexibility. Our data revealed that gaming on a smartphone while walking strongly affected participants' gait performance (i.e., up to 26.8% lower gait speed and 60.2% higher gait variability), and decrements in gait performance were related to higher cognitive control processes. Cognitive resources that were associated with performance decrements in dual-task walking include response inhibition, cognitive flexibility, working memory, and a well-structured mental representation of the gait. From that, it appears that even in healthy young adults better cognitive resources may help to maintain high gait performance in situations, in which we have to deal with dual- or multi-task demands (e.g., using a smartphone) while walking.

Computational assessment of long-term memory structures from SDA-M related to action sequences

Assistance systems should be able to adapt to individual task-related skills and knowledge. Structural-dimensional analysis of mental representations (SDA-M) is an established method for retrieving human memory structures related to specific activities. For this purpose, SDA-M involves a semi-automatized survey of users (the “split procedure”), which yields data about users’ associations between action representations in long-term memory. Up to now this data about associations has commonly been clustered and visualized by SDA-M software in the form of dendrograms that can be used by human experts as a tool to (manually) assess users’ individual expertise and identify potential issues with respect to predefined action sequences. This article presents new algorithmic approaches for automatizing the process of assessing task-related memory structures based on SDA-M data to predict probable errors in action sequences. This automation enables direct integration into technical systems, e.g. user-adaptive assistance systems. An evaluation study has compared the automatized computational assessments to predictions made by human scholars based on visualizations of SDA-M data. The different algorithms’ outputs matched human experts’ manual assessments in 84% to 86% of the test cases.

A Systematic Investigation of the Effect of Action Observation Training and Motor Imagery Training on the Development of Mental Representation Structure and Skill Performance

Action observation training and motor imagery training have independently been studied and considered as an effective training strategy for improving motor skill learning. However, comparative studies of the two training strategies are relatively few. The purpose of this study was to investigate the effects of action observation training and motor imagery training on the development of mental representation structure and golf putting performance as well as the relation between the changes in mental representation structure and skill performance during the early learning stage. Forty novices were randomly assigned to one of four groups: action observation training, motor imagery training, physical practice and no practice. The mental representation structure and putting performance were measured before and after 3 days of training, then after a 2-day retention period. The results showed that mental representation structure and the accuracy of the putting performance were improved over time through the two types of cognitive training (i.e., action observation training and motor imagery training). In addition, we found a significant positive correlation between changes in mental representation structure and skill performance for the action observation training group only. Taken together, these results suggest that both cognitive adaptations and skill improvement occur through the training of the two simulation states of action, and that perceptual-cognitive changes are associated with the change of skill performance for action observation training.

The mental representation of the human gait in patients with severe knee osteoarthrosis: a clinical study to aid understanding of impairment and disability

OBJECTIVE

Objectives were (1) to explore differences in gait-specific long-term memory structures and gait performance between knee osteoarthrosis patients and healthy subjects and (2) to identify the extent to which the gait-specific mental representation is associated with gait performance.

DESIGN

Cross-sectional study.

SUBJECTS

In total, 18 knee osteoarthrosis patients and 18 control subjects.

METHODS

Spatio-temporal (gait speed, step length) and temporophasic (stance time, swing time, single support time, total double support time) gait parameters and gait variability were measured with an electronic walkway (OptoGait). The mental representation was assessed using the structural dimensional analysis of mental representations (SDA-M).

RESULTS

(1) Patients showed significantly longer stance times ( P < 0.002) and total double support times, shorter swing times and single support times, a decreased gait speed ( P-values < 0.001) and structural differences in the gait-specific mental representation as compared with the healthy controls. (2) Correlation analyses revealed the mental representation of the human gait to be associated with actual gait performance in osteoarthrosis patients. Double support times were positively associated with the structural quality of the mental representation and step length variability was positively associated with the number of sequencing errors in the representation.

CONCLUSION

The gait-specific mental representation and actual gait performance differ between patients with severe knee osteoarthrosis and healthy controls, and both are linked to one another. This finding suggests that musculoskeletal disorders can lead to changes in the mental representation of the gait, and as such the SDA-M could provide useful information to improve the rehabilitation following osteoarthrosis.

The Representation of Motor (Inter)action, States of Action, and Learning: Three Perspectives on Motor Learning by Way of Imagery and Execution

Learning in intelligent systems is a result of direct and indirect interaction with the environment. While humans can learn by way of different states of (inter)action such as the execution or the imagery of an action, their unique potential to induce brain- and mind-related changes in the motor action system is still being debated. The systematic repetition of different states of action (e.g., physical and/or mental practice) and their contribution to the learning of complex motor actions has traditionally been approached by way of performance improvements. More recently, approaches highlighting the role of action representation in the learning of complex motor actions have evolved and may provide additional insight into the learning process. In the present perspective paper, we build on brain-related findings and sketch recent research on learning by way of imagery and execution from a hierarchical, perceptual-cognitive approach to motor control and learning. These findings provide insights into the learning of intelligent systems from a perceptual-cognitive, representation-based perspective and as such add to our current understanding of action representation in memory and its changes with practice. Future research should build bridges between approaches in order to more thoroughly understand functional changes throughout the learning process and to facilitate motor learning, which may have particular importance for cognitive systems research in robotics, rehabilitation, and sports.

The BASE-Program-A Multidimensional Approach for Health Promotion in Companies

Multidimensional assessments for conducting interventions are needed to achieve positive health effects within companies. BASE is an acronym, consisting of B = "Bedarfsbestimmung" (requirements); A = "Arbeitsplatzorganisation" (organisation of work); S = "Schulung des belastungsverträglichen Alltagshandelns" (coaching preventive behaviour at work); E = "Eigenverantwortung und Selbstwirksamkeit" (self-responsibility and self-efficacy). It is a prevention program designed to avoid and reduce work-related musculoskeletal diseases. It was developed to support prevention strategies within companies. It comprises aspects of health protection, ergonomics, exercise and self-efficacy. A comprehensive assessment will identify strain e.g., musculoskeletal discomforts due to body positions or psychological stress. Moreover, the general health status, preferences and barriers for participating in health promotion programs are evaluated. This analysis leads to practical and goal-oriented recommendations and interventions which suit the needs of companies and employees. These are executed onsite in real workplace situations and involve the introduction of first-hand experience in behavioural change. Therefore, this practical approach enhances the employees' acceptance and self-efficacy for health promotion. This can result in long-term health promoting behaviour. This article presents the outcome and sustainability effects of BASE in three different application fields (logistic, industrial and office workers).

Cognitive Representation of Human Action: Theory, Applications, and Perspectives

In this perspective article, we propose a cognitive architecture model of human action that stresses the importance of cognitive representations stored in long-term memory as reference structures underlying and guiding voluntary motor performance. We introduce an experimental approach to ascertain cognitive representation structures and provide evidence from a variety of different studies, ranging from basic research in manual action to application-oriented research, such as athlete performance and rehabilitation. As results from these studies strongly support the presence of functional links between cognitive and motor processes, we regard this approach as a suitable and valuable tool for a variety of different disciplines related to cognition and movement. We conclude this article by highlighting current advances in ongoing research projects aimed at improving interaction capabilities in technical systems, particularly for rehabilitation and everyday support of the elderly, and outline future research directions.

Perceptual-Cognitive Changes During Motor Learning: The Influence of Mental and Physical Practice on Mental Representation, Gaze Behavior, and Performance of a Complex Action

Despite the wealth of research on differences between experts and novices with respect to their perceptual-cognitive background (e.g., mental representations, gaze behavior), little is known about the change of these perceptual-cognitive components over the course of motor learning. In the present study, changes in one's mental representation, quiet eye behavior, and outcome performance were examined over the course of skill acquisition as it related to physical and mental practice. Novices (N = 45) were assigned to one of three conditions: physical practice, combined physical plus mental practice, and no practice. Participants in the practice groups trained on a golf putting task over the course of 3 days, either by repeatedly executing the putt, or by both executing and imaging the putt. Findings revealed improvements in putting performance across both practice conditions. Regarding the perceptual-cognitive changes, participants practicing mentally and physically revealed longer quiet eye durations as well as more elaborate representation structures in comparison to the control group, while this was not the case for participants who underwent physical practice only. Thus, in the present study, combined mental and physical practice led to both formation of mental representations in long-term memory and longer quiet eye durations. Interestingly, the length of the quiet eye directly related to the degree of elaborateness of the underlying mental representation, supporting the notion that the quiet eye reflects cognitive processing. This study is the first to show that the quiet eye becomes longer in novices practicing a motor action. Moreover, the findings of the present study suggest that perceptual and cognitive adaptations co-occur over the course of motor learning.

The mental representation of the human gait in young and older adults

The link between mental representation (MREP) structures and motor performance has been evidenced for a great variety of movement skills, but not for the human gait. Therefore the present study sought to investigate the cognitive memory structures underlying the human gait in young and older adults. In a first experiment, gait parameters at comfortable gait speed (OptoGait) were compared with gait-specific MREPs (structural dimensional analysis of MREP; SDA-M) in 36 young adults. Participants were divided into a slow- and fast-walking group. The proven relationship between gait speed and executive functions such as working memory led to the hypothesis that gait pattern and MREP differ between slow- and fast-walking adults. In a second experiment, gait performance and MREPs were compared between 24 young (27.9 years) and 24 elderly (60.1 years) participants. As age-related declines in gait performance occur from the seventh decade of life onward, we hypothesized that gait parameters would not be affected until the age of 60 years accompanied by unchanged MREP. Data of experiment one revealed that gait parameters and MREPs differed significantly between slow and fast walkers. Notably, eleven previously incurred musculoskeletal injuries were documented for the slow walkers but only two injuries and one disorder for fast walkers. Experiment two revealed no age-related differences in gait parameters or MREPs between healthy young and older adults. In conclusion, the differences in gait parameters associated with lower comfortable gait speeds are reflected by differences in MREPs, whereby SDA-M data indicate that the single limb support phase may serve as a critical functional period. These differences probably resulted from previously incurred musculoskeletal injuries. Our data further indicate that the human gait and its MREP are stable until the age of 60. SDA-M may be considered as a valuable clinical tool for diagnosis of gait abnormalities and monitoring of therapeutic effectiveness.

Decoding of human hand actions to handle missing limbs in neuroprosthetics

The only way we can interact with the world is through movements, and our primary interactions are via the hands, thus any loss of hand function has immediate impact on our quality of life. However, to date it has not been systematically assessed how coordination in the hand's joints affects every day actions. This is important for two fundamental reasons. Firstly, to understand the representations and computations underlying motor control "in-the-wild" situations, and secondly to develop smarter controllers for prosthetic hands that have the same functionality as natural limbs. In this work we exploit the correlation structure of our hand and finger movements in daily-life. The novelty of our idea is that instead of averaging variability out, we take the view that the structure of variability may contain valuable information about the task being performed. We asked seven subjects to interact in 17 daily-life situations, and quantified behavior in a principled manner using CyberGlove body sensor networks that, after accurate calibration, track all major joints of the hand. Our key findings are: (1) We confirmed that hand control in daily-life tasks is very low-dimensional, with four to five dimensions being sufficient to explain 80-90% of the variability in the natural movement data. (2) We established a universally applicable measure of manipulative complexity that allowed us to measure and compare limb movements across tasks. We used Bayesian latent variable models to model the low-dimensional structure of finger joint angles in natural actions. (3) This allowed us to build a naïve classifier that within the first 1000 ms of action initiation (from a flat hand start configuration) predicted which of the 17 actions was going to be executed-enabling us to reliably predict the action intention from very short-time-scale initial data, further revealing the foreseeable nature of hand movements for control of neuroprosthetics and tele operation purposes. (4) Using the Expectation-Maximization algorithm on our latent variable model permitted us to reconstruct with high accuracy (<5-6° MAE) the movement trajectory of missing fingers by simply tracking the remaining fingers. Overall, our results suggest the hypothesis that specific hand actions are orchestrated by the brain in such a way that in the natural tasks of daily-life there is sufficient redundancy and predictability to be directly exploitable for neuroprosthetics.

The influence of visual feedback on the mental representation of gait in patients with THR: a new approach for an experimental rehabilitation strategy

Due to total hip replacement (THR), patients reveal abnormal gait patterns which post-operative do often not return to "normal". The restoration towards normal gait reduces stress on the adjacent joints which consequently reduces risk of osteoarthrosis development. Motor-performance is related to the structure of the movement in long-term memory, thus it seems to be essential to imprint correct gait patterns in there. Mental representation structures can develop over the course of training and visual feedback presumably helps regaining a better representation of gait in long-term memory. The purpose of this study is to evaluate the effect of visual feedback on mental representation in patients with THR. In a randomized controlled trial, 20 women (57 ± 6 years) with THR have been enrolled. Subjects were randomly assigned to a control group (CG) or intervention group (IG). Additionally to inpatient treatment, all subjects participated in a standardized gait training including either an intervention based on verbal information from a physiotherapist (CG) or an intervention based on real-time visual feedback (IG). Mental representation was measured in pre-test and post-test using the structure-dimensional analysis. Results indicate significant improvements in mental representation of gait in the post-test only in IG, suggesting that beneficial effects were provoked by visual feedback.

Expertise affects representation structure and categorical activation of grasp postures in climbing

In indoor rock climbing, the perception of object properties and the adequate execution of grasping actions highly determine climbers' performance. In two consecutive experiments, effects of climbing expertise on the cognitive activation of grasping actions following the presentation of climbing holds was investigated. Experiment 1 evaluated the representation of climbing holds in the long-term memory of climbers and non-climbers with the help of a psychometric measurement method. Within a hierarchical splitting procedure subjects had to decide about the similarity of required grasping postures. For the group of climbers, representation structures corresponded clearly to four grip types. In the group of non-climbers, representation structures differed more strongly than in climbers and did not clearly refer to grip types. To learn about categorical knowledge activation in Experiment 2, a priming paradigm was applied. Images of hands in grasping postures were presented as targets and images of congruent, neutral, or incongruent climbing holds were used as primes. Only in climbers, reaction times were shorter and error rates were smaller for the congruent condition than for the incongruent condition. The neutral condition resulted in intermediate performance. The findings suggest that perception of climbing holds activates the commonly associated grasping postures in climbers but not in non-climbers. The findings of this study give evidence that the categorization of visually perceived objects is fundamentally influenced by the cognitive-motor potential for interaction, which depends on the observer's experience and expertise. Thus, motor expertise not only facilitates precise action perception, but also benefits the perception of action-relevant objects.

Mental representation and motor imagery training

Research in sports, dance and rehabilitation has shown that basic action concepts (BACs) are fundamental building blocks of mental action representations. BACs are based on chunked body postures related to common functions for realizing action goals. In this paper, we outline issues in research methodology and an experimental method, the structural dimensional analysis of mental representation (SDA-M), to assess action-relevant representational structures that reflect the organization of BACs. The SDA-M reveals a strong relationship between cognitive representation and performance if complex actions are performed. We show how the SDA-M can improve motor imagery training and how it contributes to our understanding of coaching processes. The SDA-M capitalizes on the objective measurement of individual mental movement representations before training and the integration of these results into the motor imagery training. Such motor imagery training based on mental representations (MTMR) has been applied successfully in professional sports such as golf, volleyball, gymnastics, windsurfing, and recently in the rehabilitation of patients who have suffered a stroke.

Mental representation and mental practice: experimental investigation on the functional links between motor memory and motor imagery

Recent research on mental representation of complex action has revealed distinct differences in the structure of representational frameworks between experts and novices. More recently, research on the development of mental representation structure has elicited functional changes in novices' representations as a result of practice. However, research investigating if and how mental practice adds to this adaptation process is lacking. In the present study, we examined the influence of mental practice (i.e., motor imagery rehearsal) on both putting performance and the development of one's representation of the golf putt during early skill acquisition. Novice golfers (N = 52) practiced the task of golf putting under one of four different practice conditions: mental, physical, mental-physical combined, and no practice. Participants were tested prior to and after a practice phase, as well as after a three day retention interval. Mental representation structures of the putt were measured, using the structural dimensional analysis of mental representation. This method provides psychometric data on the distances and groupings of basic action concepts in long-term memory. Additionally, putting accuracy and putting consistency were measured using two-dimensional error scores of each putt. Findings revealed significant performance improvements over the course of practice together with functional adaptations in mental representation structure. Interestingly, after three days of practice, the mental representations of participants who incorporated mental practice into their practice regime displayed representation structures that were more similar to a functional structure than did participants who did not incorporate mental practice. The findings of the present study suggest that mental practice promotes the cognitive adaptation process during motor learning, leading to more elaborate representations than physical practice only.

Motor control as the control of perception

This paper describes a test of Perceptual Control Theory (PCT), which views motor control as part of a process of controlling perceptual inputs rather than motor outputs. Sixteen undergraduate students (M age = 19.9 yr.) were asked to control one of three different perceptual aspects of an animated display--a shape, a motion or a sequence--using the same motor output, a key press. Animation rate was varied while quality of control was measured in terms of the proportion of time that the perception was maintained in the goal state. The results showed that increased animation rate made it hardest to control the more complex perceptions (motion and sequence) even though the same output was used to control all perceptions. This result is consistent with PCT, which predicts that the temporal constraints on control are ultimately a function of the type of perception controlled rather than the type of output used to control it.

Whole-body posture planning in anticipation of a manual prehension task: prospective and retrospective effects

This study examined the extent to which the anticipation of a manual action task influences whole-body postural planning and orientation. Our participants walked up to a drawer, opened the drawer, then grasped and moved an object in the drawer to another location in the same drawer. The starting placement of the object within the drawer and the final placement of the object in the drawer were varied across trials in either a blocked design (i.e., in trials where the same start and end location were repeated consecutively) or in a mixed fashion. Of primary interest was the posture adopted at the moment of grasping the drawer handle before pulling it out prior to the object manipulation task. Of secondary interest was whether there were sequential effects such that postures adopted in preceding trials influenced postures in subsequent trials. The results indicated that the spatial properties of the forthcoming object manipulation influenced both the postures adopted by the participants and the degree to which the drawer was opened, suggesting a prospective effect. In addition, the adopted postures were more consistent in blocked trials than in mixed trials, suggesting an additional retrospective effect. Overall, our findings suggest that motor planning occurs at the level of the whole body, and reflects both prospective and retrospective influences.

A hexapod walker using a heterarchical architecture for action selection

Moving in a cluttered environment with a six-legged walking machine that has additional body actuators, therefore controlling 22 DoFs, is not a trivial task. Already simple forward walking on a flat plane requires the system to select between different internal states. The orchestration of these states depends on walking velocity and on external disturbances. Such disturbances occur continuously, for example due to irregular up-and-down movements of the body or slipping of the legs, even on flat surfaces, in particular when negotiating tight curves. The number of possible states is further increased when the system is allowed to walk backward or when front legs are used as grippers and cannot contribute to walking. Further states are necessary for expansion that allow for navigation. Here we demonstrate a solution for the selection and sequencing of different (attractor) states required to control different behaviors as are forward walking at different speeds, backward walking, as well as negotiation of tight curves. This selection is made by a recurrent neural network (RNN) of motivation units, controlling a bank of decentralized memory elements in combination with the feedback through the environment. The underlying heterarchical architecture of the network allows to select various combinations of these elements. This modular approach representing an example of neural reuse of a limited number of procedures allows for adaptation to different internal and external conditions. A way is sketched as to how this approach may be expanded to form a cognitive system being able to plan ahead. This architecture is characterized by different types of modules being arranged in layers and columns, but the complete network can also be considered as a holistic system showing emergent properties which cannot be attributed to a specific module.

Event-related brain potentials for goal-related power grips

Recent research has shown that neurophysiological activation during action planning depends on the orientation to initial or final action goals for precision grips. However, the neural signature for a distinct class of grasping, power grips, is still unknown. The aim of the present study was to differentiate between cerebral activity, by means of event-related potentials (ERPs), and its temporal organization during power grips executed with an emphasis on either the initial or final parts of movement sequences. In a grasp and transportation task, visual cues emphasized either the grip (the immediate goal) or the target location (the final goal). ERPs differed between immediate and final goal-cued conditions, suggesting different means of operation dependent on goal-relatedness. Differences in mean amplitude occurred earlier for power grips than for recently reported precision grips time-locked to grasping over parieto-occipital areas. Time-locked to final object placement, differences occurred within a similar time window for power and precision grips over frontal areas. These results suggest that a parieto-frontal network of activation is of crucial importance for grasp planning and execution. Our results indicate that power grip preparation and execution for goal-related actions are controlled by similar neural mechanisms as have been observed during precision grips, but with a distinct temporal pattern.

Representing the egocentric auditory space: relationships of surrounding region concepts

We investigated the representation of azimuthal directions of sound sources under two different conditions. In the first experiment, we examined the participants' mental representation of sound source directions via similarity judgments. Auditory stimuli originating from sixteen loudspeakers positioned equidistantly around the participant were presented in pairs, with the first stimulus serving as the anchor, and thereby providing the context for the second stimulus. For each pair of stimuli, participants had to rate the sound source directions as either similar or dissimilar. In the second experiment, the same participants categorized single sound source directions using verbal direction labels (front, back, left, right, and combinations of any two of these). In both experiments, the directions within the front and back regions were more distinctively categorized than those on the sides, and the sides' categories included more directions than those of the front or back. Furthermore, we found evidence that the left-right decision comprises the basic differentiation of the surrounding regions. These findings illustrate what seem to be central features of the representation of directions in auditory space.

Functional relationship between cognitive representations of movement directions and visuomotor adaptation performance

The aim of our study was to explore whether or not different types of learners in a sensorimotor task possess characteristically different cognitive representations. Participants' sensorimotor adaptation performance was measured with a pointing paradigm which used a distortion of the visual feedback in terms of a left-right reversal. The structure of cognitive representations was assessed using a newly established experimental method, the Cognitive Measurement of Represented Directions. A post hoc analysis revealed inter-individual differences in participants' adaptation performance, and three different skill levels (skilled, average, and poor adapters) have been defined. These differences in performance were correlated with the structure of participants' cognitive representations of movement directions. Analysis of these cognitive representations revealed performance advantages for participants possessing a global cognitive representation of movement directions (aligned to cardinal movement axes), rather than a local representation (aligned to each neighboring direction). Our findings are evidence that cognitive representation structures play a functional role in adaptation performance.