Conceptual unifying constraints override sensorimotor interference during anticipatory control of bimanual actions

Crosstalk disrupts the production of motor imagery brain signals in brain-computer interfaces

Brain-computer interfaces (BCIs) target specific brain activity for neuropsychological rehabilitation, and also allow patients with motor disabilities to control mobility and communication devices. Motor imagery of single-handed actions is used in BCIs but many users cannot control the BCIs effectively, limiting applications in the health systems. Crosstalk is unintended brain activations that interfere with bimanual actions and could also occur during motor imagery. To test if crosstalk impaired BCI user performance, we recorded EEG in 46 participants while they imagined movements in four experimental conditions using motor imagery: left hand (L), right hand (R), tongue (T) and feet (F). Pairwise classification accuracies of the tasks were compared (LR, LF, LT, RF, RT, FT), using common spatio-spectral filters and linear discriminant analysis. We hypothesized that LR classification accuracy would be lower than every other combination that included a hand imagery due to crosstalk. As predicted, classification accuracy for LR (58%) was reliably the lowest. Interestingly, participants who showed poor LR classification also demonstrated at least one good TR, TL, FR or FL classification; and good LR classification was detected in 16% of the participants. For the first time, we showed that crosstalk occurred in motor imagery, and affected BCI performance negatively. Such effects are effector-sensitive regardless of the BCI methods used; and likely not apparent to the user or the BCI developer. This means that tasks choice is crucial when designing BCI. Critically, the effects of crosstalk appear mitigatable. We conclude that understanding crosstalk mitigation is important for improving BCI applicability.

Supplementary Information

The online version of this article contains supplementary material available (10.1007/s13755-021-00142-y).

Influence of visual-coupling on bimanual coordination in unilateral spastic cerebral palsy

Controlling two objects simultaneously during a bimanual task is a cognitively demanding process; both hands need to be temporally and spatially coordinated to achieve the shared task goal. Children with unilateral spastic cerebral palsy (USCP) exhibit severe sensory and motor impairments to one side of their body that make the process of coordinating bimanual movements particularly exhausting. Prior studies have shown that performing visually-coupled task could reduce cognitive interference associated with performing 'two tasks at once' in an uncoupled bimanual task. For children with USCP, who also present with cognitive delay, performing this type of task may allow them to process and plan their movement faster. We tested this hypothesis by examining the grip force control of 7 children with USCP during unimanual and visually-coupled bimanual tasks. Results demonstrated that despite the visual coupling, the bimanual coordination of these children remained impaired. However, there may be a potential benefit of visually-coupled task in encouraging both hands to initiate in concert. The implication of the study for children with USCP is discussed.

Bimanual coordination during a physically coupled task in unilateral spastic cerebral palsy children

Background

Single object bimanual manipulation, or physically-coupled bimanual tasks, are ubiquitous in daily lives. However, the predominant focus of previous studies has been on uncoupled bimanual actions, where the two hands act independently to manipulate two disconnected objects. In this paper, we explore interlimb coordination among children with unilateral spastic cerebral palsy (USCP), by investigating upper limb motor control during a single object bimanual lifting task.

Methods

15 children with USCP and 17 typically developing (TD) children performed a simple single-object bimanual lifting task. The object was an instrumented cube that can record the contact force on each of its faces alongside estimating its trajectory during a prescribed two-handed lifting motion. The subject’s performance was measured in terms of the duration of individual phases, linearity and monotonicity of the grasp-to-load force synergy, interlimb force asymmetry, and movement smoothness.

Results

Similar to their TD counterparts, USCP subjects were able to produce a linear grasp-to-load force synergy. However, they demonstrated difficulties in producing monotonic forces and generating smooth movements. No impairment of anticipatory control was observed within the USCP subjects. However, our analysis showed that the USCP subjects shifted the weight of the cube onto their more-abled side, potentially to minimise the load on the impaired side, which suggests a developed strategy of compensating for inter-limb asymmetries, such as muscle strength.

Conclusion

Bimanual interaction with a single mutual object has the potential to facilitate anticipation and sequencing of force control in USCP children unlike previous studies which showed deficits during uncoupled bimanual actions. We suggest that this difference could be partly due to the provision of adequate cutaneous and kinaesthetic information gathered from the dynamic exchange of forces between the two hands, mediated through the physical coupling.

Does motor expertise facilitate amplitude differentiation of lower limb-movements in an asymmetrical bipedal coordination task?

The motor system's natural tendency is to move the limbs over equal amplitudes, for example in walking. However, in many situations in which people must perform complex movements, a certain degree of amplitude differentiation of the limbs is required. Visual and haptic feedback have recently been shown to facilitate such independence of limb movements. However, it is unknown whether motor expertise moderates the extent to which individuals are able to differentiate the amplitudes of their limb-movements while being supported with visual and haptic feedback. To answer this question 14 pre-professional dancers were compared to 14 non-dancers on simultaneously generating a small displacement with one foot, and a larger one with the other foot, in four different feedback conditions. In two conditions, haptic guidance was offered, either in a passive or active mode. In the other two conditions, veridical and enhanced visual feedback were provided. Surprisingly, no group differences were found regarding the degree to which the visual or haptic feedback assisted the generation of the different target amplitudes of the feet (mean amplitude difference between the feet). The correlation between the displacements of the feet and the standard deviation of the continuous relative phase between the feet, reflecting the degree of independence of the feet movements, also failed to show between-group differences. Sample entropy measures, indicating the predictability of the foot movements, did show a group difference. In the haptically-assisted conditions, the dancers demonstrated more predictable coordination patterns than the non-dancers as reflected by lower sample entropy values whereas the reverse was true in the visual-feedback conditions. The results demonstrate that motor expertise does not moderate the extent to which haptic tracking facilitates the differentiation of the amplitudes of the lower limb movements in an asymmetrical bipedal coordination task.

Effects of integrated feedback on discrete bimanual movements in choice reaction time

The ability to coordinate the simultaneous movements of our arms is limited by a coalition of constraints. Some of these constraints can be overcome when the task conceptualisation is improved. The present study investigated how the movement preparation of bimanual reaching movements was affected by integrated visual feedback of the responses. Previous research has shown that the preparation of bimanual asymmetric movements takes longer than bimanual symmetric movements. The goal of the present study was to determine whether integrated, Lissajous feedback could eliminate this bimanual asymmetric cost. Fifteen participants made unimanual and bimanual symmetric and asymmetric reaches with separate feedback, where there was a cursor and a target for each hand. Participants also made bimanual symmetric and asymmetric movements with integrated feedback; a single cursor and a single target represented the locations and goals of both arms in this condition. The results showed a bimanual asymmetric cost with separate feedback, and that this cost persisted with integrated feedback. We suggest that integrated feedback improved continuous and discrete bimanual movements in other experiments by facilitating error detection and correction processes. We hypothesise that the bimanual asymmetric cost persisted in the present experiment because the uncertainty associated with choice reaction time prevented the facilitated error processing from improving the preparation of the next trial.

Haptic feedback helps bipedal coordination

The present study investigated whether special haptic or visual feedback would facilitate the coordination of in-phase, cyclical feet movements of different amplitudes. Seventeen healthy participants sat with their feet on sliding panels that were moved externally over the same or different amplitudes. The participants were asked to generate simultaneous knee flexion-extension movements, or to let their feet be dragged, resulting in reference foot displacements of 150 mm and experimental foot displacements of 150, 120, or 90 mm. Four types of feedback were given: (1) special haptic feedback, involving actively following the motions of the sliders manipulated by two confederates, (2) haptic feedback resulting from passive motion, (3) veridical visual feedback, and (4) enhanced visual feedback. Both with respect to amplitude assimilation effects, correlations and standard deviation of relative phase, the results showed that enhanced visual feedback did not facilitate bipedal independence, but haptic feedback with active movement did. Implications of the findings for movement rehabilitation contexts are discussed.

Differentiating Visual from Response Sequencing during Long-term Skill Learning

The dual-system model of sequence learning posits that during early learning there is an advantage for encoding sequences in sensory frames; however, it remains unclear whether this advantage extends to long-term consolidation. Using the serial RT task, we set out to distinguish the dynamics of learning sequential orders of visual cues from learning sequential responses. On each day, most participants learned a new mapping between a set of symbolic cues and responses made with one of four fingers, after which they were exposed to trial blocks of either randomly ordered cues or deterministic ordered cues (12-item sequence). Participants were randomly assigned to one of four groups (n = 15 per group): Visual sequences (same sequence of visual cues across training days), Response sequences (same order of key presses across training days), Combined (same serial order of cues and responses on all training days), and a Control group (a novel sequence each training day). Across 5 days of training, sequence-specific measures of response speed and accuracy improved faster in the Visual group than any of the other three groups, despite no group differences in explicit awareness of the sequence. The two groups that were exposed to the same visual sequence across days showed a marginal improvement in response binding that was not found in the other groups. These results indicate that there is an advantage, in terms of rate of consolidation across multiple days of training, for learning sequences of actions in a sensory representational space, rather than as motoric representations.

Transposing musical skill: sonification of movement as concurrent augmented feedback enhances learning in a bimanual task

Concurrent feedback provided during acquisition can enhance performance of novel tasks. The 'guidance hypothesis' predicts that feedback provision leads to dependence and poor performance in its absence. However, appropriately structured feedback information provided through sound ('sonification') may not be subject to this effect. We test this directly using a rhythmic bimanual shape-tracing task in which participants learned to move at a 4:3 timing ratio. Sonification of movement and demonstration was compared to two other learning conditions: (1) Sonification of task demonstration alone and (2) completely silent practice (control). Sonification of movement emerged as the most effective form of practice, reaching significantly lower error scores than control. Sonification of solely the demonstration, which was expected to benefit participants by perceptually unifying task requirements, did not lead to better performance than control. Good performance was maintained by participants in the Sonification condition in an immediate retention test without feedback, indicating that the use of this feedback can overcome the guidance effect. On a 24-h retention test, performance had declined and was equal between groups. We argue that this and similar findings in the feedback literature are best explained by an ecological approach to motor skill learning which places available perceptual information at the highest level of importance.

Cognitive framing in action

Cognitive framing effects have been widely reported in higher-level decision-making and have been ascribed to rules of thumb for quick thinking. No such demonstrations have been reported for physical action, as far as we know, but they would be expected if cognition for physical action is fundamentally similar to cognition for higher-level decision-making. To test for such effects, we asked participants to reach for a horizontally-oriented pipe to move it from one height to another while turning the pipe 180° to bring one end (the "business end") to a target on the left or right. From a physical perspective, participants could have always rotated the pipe in the same angular direction no matter which end was the business end; a given participant could have always turned the pipe clockwise or counter-clockwise. Instead, our participants turned the business end counter-clockwise for left targets and clockwise for right targets. Thus, the way the identical physical task was framed altered the way it was performed. This finding is consistent with the hypothesis that cognition for physical action is fundamentally similar to cognition for higher-level decision-making. A tantalizing possibility is that higher-level decision heuristics have roots in the control of physical action, a hypothesis that accords with embodied views of cognition.

How does consciousness for action relate to attention for action?

The interesting target article by Morsella et al. addresses critical issues that impact our understanding of consciousness. It is surprising, however, to see no treatment of the relationship between attention and consciousness, particularly given available models. Whether olfaction is most suitable as a model system to study consciousness for action also seems questionable. These issues are elaborated in the present commentary.

Unified nature of bimanual movements revealed by separating the preparation of each arm

Movement preparation of bimanual asymmetric movements is longer than bimanual symmetric movements in choice reaction time conditions, even when movements are cued directly by illuminating the targets (Blinch et al. in Exp Brain Res 232(3):947-955, 2014). This bimanual asymmetric cost may be caused by increased processing demands on response programming, but this requires further investigation. The present experiment tested the demands on response programming for bimanual movements by temporally separating the preparation of each arm. This was achieved by precuing the target of one arm before the imperative stimulus. We asked: What was prepared in advance when one arm was precued? The answer to this question would suggest which process causes the bimanual asymmetric cost. Advance movement preparation was examined by comparing reaction times with and without a precue for the left target and by occasionally replacing the imperative stimulus with a loud, startling tone (120 dB). A startle tone releases whatever movement is prepared in advance with a much shorter reaction time than control trials (Carlsen et al. in Clin Neurophysiol 123(1):21-33, 2012). Participants made bimanual symmetric and asymmetric reaching movements in simple and 2-choice reaction time conditions and a condition with a precue for the left target. We found a bimanual asymmetric cost in 2-choice conditions, and the asymmetric cost was significantly smaller when the left target was precued. These results, and the results from startle trials, suggest (1) that the precued movement was not fully programmed but partially programmed before the imperative stimulus and (2) that the asymmetric cost was caused by increased processing demands on response programming. Overall, the results support the notion that bimanual movements are not the sum of two unimanual movements; instead, the two arms of a bimanual movement are unified into a functional unit. When one target is precued, this critical unification likely occurs during response programming.

Symmetries in action: on the interactive nature of planning constraints for bimanual object manipulation

An important functional question for understanding how people perform physical actions is to understand how they manipulate objects. Previous research suggests that people prefer to move their hands symmetrically. For bimanual object manipulation, such symmetry may take on several forms, however. Actions may be symmetrical when objects are grasped (start symmetry), when they are placed on their target locations (target symmetry), and/or relative to the objects being moved (object symmetry). We studied how these forms of symmetry influenced grasp selection when participants moved two plungers from two start locations to two target locations. We varied the heights of these locations across conditions. The grasp locations participants adopted indicated a preference for object symmetry. This preference was even stronger when initial symmetry coincided with object symmetry. These results provide a tractable illustration of how multiple planning constraints may interact to give rise to both regularity and flexibility in motor behavior.

Problems in planning bimanually incongruent grasp postures relate to simultaneous response specification processes

The purpose of the current experiments was to examine whether the problems associated with grasp posture planning during bimanually incongruent movements are due to crosstalk at the motor programming level. Participants performed a grasping and placing task in which they grasped two objects from a table and placed them onto a board to targets that required identical (congruent) or non-identical degrees of rotation (incongruent). The interval between the presentation of the first stimulus and the second stimulus (stimulus onset asynchrony: SOA) was manipulated. Results demonstrate that the problems associated with bimanually incongruent grasp posture planning are reduced at SOA durations longer than 1000ms, indicating that the costs associated with bimanual incongruent movements arise from crosstalk at the motor programming level. In addition, reach-to-grasp times were shorter, and interlimb limb coupling was higher, for congruent, compared to incongruent, object end-orientation conditions in both Experiment 1 and 2. The bimanual interference observed during reach-to-grasp execution is postulated to arise from limitations in the visual motor system or from conceptual language representations. The present results emphasize that bimanual interference arises from constraints active at multiple levels of the neurobiological-cognitive system.

The effects of task demands on bimanual skill acquisition

Bimanual coordination is essential for everyday activities. It is thought that different degrees of demands may affect learning of new bimanual patterns. One demand is at the level of performance and involves breaking the tendency to produce mirror-symmetric movements. A second is at a perceptual level and involves controlling each hand to separate (i.e., split) goals. A third demand involves switching between different task contexts (e.g., a different uni- or bimanual task), instead of continuously practicing one task repeatedly. Here, we studied the effect of these task demands on motor planning (reaction time) and execution (error) while subjects learned a novel bimanual isometric pinch force task. In Experiment 1, subjects continuously practiced in one of the two extremes of the following bimanual conditions: (1) symmetric force demands and a perceptually unified target for each hand or (2) asymmetric force demands and perceptually split targets. Subjects performing in the asymmetric condition showed some interference between hands, but all subjects, regardless of group, could learn the isometric pinch force task similarly. In Experiment 2, subjects practiced these and two other conditions, but in a paradigm where practice was briefly interrupted by the performance of either a unimanual or a different bimanual condition. Reaction times were longer and errors were larger well after the interruption when the main movement to be learned required asymmetric forces. There was no effect when the main movement required symmetric forces. These findings demonstrate two main points. First, people can learn bimanual tasks with very different demands on the same timescale if they are not interrupted. Second, interruption during learning can negatively impact both planning and execution and this depends on the demands of the bimanual task to be learned. This information will be important for training patient populations, who may be more susceptible to increased task demands.

Diffusion tensor imaging metrics of the corpus callosum in relation to bimanual coordination: effect of task complexity and sensory feedback

When manipulating objects with both hands, the corpus callosum (CC) is of paramount importance for interhemispheric information exchange. Hence, CC damage results in impaired bimanual performance. Here, healthy young adults performed a complex bimanual dial rotation task with or without augmented visual feedback and according to five interhand frequency ratios (1:1, 1:3, 2:3, 3:1, 3:2). The relation between bimanual task performance and microstructural properties of seven CC subregions (i.e., prefrontal, premotor/supplementary motor, primary motor, primary sensory, occipital, parietal, and temporal) was studied by means of diffusion tensor imaging (DTI). Findings revealed that bimanual coordination deteriorated in the absence as compared to the presence of augmented visual feedback. Simple frequency ratios (1:1) were performed better than the multifrequency ratios (non 1:1). Moreover, performance was more accurate when the preferred hand (1:3-2:3) as compared to the nonpreferred hand (3:1-3:2) moved faster and during noninteger (2:3-3:2) as compared to integer frequency ratios (1:3-3:1). DTI findings demonstrated that bimanual task performance in the absence of augmented visual feedback was significantly related to the microstructural properties of the primary motor and occipital region of the CC, suggesting that white matter microstructure is associated with the ability to perform bimanual coordination patterns in young adults.

The allocation of attention to learning of goal-directed actions: a cognitive neuroscience framework focusing on the Basal Ganglia

The present paper builds on the idea that attention is largely in service of our actions. A framework and model which captures the allocation of attention for learning of goal-directed actions is proposed and developed. This framework highlights an evolutionary model based on the notion that rudimentary functions of the basal ganglia have become embedded into increasingly higher levels of networks which all contribute to adaptive learning. Supporting the proposed model, background literature is presented alongside key evidence based on experimental studies in the so-called "split-brain" (surgically divided cerebral hemispheres), and selected evidence from related areas of research. Although overlap with other existing findings and models is acknowledged, the proposed framework is an original synthesis of cognitive experimental findings with supporting evidence of a neural system and a carefully formulated model of attention. It is the hope that this new synthesis will be informative in fields of cognition and other fields of brain sciences and will lead to new avenues for experimentation across domains.

The impact of perceptual, cognitive and motor factors on bimanual coordination

Bimanual coordination is governed by constraints that permit congruent movements to be performed more easily than incongruent movements. Theories concerning the origin of these constraints range from low level motor-muscle explanations to high level perceptual-cognitive ones. To elucidate the processes underlying coordinative constraints, we asked subjects to use a pair of left-right joysticks to acquire corresponding pairs of congruent and incongruent targets presented on a video monitor under task conditions designed to systematically modulate the impact of several perceptual-cognitive processes commonly required for bimanual task performance. These processes included decoding symbolic cues, detecting goal targets, conceptualizing movements in terms of goal target configuration, planning movement trajectories, producing saccades and perceiving visual feedback. Results demonstrate that constraints arise from target detection and trajectory planning processes that can occur prior to movement initiation as well as from inherent muscle properties that emerge during movement execution, and that the manifestation of these constraints can be significantly altered by the ability to visually monitor movement progress.

John Hughlings Jackson's evolutionary neurology: a unifying framework for cognitive neuroscience

John Hughlings Jackson was a pioneer in neurology who thought deeply about the structure of the brain and how that manifested itself in the various syndromes that he saw in the clinic. He enunciated a theory of the evolution and dissolution of neural function based on the idea that basic sensorimotor processes become embedded in networks of connections that relate them in successively more complex ways to allow for performance of more and more nuanced and adaptive functions. Hughlings Jackson noted the curious link between human thought, action and speech. He further recognized that disinhibition or release from control and direction marked neurological damage. His integrative framework remains deeply relevant to the plethora of results being produced by the careful and diverse experimentation currently undertaken with the aid of brain imaging techniques of which he could only dream. In celebration of the memory of John Hughlings Jackson, we revisit his concept of neural evolution and development, which led to what eventually became a leading model of brain organization, whereby a new order of behavioural control--the conscious mind--is created out of simpler elements, in a manner similar to Herbert Spencer's evolutionary theory. By this Hughlings Jackson did not mean anything dualistic but merely that the highest layer of evolution of nervous arrangements was 'highly complicated' and that dissolution of that higher level leaves 'a lower consciousness and a shallower nervous system'.