Overcoming the ceiling effects of experts' motor expertise through active haptic training

Adaptive tactile interaction transfer via digitally embroidered smart gloves

Human-machine interfaces for capturing, conveying, and sharing tactile information across time and space hold immense potential for healthcare, augmented and virtual reality, human-robot collaboration, and skill development. To realize this potential, such interfaces should be wearable, unobtrusive, and scalable regarding both resolution and body coverage. Taking a step towards this vision, we present a textile-based wearable human-machine interface with integrated tactile sensors and vibrotactile haptic actuators that are digitally designed and rapidly fabricated. We leverage a digital embroidery machine to seamlessly embed piezoresistive force sensors and arrays of vibrotactile actuators into textiles in a customizable, scalable, and modular manner. We use this process to create gloves that can record, reproduce, and transfer tactile interactions. User studies investigate how people perceive the sensations reproduced by our gloves with integrated vibrotactile haptic actuators. To improve the effectiveness of tactile interaction transfer, we develop a machine-learning pipeline that adaptively models how each individual user reacts to haptic sensations and then optimizes haptic feedback parameters. Our interface showcases adaptive tactile interaction transfer through the implementation of three end-to-end systems: alleviating tactile occlusion, guiding people to perform physical skills, and enabling responsive robot teleoperation.

Active perceptual learning involves motor exploration and adaptation of predictive sensory integration

Our ability to perceive both externally generated and self-generated sensory stimuli can be enhanced through training, known as passive and active perceptual learning (APL). Here, we sought to explore the mechanisms underlying APL by using active haptic training (AHT), which has been demonstrated to enhance the somatosensory perception of a finger in a trained motor skill. In total 120 pianists participated in this study. First, AHT reorganized the muscular coordination during the piano keystroke. Second, AHT increased the relative reliance on afferent sensory information relative to predicted one, in contrast to no increment of overall perceptual sensitivity. Finally, AHT improved feedback movement control of keystrokes. These results suggest that APL involves active exploration and adaptation of predictive sensory integration, which underlies the co-enhancement of active perception and feedback control of movements of well-trained individuals.

The Effect of Different Combinations of Practice Schedules on Motor Response Stability during Practice

Many results in motor learning have indicated that relative and absolute timing dimensions are modulated by factors that modify response stability among trials. One of these factors is the combination of constant and variable practices. Although many researchers have investigated the combination of practice schedules, these researchers have used measurements that do not assess performance and motor response separately. This study aimed to investigate the effect of different combinations of practice schedules on motor response stability during practice. Participants performed a sequential key-pressing task with two goals: (1) to learn the relative timing dimension and (2) the absolute timing dimension. Participants were assigned to one of two groups: constant-variable or variable-constant. Our findings indicate an influence of the increase in variability over the practice in the constant-variable group. Precisely, the increase in variability of total time in the second half (constant-variable group) of practice was followed by the maintenance of the same level of cross-correlate between absolute timing error and variability of total time. Finally, our findings support the hypothesis that practicing in a constant schedule favors the relative timing dimension of learning regardless of the order in which the constant practice is provided.

Passive somatosensory training enhances piano skill in adolescent and adult pianists: A preliminary study

Sensory afferent information, such as auditory and somatosensory feedback while moving, plays a crucial role in both control and learning of motor performance across the lifespan. Music performance requires skillful integration of multimodal sensory information for the production of dexterous movements. However, it has not been understood what roles somatosensory afferent information plays in the acquisition and sophistication of specialized motor skills of musicians across different stages of development. In the present preliminary study, we addressed this issue by using a novel technique with a hand exoskeleton robot that can externally move the fingers of pianists. Short-term exposure to fast and complex finger movements generated by the exoskeleton (i.e., passive movements) increased the maximum rate of repetitive piano keystrokes by the pianists. This indicates that somatosensory inputs derived from the externally generated motions enhanced the quickness of the sequential finger movements in piano performance, even though the pianists did not voluntarily move the fingers. The enhancement of motor skill through passive somatosensory training using the exoskeleton was more pronounced in adolescent pianists than adult pianists. These preliminary results implicate a sensitive period of neuroplasticity of the somatosensory-motor system of trained pianists, which emphasizes the importance of somatosensory-motor training in professional music education during adolescence.

The plyometric activity as a conditioning to enhance strength and precision of the finger movements in pianists

Stability of timing and force production in repetitive movements characterizes skillful motor behaviors such as surgery and playing musical instruments. However, even trained individuals such as musicians undergo further extensive training for the improvement of these skills. Previous studies that investigated the lower extremity movements such as jumping and sprinting demonstrated enhancement of the maximum force and rate of force development immediately after the plyometric exercises. However, it remains unknown whether the plyometric exercises enhance the stability of timing and force production of the dexterous finger movements in trained individuals. Here we address this issue by examining the effects of plyometric exercise specialized for finger movements on piano performance. We compared the training-related changes in the piano-key motion and several physiological features of the finger muscles (e.g., electromyography, rate of force development, and muscle temperature) by well-trained pianists. The conditioning demonstrated a decrease of the variation in timing and velocity of successive keystrokes, along with a concomitant increase in the rate of force development of the four fingers, but not the thumb, although there was no change in the finger muscular activities through the activity. By contrast, such a conditioning effect was not evident following a conventional repetitive piano practice. In addition, a significant increase in the forearm muscle temperature was observed specifically through performing the plyometric exercise with the fingers, implying its association with improved performance. These results indicate effectiveness of the plyometric exercises for improvement of strength, precision, and physiological efficiency of the finger movements even in expert pianists, which implicates that ways of practicing play a key role in enhancing experts' expertise.

Noncontact and High-Precision Sensing System for Piano Keys Identified Fingerprints of Virtuosity

Dexterous tool use is typically characterized by fast and precise motions performed by multiple fingers. One representative task is piano playing, which involves fast performance of a sequence of complex motions with high spatiotemporal precision. However, for several decades, a lack of contactless sensing technologies that are capable of precision measurement of piano key motions has been a bottleneck for unveiling how such an outstanding skill is cultivated. Here, we developed a novel sensing system that can record the vertical position of all piano keys with a time resolution of 1 ms and a spatial resolution of 0.01 mm in a noncontact manner. Using this system, we recorded the piano key motions while 49 pianists played a complex sequence of tones that required both individuated and coordinated finger movements to be performed as fast and accurately as possible. Penalized regression using various feature variables of the key motions identified distinct characteristics of the key-depressing and key-releasing motions in relation to the speed and accuracy of the performance. For the maximum rate of the keystrokes, individual differences across the pianists were associated with the peak key descending velocity, the key depression duration, and key-lift timing. For the timing error of the keystrokes, the interindividual differences were associated with the peak ascending velocity of the key and the inter-strike variability of both the peak key descending velocity and the key depression duration. These results highlight the importance of dexterous control of the vertical motions of the keys for fast and accurate piano performance.

Back to feedback: aberrant sensorimotor control in music performance under pressure

Precisely timed production of dexterous actions is often destabilized in anxiogenic situations. Previous studies demonstrated that cognitive functions such as attention and working memory as well as autonomic nervous functions are susceptible to psychological stress in skillful performance while playing sports or musical instruments. However, it is not known whether the degradation of sensorimotor functions underlies such a compromise of skillful performance due to psychophysiological distress. Here, we addressed this issue through a set of behavioral experiments. After artificially delaying the timing of tone production while playing the piano, the local tempo was abnormally disrupted only under pressure. The results suggest that psychological stress degraded the temporal stability of movement control due to an abnormal increase in feedback gain. A learning experiment further demonstrated that the temporal instability of auditory-motor control under pressure was alleviated after practicing piano while ignoring delayed auditory feedback but not after practicing while compensating for the delayed feedback. Together, these findings suggest an abnormal transition from feedforward to feedback control in expert piano performance with psychological stress, which can be mitigated through specialized sensorimotor training that involves piano practice while volitionally ignoring the artificially delayed provision of auditory feedback.

In order to establish if the degradation of sensorimotor functions underlies the stress-associated disruption of skilful performance, Furuya et al examined participants undergoing a piano playing task under stress. Their data suggests the occurrence of a stress-induced transition from feedforward to feedback control, which can be mitigated through sensorimotor training involving piano practice while volitionally ignoring the artificially delayed provision of auditory feedback.

Enhancement of loudness discrimination acuity for self-generated sound is independent of musical experience

Musicians tend to have better auditory and motor performance than non-musicians because of their extensive musical experience. In a previous study, we established that loudness discrimination acuity is enhanced when sound is produced by a precise force generation task. In this study, we compared the enhancement effect between experienced pianists and non-musicians. Without the force generation task, loudness discrimination acuity was better in pianists than non-musicians in the condition. However, the force generation task enhanced loudness discrimination acuity similarly in both pianists and non-musicians. The reaction time was also reduced with the force control task, but only in the non-musician group. The results suggest that the enhancement of loudness discrimination acuity with the precise force generation task is independent of musical experience and is, therefore, a fundamental function in auditory-motor interaction.