Sensori-Motor Learning with Movement Sonification: Perspectives from Recent Interdisciplinary Studies

Developing Children's Motor Skills by Having Fun With Orff's Approach

Environmental factors have a clear effect on children's motor development and can be manipulated; while biological factors are also powerfully influential, they are more resistant to intervention. Especially in restrictive environments (e.g., the recent pandemic-related shutdown), there can be increased importance to simple, fun, at-home, indoor programs. In this study, we investigated the effects on motor skill development of a movement education program based on the Orff approach, which has long been used in music education. This was an experimental research design involving 78 children, aged 6-7 years old, in two groups (Control Group, n = 39; Experimental Group, n = 39). We used a simple randomization method to determine the groups. We measured the children's skill levels with the Test of Gross Motor Development-2 (TGMD-2), and we analyzed data using 2 × 2 repeated-measures analyses of variance (ANOVA). Our results revealed significantly greater improvement in the children's scores on object control (p < .001), locomotor skills (p < .001), and total TGMD-2 (p < .001) in the experimental group than in the control group. Thus, programs designed with music and rhythm can help ensure that children have fun and are better motivated to participate in activities that have a positive effect on their motor skills.

Online reach adjustments induced by real-time movement sonification

Movement sonification can improve motor control in both healthy subjects (e.g., learning or refining a sport skill) and those with sensorimotor deficits (e.g., stroke patients and deafferented individuals). It is not known whether improved motor control and learning from movement sonification are driven by feedback-based real-time ("online") trajectory adjustments, adjustments to internal models over multiple trials, or both. We searched for evidence of online trajectory adjustments (muscle twitches) in response to movement sonification feedback by comparing the kinematics and error of reaches made with online (i.e., real-time) and terminal sonification feedback. We found that reaches made with online feedback were significantly more jerky than reaches made with terminal feedback, indicating increased muscle twitching (i.e., online trajectory adjustment). Using a between-subject design, we found that online feedback was associated with improved motor learning of a reach path and target over terminal feedback; however, using a within-subjects design, we found that switching participants who had learned with online sonification feedback to terminal feedback was associated with a decrease in error. Thus, our results suggest that, with our task and sonification, movement sonification leads to online trajectory adjustments which improve internal models over multiple trials, but which themselves are not helpful online corrections.

Perceptual coupling in human dyads: Kinematics does not affect interpersonal synchronization

The minimal, essential condition for individuals to interact is that they exchange information via at least one sensory channel. Once informational coupling is established, it enables basic forms of coordinated behavior to spontaneously emerge from the interaction. Our previous study revealed different coordination dynamics in dyads engaged in a joint finger-tapping task based on visual versus auditory coupling. This observation led us to propose the 'modality-dependent hypothesis', which posits that coordination dynamics are influenced by the sensory modality mediating informational coupling. However, recognizing that different modalities have inherent differences in accessing spatiotemporal features of perceived movement, we formulated the alternative 'kinematic hypothesis'. This hypothesis posits that differences in dynamics would vanish given equivalent kinematic information across modalities. The study involved forty (N = 40) participants, grouped into twenty (N = 20) dyads, who engaged in a joint finger-tapping task. This task was conducted under varying conditions of visual and auditory coupling, with manipulations in the access to kinematic information, categorized as discrete and continuous. Contrary to our initial predictions, the results strongly supported the 'modality-dependent hypothesis'. We observed that visual and auditory coupling consistently yielded distinct attractor dynamics, regardless of the access to kinematic information. Furthermore, all conditions of auditory coupling resulted in higher levels of synchronization than their visual counterparts. These findings suggest that the differences in interpersonal synchronization are predominantly influenced by the sensory modality, rather than the continuity of kinematic information. Our study highlights the significance of sensorimotor interactions in interpersonal synchronization and addresses the potential of sonification strategies in supporting motor training and rehabilitation.

The Sound of Memory: Investigating Music Therapy's Cognitive Benefits in Patients with Dementia-A Network Meta-Analysis of Randomized Controlled Trials

Numerous previous studies have shown the effectiveness of music therapy in enhancing cognitive functions in patients with dementia. Despite this, robust evidence in this field, especially concerning the comparison of different music therapy types, is lacking. Therefore, randomized controlled trials (RCTs) focusing on music therapy and cognitive functions in dementia patients, termed by "music" AND "dementia" OR "Alzheimer's disease" AND "cognitive", were identified from primary electronic databases to conduct this network meta-analysis (NMA). The primary outcome focused on the impact on cognitive functions, and the secondary outcome was the comparison of dropout rates between the intervention groups and the usual care control groups. Standardized mean difference (SMD) values and the corresponding 95% confidence intervals (CIs) were computed for effect evaluation. This study protocol has been registered in IPLASY (INPLASY202430082). A total of 14 RCTs with 1056 participants were enrolled, examining interventions including Active Music Therapy (AMT), Active Music Therapy with Singing (AMT + Sing), Rhythmic Music Therapy (RMT), Listening to Music (LtM), and Singing (Sing). The results indicated that RMT, AMT + Sing, and AMT all significantly improve cognitive functions in dementia patients, of which the SMD were 0.76 (95% CI = 0.32-1.21), 0.79 (95% CI = 0.03-1.49), and 0.57 (0.18-0.96), respectively. Compared with the control group (usual care), no music therapy type was associated with an increased dropout risk. In conclusion, music therapy can improve cognitive functions in patients with dementia without increasing the risk of dropout, particularly RMT, AMT + Sing, and AMT.

Movement Sonification Techniques to Improve Balance in Parkinson's Disease: A Pilot Randomized Controlled Trial

BACKGROUND

Movement sonification has been recently introduced into the field of neuromotor rehabilitation alongside Neurologic Music Therapy and music-based interventions. This study introduces the use of musical auditory cues encompassing the melodic-harmonic aspect of music.

METHODS

Nineteen patients with Parkinson's disease were randomly assigned to the experimental (n = 10) and control (n = 9) groups and underwent thrice-weekly sessions of the same gait training program, with or without sonification. Functional and motor parameters, as well as fatigue, quality of life, and the impact of intervention on patients' well-being, were assessed at baseline (PRE), the end of treatment (POST), and at follow-up (FU). Between-group differences were assessed for each outcome measure using linear mixed-effects models. The outcome measure was entered as the dependent variable, group and time as fixed effects, and time by group as the interaction effect.

RESULTS

Mini BESTest and Dynamic Gait Index scores significantly improved in the experimental group (p = 0.01 and p = 0.03, respectively) from PRE to FU, demonstrating a significant impact of the sonification treatment on balance. No other significant differences were observed in the outcome measures.

CONCLUSIONS

Larger sample sizes are needed to confirm the effectiveness of sonification approaches in Parkinson's disease, as well as in other neurological disorders.

Effect of sonification types in upper-limb movement: a quantitative and qualitative study in hemiparetic and healthy participants

BACKGROUND

Movement sonification, the use of real-time auditory feedback linked to movement parameters, have been proposed to support rehabilitation. Nevertheless, if promising results have been reported, the effect of the type of sound used has not been studied systematically. The aim of this study was to investigate in a single session the effect of different types of sonification both quantitatively and qualitatively on patients with acquired brain lesions and healthy participants.

METHODS

An experimental setup enabling arm sonification was developed using three different categories of sonification (direct sound modulation, musical interaction, and soundscape). Simple moving forward movements performed while sliding on a table with both arms were investigated with all participants. Quantitative analysis on the movement timing were performed considering various parameters (sound condition, affected arm and dominance, sonification categories). Qualitative analysis of semi-structured interviews were also conducted, as well as neuropsychological evaluation of music perception.

RESULTS

For both the patient and healthy groups (15 participants each), average duration for performing the arm movement is significantly longer with sonification compared to the no-sound condition (p < 0.001). Qualitative analysis of semi-structured interviews revealed different aspects of motivational and affective aspects of sonification. Most participants of both groups preferred to complete the task with sound (29 of 30 participants), and described the experience as playful (22 of 30 participants). More precisely, the soundscape (nature sounds) was the most constantly preferred (selected first by 14 of 30 participants).

CONCLUSION

Overall, our results confirm that the sonification has an effect on the temporal execution of the movement during a single-session. Globally, sonification is welcomed by the participants, and we found convergent and differentiated appreciations of the different sonification types.

Integration of proprioception in upper limb prostheses through non-invasive strategies: a review

Proprioception plays a key role in moving our body dexterously and effortlessly. Nevertheless, the majority of investigations evaluating the benefits of providing supplemental feedback to prosthetics users focus on delivering touch restitution. These studies evaluate the influence of touch sensation in an attempt to improve the controllability of current robotic devices. Contrarily, investigations evaluating the capabilities of proprioceptive supplemental feedback have yet to be comprehensively analyzed to the same extent, marking a major gap in knowledge within the current research climate. The non-invasive strategies employed so far to restitute proprioception are reviewed in this work. In the absence of a clearly superior strategy, approaches employing vibrotactile, electrotactile and skin-stretch stimulation achieved better and more consistent results, considering both kinesthetic and grip force information, compared with other strategies or any incidental feedback. Although emulating the richness of the physiological sensory return through artificial feedback is the primary hurdle, measuring its effects to eventually support the integration of cumbersome and energy intensive hardware into commercial prosthetic devices could represent an even greater challenge. Thus, we analyze the strengths and limitations of previous studies and discuss the possible benefits of coupling objective measures, like neurophysiological parameters, as well as measures of prosthesis embodiment and cognitive load with behavioral measures of performance. Such insights aim to provide additional and collateral outcomes to be considered in the experimental design of future investigations of proprioception restitution that could, in the end, allow researchers to gain a more detailed understanding of possibly similar behavioral results and, thus, support one strategy over another.

Putting music to trial: Consensus on key methodological challenges investigating music-based rehabilitation

Major advances in music neuroscience have fueled a growing interest in music-based neurological rehabilitation among researchers and clinicians. Musical activities are excellently suited to be adapted for clinical practice because of their multisensory nature, their demands on cognitive, language, and motor functions, and music's ability to induce emotions and regulate mood. However, the overall quality of music-based rehabilitation research remains low to moderate for most populations and outcomes. In this consensus article, expert panelists who participated in the Neuroscience and Music VII conference in June 2021 address methodological challenges relevant to music-based rehabilitation research. The article aims to provide guidance on challenges related to treatment, outcomes, research designs, and implementation in music-based rehabilitation research. The article addresses how to define music-based rehabilitation, select appropriate control interventions and outcomes, incorporate technology, and consider individual differences, among other challenges. The article highlights the value of the framework for the development and evaluation of complex interventions for music-based rehabilitation research and the need for stronger methodological rigor to allow the widespread implementation of music-based rehabilitation into regular clinical practice.

Spatial Memory and Blindness: The Role of Visual Loss on the Exploration and Memorization of Spatialized Sounds

Spatial memory relies on encoding, storing, and retrieval of knowledge about objects’ positions in their surrounding environment. Blind people have to rely on sensory modalities other than vision to memorize items that are spatially displaced, however, to date, very little is known about the influence of early visual deprivation on a person’s ability to remember and process sound locations. To fill this gap, we tested sighted and congenitally blind adults and adolescents in an audio-spatial memory task inspired by the classical card game “Memory.” In this research, subjects (blind, n = 12; sighted, n = 12) had to find pairs among sounds (i.e., animal calls) displaced on an audio-tactile device composed of loudspeakers covered by tactile sensors. To accomplish this task, participants had to remember the spatialized sounds’ position and develop a proper mental spatial representation of their locations. The test was divided into two experimental conditions of increasing difficulty dependent on the number of sounds to be remembered (8 vs. 24). Results showed that sighted participants outperformed blind participants in both conditions. Findings were discussed considering the crucial role of visual experience in properly manipulating auditory spatial representations, particularly in relation to the ability to explore complex acoustic configurations.

Collectively playable wearable music: Practice-situated approaches to participatory relational inquiry

We present two practice-situated participatory investigations using networked wearable sensors to develop movement-responsive collectively playable musical instruments: a series of four collocated workshops for expert dancers and a distance learning course in which students use wearable technology to enhance embodied learning and feelings of connectedness telematically. We reflect on our exploration of techniques for structuring ensemble improvisations augmented with bespoke digital musical instruments using aggregate statistical measures, such as variance of participants' physical orientation as an index of group intention. Participatory design exchanges top-down design methodologies with bottom-up approaches consulting actors' interests. We follow this approach by evolving our instruments through abductive experiments and trial-and-error tinkering, without strong theories, methods, or models, using elementary signal processing techniques that are meaningfully understood and modified by participants. Our experiences suggest useful scaffolding techniques for educational transdisciplinary research-creation communities seeking to explore relational ensemble dynamics in telematic and/or physically collocated settings using accessible wearable technologies. Through creative inquiry and participation, technical objects can become bearers of sense and meaning rather than instating mystifying or alienating relations for the participants.

Effects of pitch and musical sounds on body-representations when moving with sound

The effects of music on bodily movement and feelings, such as when people are dancing or engaged in physical activity, are well-documented—people may move in response to the sound cues, feel powerful, less tired. How sounds and bodily movements relate to create such effects? Here we deconstruct the problem and investigate how different auditory features affect people’s body-representation and feelings even when paired with the same movement. In three experiments, participants executed a simple arm raise synchronised with changing pitch in simple tones (Experiment 1), rich musical sounds (Experiment 2) and within different frequency ranges (Experiment 3), while we recorded indirect and direct measures on their movement, body-representations and feelings. Changes in pitch influenced people’s general emotional state as well as the various bodily dimensions investigated—movement, proprioceptive awareness and feelings about one’s body and movement. Adding harmonic content amplified the differences between ascending and descending sounds, while shifting the absolute frequency range had a general effect on movement amplitude, bodily feelings and emotional state. These results provide new insights in the role of auditory and musical features in dance and exercise, and have implications for the design of sound-based applications supporting movement expression, physical activity, or rehabilitation.

Hand rehabilitation with sonification techniques in the subacute stage of stroke

After a stroke event, most survivors suffer from arm paresis, poor motor control and other disabilities that make activities of daily living difficult, severely affecting quality of life and personal independence. This randomized controlled trial aimed at evaluating the efficacy of a music-based sonification approach on upper limbs motor functions, quality of life and pain perceived during rehabilitation. The study involved 65 subacute stroke individuals during inpatient rehabilitation allocated into 2 groups which underwent usual care dayweek) respectively of standard upper extremity motor rehabilitation or upper extremity treatment with sonification techniques. The Fugl-Meyer Upper Extremity Scale, Box and Block Test and the Modified Ashworth Scale were used to perform motor assessment and the McGill Quality of Life-it and the Numerical Pain Rating Scale to assess quality of life and pain. The assessment was performed at baseline, after 2 weeks, at the end of treatment and at follow-up (1 month after the end of treatment). Total scores of the Fugl-Meyer Upper Extremity Scale (primary outcome measure) and hand and wrist sub scores, manual dexterity scores of the affected and unaffected limb in the Box and Block Test, pain scores of the Numerical Pain Rating Scale (secondary outcomes measures) significantly improved in the sonification group compared to the standard of care group (time*group interaction < 0.05). Our findings suggest that music-based sonification sessions can be considered an effective standardized intervention for the upper limb in subacute stroke rehabilitation.

Audio Feedback for Device-Supported Balance Training: Parameter Mapping and Influencing Factors

Recent studies suggest that real-time auditory feedback is an effective method to facilitate motor learning. The evaluation of the parameter mapping (sound-to-movement mapping) is a crucial, yet frequently neglected step in the development of audio feedback. We therefore conducted two experiments to evaluate audio parameters with target finding exercises designed for balance training. In the first experiment with ten participants, five different audio parameters were evaluated on the X-axis (mediolateral movement). Following that, in a larger experiment with twenty participants in a two-dimensional plane (mediolateral and anterior-posterior movement), a basic and synthetic audio model was compared to a more complex audio model with musical characteristics. Participants were able to orient themselves and find the targets with the audio models. In the one-dimensional condition of experiment one, percussion sounds and synthetic sound wavering were the overall most effective audio parameters. In experiment two, the synthetic model was more effective and better evaluated by the participants. In general, basic sounds were more helpful than complex (musical) sound models. Musical abilities and age were correlated with certain exercise scores. Audio feedback is a promising approach for balance training and should be evaluated with patients. Preliminary evaluation of the respective parameter mapping is highly advisable.

Cellists' sound quality is shaped by their primary postural behavior

During the last 20 years, the role of musicians’ body movements has emerged as a central question in instrument practice: Why do musicians make so many postural movements, for instance, with their torsos and heads, while playing musical instruments? The musical significance of such ancillary gestures is still an enigma and therefore remains a major pedagogical challenge, since one does not know if these movements should be considered essential embodied skills that improve musical expressivity. Although previous studies established clear connections between musicians’ body movements and musical structures (particularly for clarinet, piano or violin performances), no evidence of direct relationships between body movements and the quality of the produced timbre has ever been found. In this study, focusing on the area of bowed-string instruments, we address the problem by showing that cellists use a set of primary postural directions to develop fluid kinematic bow features (velocity, acceleration) that prevent the production of poor quality (i.e., harsh, shrill, whistling) sounds. By comparing the body-related angles between normal and posturally constrained playing situations, our results reveal that the chest rotation and vertical inclination made by cellists act as coordinative support for the kinematics of the bowing gesture. These findings support the experimental works of Alexander, especially those that showed the role of head movements with respect to the upper torso (the so-called primary control) in ensuring the smooth transmission of fine motor control in musicians all the way to the produced sound. More generally, our research highlights the importance of focusing on this fundamental postural sense to improve the quality of human activities across different domains (music, dance, sports, rehabilitation, working positions, etc.).

Modulation of ellipses drawing by sonification

Most studies on the regulation of speed and trajectory during ellipse drawing have used visual feedback. We used online auditory feedback (sonification) to induce implicit movement changes independently from vision. The sound was produced by filtering a pink noise with a band-pass filter proportional to movement speed. The first experiment was performed in 2D. Healthy participants were asked to repetitively draw ellipses during 45 s trials whilst maintaining a constant sonification pattern (involving pitch variations during the cycle). Perturbations were produced by modifying the slope of the mapping without informing the participants. All participants adapted spontaneously their speed: they went faster if the slope decreased and slower if it increased. Higher velocities were achieved by increasing both the frequency of the movements and the perimeter of the ellipses, but slower velocities were achieved mainly by decreasing the perimeter of the ellipses. The shape and the orientation of the ellipses were not significantly altered. The analysis of the speed-curvature power law parameters showed consistent modulations of the speed gain factor, while the exponent remained stable. The second experiment was performed in 3D and showed similar results, except that the main orientation of the ellipse also varied with the changes in speed. In conclusion, this study demonstrated implicit modulation of movement speed by sonification and robust stability of the ellipse geometry. Participants appeared to limit the decrease in movement frequency during slowing down to maintain a rhythmic and not discrete motor regimen.

The Influence of Auditory Cues on Bodily and Movement Perception

The sounds that result from our movement and that mark the outcome of our actions typically convey useful information concerning the state of our body and its movement, as well as providing pertinent information about the stimuli with which we are interacting. Here we review the rapidly growing literature investigating the influence of non-veridical auditory cues (i.e., inaccurate in terms of their context, timing, and/or spectral distribution) on multisensory body and action perception, and on motor behavior. Inaccurate auditory cues provide a unique opportunity to study cross-modal processes: the ability to detect the impact of each sense when they provide a slightly different message is greater. Additionally, given that similar cross-modal processes likely occur regardless of the accuracy or inaccuracy of sensory input, studying incongruent interactions are likely to also help us predict interactions between congruent inputs. The available research convincingly demonstrates that perceptions of the body, of movement, and of surface contact features (e.g., roughness) are influenced by the addition of non-veridical auditory cues. Moreover, auditory cues impact both motor behavior and emotional valence, the latter showing that sounds that are highly incongruent with the performed movement induce feelings of unpleasantness (perhaps associated with lower processing fluency). Such findings are relevant to the design of auditory cues associated with product interaction, and the use of auditory cues in sport performance and therapeutic situations given the impact on motor behavior.

Auditory Coding of Reaching Space

Reaching movements are usually initiated by visual events and controlled visually and kinesthetically. Lately, studies have focused on the possible benefit of auditory information for localization tasks, and also for movement control. This explorative study aimed to investigate if it is possible to code reaching space purely by auditory information. Therefore, the precision of reaching movements to merely acoustically coded target positions was analyzed. We studied the efficacy of acoustically effect-based and of additional acoustically performance-based instruction and feedback and the role of visual movement control. Twenty-four participants executed reaching movements to merely acoustically presented, invisible target positions in three mutually perpendicular planes in front of them. Effector-endpoint trajectories were tracked using inertial sensors. Kinematic data regarding the three spatial dimensions and the movement velocity were sonified. Thus, acoustic instruction and real-time feedback of the movement trajectories and the target position of the hand were provided. The subjects were able to align their reaching movements to the merely acoustically instructed targets. Reaching space can be coded merely acoustically, additional visual movement control does not enhance reaching performance. On the basis of these results, a remarkable benefit of kinematic movement acoustics for the neuromotor rehabilitation of everyday motor skills can be assumed.

Sound matters: The impact of auditory deprivation on movement precision in rowing

Purpose of the study was to quantify the importance of auditory feedback for movement precision in elevated rowing task difficulty with elite athletes under normal and masked hearing conditions. It was hypothesized that rowing with masked hearing would reduce the precision of movement, particularly at the non-usual/less-preferred stroke frequencies (SF). Self-reported questionnaires helped to understand the difficulty of the task. Twenty rowers completed 2 × 1000 m-distance-blocks, each separated into 4 × 250 m, with increasing SF 18, 20, 22 24 strokes/minute once with normal and once with masked hearing. Precision was determined as the deviation between the SF target and the SF actually performed (DSF). Athletes' subjective perception was requested before and after the experiment. A 2 (hearing condition) × 4 (SF 18, 20, 22, 24) repeated measures ANOVA showed systematically larger DSF during masked hearing for all SFs compared to the DSF in the normal hearing condition (p < .01). Further, the highest DSFs were found for SF 18 and 24 in both hearing conditions (no interaction effect). The athletes' perception of the relevance of natural movement sounds for their rowing performance changed when evaluated before and after the experiment. Rowing without hearing was evaluated as mentally more demanding than physically. The results confirmed our initial assumptions and showed the relevance of natural auditory information for movement precision in rowing practice, even at a high level of expertise.

Audio-Motor Training Enhances Auditory and Proprioceptive Functions in the Blind Adult

Several reports indicate that spatial perception in blind individuals can be impaired as the lack of visual experience severely affects the development of multisensory spatial correspondences. Despite the growing interest in the development of technological devices to support blind people in their daily lives, very few studies have assessed the benefit of interventions that help to refine sensorimotor perception. In the present study, we directly investigated the impact of a short audio-motor training on auditory and proprioceptive spatial perception in blind individuals. Our findings indicate that auditory and proprioceptive spatial capabilities can be enhanced through interventions designed to foster sensorimotor perception in the form of audio-motor correspondences, demonstrating the importance of the early introduction of sensorimotor training in therapeutic intervention for blind individuals.

Enriching footsteps sounds in gait rehabilitation in chronic stroke patients: a pilot study

In the context of neurorehabilitation, sound is being increasingly applied for facilitating sensorimotor learning. In this study, we aimed to test the potential value of auditory stimulation for improving gait in chronic stroke patients by inducing alterations of the frequency spectra of walking sounds via a sound system that selectively amplifies and equalizes the signal in order to produce distorted auditory feedback. Twenty-two patients with lower extremity paresis were exposed to real-time alterations of their footstep sounds while walking. Changes in body perception, emotion, and gait were quantified. Our results suggest that by altering footsteps sounds, several gait parameters can be modified in terms of left-right foot asymmetry. We observed that augmenting low-frequency bands or amplifying the natural walking sounds led to a reduction in the asymmetry index of stance and stride times, whereas it inverted the asymmetry pattern in heel-ground exerted force. By contrast, augmenting high-frequency bands led to opposite results. These gait changes might be related to updating of internal forward models, signaling the need for adjustment of the motor system to reduce the perceived discrepancies between predicted-actual sensory feedbacks. Our findings may have the potential to enhance gait awareness in stroke patients and other clinical conditions, supporting gait rehabilitation.

The SonicHand Protocol for Rehabilitation of Hand Motor Function: A Validation and Feasibility Study

Musical sonification therapy is a new technique that can reinforce conventional rehabilitation treatments by increasing therapy intensity and engagement through challenging and motivating exercises. The aim of this paper is to evaluate the feasibility and validity of the SonicHand protocol, a new training and assessment method for the rehabilitation of hand function. The study was conducted in 15 healthy individuals and 15 stroke patients. The feasibility of implementation of the training protocol was tested in stroke patients only, who practiced a series of exercises concurrently to music sequences produced by specific movements. The assessment protocol evaluated hand motor performance during pronation/supination, wrist horizontal flexion/extension, and hand grasp without sonification. From hand position data, 15 quantitative parameters were computed evaluating mean velocity, movement smoothness, and angular excursions of hand/fingers. We validated this assessment in terms of its ability to discriminate between patients and healthy subjects, test-retest reliability and concurrent validity with the upper limb section of the Fugl-Meyer scale (FM), the functional independence measure (FIM), and the Box and Block Test (BBT). All patients showed a good understanding of the assigned tasks and were able to correctly execute the proposed training protocol, confirming its feasibility. A moderate-to-excellent intraclass correlation coefficient was found in 8/15 computed parameters. The moderate-to-strong correlation was found between the measured parameters and the clinical scales. The SonicHand training protocol is feasible and the assessment protocol showed good to excellent between-group discrimination ability, reliability, and concurrent validity, thus enabling the implementation of new personalized and motivating training programs employing sonification for the rehabilitation of hand function.

A Review on the Relationship Between Sound and Movement in Sports and Rehabilitation

The role of auditory information on perceptual-motor processes has gained increased interest in sports and psychology research in recent years. Numerous neurobiological and behavioral studies have demonstrated the close interaction between auditory and motor areas of the brain, and the importance of auditory information for movement execution, control, and learning. In applied research, artificially produced acoustic information and real-time auditory information have been implemented in sports and rehabilitation to improve motor performance in athletes, healthy individuals, and patients affected by neurological or movement disorders. However, this research is scattered both across time and scientific disciplines. The aim of this paper is to provide an overview about the interaction between movement and sound and review the current literature regarding the effect of natural movement sounds, movement sonification, and rhythmic auditory information in sports and motor rehabilitation. The focus here is threefold: firstly, we provide an overview of empirical studies using natural movement sounds and movement sonification in sports. Secondly, we review recent clinical and applied studies using rhythmic auditory information and sonification in rehabilitation, addressing in particular studies on Parkinson's disease and stroke. Thirdly, we summarize current evidence regarding the cognitive mechanisms and neural correlates underlying the processing of auditory information during movement execution and its mental representation. The current state of knowledge here reviewed provides evidence of the feasibility and effectiveness of the application of auditory information to improve movement execution, control, and (re)learning in sports and motor rehabilitation. Findings also corroborate the critical role of auditory information in auditory-motor coupling during motor (re)learning and performance, suggesting that this area of clinical and applied research has a large potential that is yet to be fully explored.

Orff-Based Music Training Enhances Children's Manual Dexterity and Bimanual Coordination

How music training and expertise influence non-musical abilities is a widely researched topic. Most studies focus on the differences between adult professional musicians and non-musicians, or examine the effects of intensive instrumental training in childhood. However, the impact of music programs developed in regular school contexts for children from low-income communities is poorly explored. We conducted a longitudinal training study in such communities to examine if collective (Orff-based) music training enhances fine motor abilities, when compared to a homologous training program in sports (basketball), and to no specific training. The training programs in music and sports had the same duration, 24 weeks, and were homologous in structure. A pre-test, training, post-test and follow-up design was adopted. Children attending the 3rd grade (n = 74, 40 girls; mean age 8.31 years) were pseudorandomly divided into three groups, music, sports and control that were matched on demographic and intellectual characteristics. Fine motor abilities were assessed with the Purdue pegboard test (eye-hand coordination and motor speed, both subsumed under manual dexterity, and bimanual coordination) and with the Grooved pegboard (manipulative dexterity) test. All groups improved in manipulative dexterity that was not affected by type of training. On bimanual coordination and manual dexterity, however, a robust and stable advantage of music training emerged. At the end of training (post-test), children from the music group significantly outperformed children from the sports and control groups, an advantage that persisted at follow-up 4 months after training at the start of the following school year. Also, at follow-up none of the children from the music group were performing below the 20th percentile in the Purdue pegboard subtests and more than half were performing at the high end level (>80th percentile). Children from the sports group also improved significantly from pre- to post-test but their performance was not significantly different from that of the control group. These results show that an affordable, collective-based music practice impacts positively on fine-motor abilities, a finding that is relevant for a better understanding of the impact of music in childhood development, and that may have implications for education at the primary grade.

Motion-Sound Mapping through Interaction

Technologies for sensing movement are expanding toward everyday use in virtual reality, gaming, and artistic practices. In this context, there is a need for methodologies to help designers and users create meaningful movement experiences. This article discusses a user-centered approach for the design of interactive auditory feedback using interactive machine learning. We discuss Mapping through Interaction, a method for crafting sonic interactions from corporeal demonstrations of embodied associations between motion and sound. It uses an interactive machine learning approach to build the mapping from user demonstrations, emphasizing an iterative design process that integrates acted and interactive experiences of the relationships between movement and sound. We examine Gaussian Mixture Regression and Hidden Markov Regression for continuous movement recognition and real-time sound parameter generation. We illustrate and evaluate this approach through an application in which novice users can create interactive sound feedback based on coproduced gestures and vocalizations. Results indicate that Gaussian Mixture Regression and Hidden Markov Regression can efficiently learn complex motion-sound mappings from few examples.

Continuous Auditory Feedback of Eye Movements: An Exploratory Study toward Improving Oculomotor Control

As eye movements are mostly automatic and overtly generated to attain visual goals, individuals have a poor metacognitive knowledge of their own eye movements. We present an exploratory study on the effects of real-time continuous auditory feedback generated by eye movements. We considered both a tracking task and a production task where smooth pursuit eye movements (SPEM) can be endogenously generated. In particular, we used a visual paradigm which enables to generate and control SPEM in the absence of a moving visual target. We investigated whether real-time auditory feedback of eye movement dynamics might improve learning in both tasks, through a training protocol over 8 days. The results indicate that real-time sonification of eye movements can actually modify the oculomotor behavior, and reinforce intrinsic oculomotor perception. Nevertheless, large inter-individual differences were observed preventing us from reaching a strong conclusion on sensorimotor learning improvements.