The myokinetic control interface: tracking implanted magnets as a means for prosthetic control

Neuromuscular implants: Interfacing with skeletal muscle for improved clinical translation of prosthetic limbs

After an amputation, advanced prosthetic limbs can be used to interface with the nervous system and restore motor function. Despite numerous breakthroughs in the field, many of the recent research advancements have not been widely integrated into clinical practice. This review highlights recent innovations in neuromuscular implants-specifically those that interface with skeletal muscle-which could improve the clinical translation of prosthetic technologies. Skeletal muscle provides a physiologic gateway to harness and amplify signals from the nervous system. Recent surgical advancements in muscle reinnervation surgeries leverage the "bio-amplification" capabilities of muscle, enabling more intuitive control over a greater number of degrees of freedom in prosthetic limbs than previously achieved. We anticipate that state-of-the-art implantable neuromuscular interfaces that integrate well with skeletal muscle and novel surgical interventions will provide a long-term solution for controlling advanced prostheses. Flexible electrodes are expected to play a crucial role in reducing foreign body responses and improving the longevity of the interface. Additionally, innovations in device miniaturization and ongoing exploration of shape memory polymers could simplify surgical procedures for implanting such interfaces. Once implanted, wireless strategies for powering and transferring data from the interface can eliminate bulky external wires, reduce infection risk, and enhance day-to-day usability. By outlining the current limitations of neuromuscular interfaces along with potential future directions, this review aims to guide continued research efforts and future collaborations between engineers and specialists in the field of neuromuscular and musculoskeletal medicine.

To What Extent Implanting Single vs Pairs of Magnets Per Muscle Affect the Localization Accuracy of the Myokinetic Control Interface? Evidence From a Simulated Environment

OBJECTIVE

We recently proposed a new concept of human-machine interface to control hand prostheses which we dubbed the myokinetic control interface. Such interface detects muscle displacement during contraction by localizing permanent magnets implanted in the residual muscles. So far, we evaluated the feasibility of implanting one magnet per muscle and monitoring its displacement relative to its initial position. However, multiple magnets could actually be implanted in each muscle, as using their relative distance as a measure of muscle contraction could improve the system robustness against environmental disturbances.

METHODS

Here, we simulated the implant of pairs of magnets in each muscle and we compared the localization accuracy of such system with the one magnet per muscle approach, considering first a planar and then an anatomically appropriate configuration. Such comparison was also performed when simulating different grades of mechanical disturbances applied to the system (i.e., shift of the sensor grid).

RESULTS

We found that implanting one magnet per muscle always led to lower localization errors under ideal conditions (i.e., no external disturbances). Differently, when mechanical disturbances were applied, magnet pairs outperformed the single magnet approach, confirming that differential measurements are able to reject common mode disturbances.

CONCLUSION

We identified important factors affecting the choice of the number of magnets to implant in a muscle.

SIGNIFICANCE

Our results provide important guidelines for the design of disturbance rejection strategies and for the development of the myokinetic control interface, as well as for a whole range of biomedical applications involving magnetic tracking.

Restoration of sensory information via bionic hands

Individuals who have lost the use of their hands because of amputation or spinal cord injury can use prosthetic hands to restore their independence. A dexterous prosthesis requires the acquisition of control signals that drive the movements of the robotic hand, and the transmission of sensory signals to convey information to the user about the consequences of these movements. In this Review, we describe non-invasive and invasive technologies for conveying artificial sensory feedback through bionic hands, and evaluate the technologies' long-term prospects.

Untethered muscle tracking using magnetomicrometry

Muscle tissue drives nearly all movement in the animal kingdom, providing power, mobility, and dexterity. Technologies for measuring muscle tissue motion, such as sonomicrometry, fluoromicrometry, and ultrasound, have significantly advanced our understanding of biomechanics. Yet, the field lacks the ability to monitor muscle tissue motion for animal behavior outside the lab. Towards addressing this issue, we previously introduced magnetomicrometry, a method that uses magnetic beads to wirelessly monitor muscle tissue length changes, and we validated magnetomicrometry via tightly-controlled in situ testing. In this study we validate the accuracy of magnetomicrometry against fluoromicrometry during untethered running in an in vivo turkey model. We demonstrate real-time muscle tissue length tracking of the freely-moving turkeys executing various motor activities, including ramp ascent and descent, vertical ascent and descent, and free roaming movement. Given the demonstrated capacity of magnetomicrometry to track muscle movement in untethered animals, we feel that this technique will enable new scientific explorations and an improved understanding of muscle function.

Clinical viability of magnetic bead implants in muscle

Human movement is accomplished through muscle contraction, yet there does not exist a portable system capable of monitoring muscle length changes in real time. To address this limitation, we previously introduced magnetomicrometry, a minimally-invasive tracking technique comprising two implanted magnetic beads in muscle and a magnetic field sensor array positioned on the body's surface adjacent the implanted beads. The implant system comprises a pair of spherical magnetic beads, each with a first coating of nickel-copper-nickel and an outer coating of Parylene C. In parallel work, we demonstrate submillimeter accuracy of magnetic bead tracking for muscle contractions in an untethered freely-roaming avian model. Here, we address the clinical viability of magnetomicrometry. Using a specialized device to insert magnetic beads into muscle in avian and lagomorph models, we collect data to assess gait metrics, bead migration, and bead biocompatibility. For these animal models, we find no gait differences post-versus pre-implantation, and bead migration towards one another within muscle does not occur for initial bead separation distances greater than 3 cm. Further, using extensive biocompatibility testing, the implants are shown to be non-irritant, non-cytotoxic, non-allergenic, and non-irritating. Our cumulative results lend support for the viability of these magnetic bead implants for implantation in human muscle. We thus anticipate their imminent use in human-machine interfaces, such as in control of prostheses and exoskeletons and in closed-loop neuroprosthetics to aid recovery from neurological disorders.

Clinical implementation of a bionic hand controlled with kineticomyographic signals

Sensing the proper signal could be a vital piece of the solution to the much evading attributes of prosthetic hands, such as robustness to noise, ease of connectivity, and intuitive movement. Towards this end, magnetics tags have been recently suggested as an alternative sensing mechanism to the more common EMG signals. Such sensing technology, however, is inherently invasive and hence only in simulation stages of magnet localization to date. Here, for the first time, we report on the clinical implementation of implanted magnetic tags for an amputee's prosthetic hand from both the medical and engineering perspectives. Specifically, the proposed approach introduces a flexor-extensor tendon transfer surgical procedure to implant the tags, artificial neural networks to extract human intention directly from the implanted magnet's magnetic fields -in short KineticoMyoGraphy (KMG) signals- rather than localizing them, and a game strategy to examine the proposed algorithms and rehabilitate the patient with his new prosthetic hand. The bionic hand's ability is then tested following the patient's intended gesture type and grade. The statistical results confirm the possible utility of surgically implanted magnetic tags as an accurate sensing interface for recognizing the intended gesture and degree of movement between an amputee and his bionic hand.

The myokinetic stimulation interface: activation of proprioceptive neural responses with remotely actuated magnets implanted in rodent forelimb muscles

OBJECTIVE

Proprioception is the sense of one's position, orientation, and movement in space, and it is of fundamental importance for motor control. When proprioception is impaired or absent, motor execution becomes error-prone, leading to poorly coordinated movements. The kinaesthetic illusion, which creates perceptions of limb movement in humans through non-invasively applying vibrations to muscles or tendons, provides an avenue for studying and restoring the sense of joint movement (kinaesthesia). This technique, however, leaves ambiguity between proprioceptive percepts that arise from muscles versus those that arise from skin receptors. Here we propose the concept of a stimulation system to activate kinaesthesia through the untethered application of localized vibration through implanted magnets.

APPROACH

In this proof-of-concept study, we use two simplified 1-DoF systems to show the feasibility of eliciting muscle-sensory responses in an animal model across multiple frequencies, including those that activate the kinaesthetic illusion (70 - 115 Hz). Furthermore, we generalized the concept by developing a 5-DoF prototype system capable of generating directional, frequency-selective vibrations with desired displacement profiles.

MAIN RESULTS

In-vivo tests with the 1-DoF systems demonstrated the feasibility to elicit muscle sensory neural responses in the median nerve of an animal model. Instead, in-vitro tests with the 5-DoF prototype demonstrated high accuracy in producing directional and frequency selective vibrations along different magnet axes.

SIGNIFICANCE

These results provide evidence for a new technique that interacts with the native neuro-muscular anatomy to study proprioception and eventually pave the way towards the development of advanced limb prostheses or assistive devices for the sensory impaired.

Data-Driven Real-Time Magnetic Tracking Applied to Myokinetic Interfaces

A new concept of human-machine interface to control hand prostheses based on displacements of multiple magnets implanted in the limb residual muscles, the myokinetic control interface, has been recently proposed. In previous works, magnets localization has been achieved following an optimization procedure to find an approximate solution to an analytical model. To simplify and speed up the localization problem, here we employ machine learning models, namely linear and radial basis functions artificial neural networks, which can translate measured magnetic information to desired commands for active prosthetic devices. They were developed offline and then implemented on field-programmable gate arrays using customized floating-point operators. We optimized computational precision, execution time, hardware, and energy consumption, as they are essential features in the context of wearable devices. When used to track a single magnet in a mockup of the human forearm, the proposed data-driven strategy achieved a tracking accuracy of 720 μm 95% of the time and latency of 12.07 μs. The proposed system architecture is expected to be more power-efficient compared to previous solutions. The outcomes of this work encourage further research on improving the devised methods to deal with multiple magnets simultaneously.

Optimal Spatial Sensor Design for Magnetic Tracking in a Myokinetic Control Interface

BACKGROUND AND OBJECTIVES

Magnetic tracking involves the use of magnetic sensors to localize one or more magnetic objectives, in those applications in which a free line-of-sight between them and the operator is hampered. We applied this concept to prosthetic hands, which could be controlled by tracking permanent magnets implanted in the forearm muscles of amputees (the myokinetic control interface). Concerning the system design, the definition of a sensor distribution which maximizes the information, while minimizing the computational cost of localization, is still an open problem. We present a simple yet effective strategy to define an optimal sensor set for tracking multiple magnets, which we called the Peaks method.

METHODS

We simulated a proximal amputation using a 3D CAD model of a human forearm, and the implantation of 11 magnets in the residual muscles. The Peaks method was applied to select a subset of sensors from an initial grid of 480 elements. The approach involves setting an appropriate threshold to select those sensors associated with the peaks in the magnetic flux density and its gradient distributions. Selected sensors were used to track the magnets during muscle contraction. For validating our strategy, an alternative method based on state-of-the-art solutions was implemented. We finally proposed a calibration phase to customize the sensor distribution on the specific patient's anatomy.

RESULTS

80 sensors were selected with the Peaks method, and 101 with the alternative one. A localization accuracy below 0.22 mm and 1.86° for position and orientation, respectively, was always achieved. Unlike alternative methods from the literature, neither iterative or analytical solution, nor a-priori knowledge on the magnet positions or trajectories were required, and yet the outcomes achieved with the two strategies proved statistically comparable. The calibration phase proved useful to adapt the sensors to the patient's stump and to increase the signal-to-noise ratio against intrinsic noise.

CONCLUSIONS

We demonstrated an efficient and general solution for solving the design optimization problem (i.e. identifying an optimal sensor set) and reducing the computational cost of localization. The optimal sensor distribution mirrors the field shape traced by the magnets on the sensing surface, being an intuitive and fast way of achieving the same results of more complex and application-specific methods. Several applications in the (bio)medical field involving magnetic tracking will benefit from the outcomes of this work.

Biosignal-Based Human-Machine Interfaces for Assistance and Rehabilitation: A Survey

As a definition, Human–Machine Interface (HMI) enables a person to interact with a device. Starting from elementary equipment, the recent development of novel techniques and unobtrusive devices for biosignals monitoring paved the way for a new class of HMIs, which take such biosignals as inputs to control various applications. The current survey aims to review the large literature of the last two decades regarding biosignal-based HMIs for assistance and rehabilitation to outline state-of-the-art and identify emerging technologies and potential future research trends. PubMed and other databases were surveyed by using specific keywords. The found studies were further screened in three levels (title, abstract, full-text), and eventually, 144 journal papers and 37 conference papers were included. Four macrocategories were considered to classify the different biosignals used for HMI control: biopotential, muscle mechanical motion, body motion, and their combinations (hybrid systems). The HMIs were also classified according to their target application by considering six categories: prosthetic control, robotic control, virtual reality control, gesture recognition, communication, and smart environment control. An ever-growing number of publications has been observed over the last years. Most of the studies (about 67%) pertain to the assistive field, while 20% relate to rehabilitation and 13% to assistance and rehabilitation. A moderate increase can be observed in studies focusing on robotic control, prosthetic control, and gesture recognition in the last decade. In contrast, studies on the other targets experienced only a small increase. Biopotentials are no longer the leading control signals, and the use of muscle mechanical motion signals has experienced a considerable rise, especially in prosthetic control. Hybrid technologies are promising, as they could lead to higher performances. However, they also increase HMIs’ complexity, so their usefulness should be carefully evaluated for the specific application.

Magnetomicrometry

We live in an era of wearable sensing, where our movement through the world can be continuously monitored by devices. Yet, we lack a portable sensor that can continuously monitor muscle, tendon, and bone motion, allowing us to monitor performance, deliver targeted rehabilitation, and provide intuitive, reflexive control over prostheses and exoskeletons. Here, we introduce a sensing modality, magnetomicrometry, that uses the relative positions of implanted magnetic beads to enable wireless tracking of tissue length changes. We demonstrate real-time muscle length tracking in an in vivo turkey model via chronically implanted magnetic beads while investigating accuracy, biocompatibility, and long-term implant stability. We anticipate that this tool will lay the groundwork for volitional control over wearable robots via real-time tracking of muscle lengths and speeds. Further, to inform future biomimetic control strategies, magnetomicrometry may also be used in the in vivo tracking of biological tissues to elucidate biomechanical principles of animal and human movement.

Feasibility of generating 90 Hz vibrations in remote implanted magnets

Limb amputation not only reduces the motor abilities of an individual, but also destroys afferent channels that convey essential sensory information to the brain. Significant efforts have been made in the area of upper limb prosthetics to restore sensory feedback, through the stimulation of residual sensory elements. Most of the past research focused on the replacement of tactile functions. On the other hand, the difficulties in eliciting proprioceptive sensations using either haptic or (neural) electrical stimulation, has limited researchers to rely on sensory substitution. Here we propose the myokinetic stimulation interface, that aims at restoring natural proprioceptive sensations by exploiting the so-called tendon illusion, elicited through the vibration of magnets implanted inside residual muscles. We present a prototype which exploits 12 electromagnetic coils to vibrate up to four magnets implanted in a forearm mockup. The results demonstrated that it is possible to generate highly directional and frequency-selective vibrations. The system proved capable of activating a single magnet, out of many. Hence, this interface constitutes a promising approach to restore naturally perceived proprioception after an amputation. Indeed, by implanting several magnets in independent muscles, it would be possible to restore proprioceptive sensations perceived as coming from single digits.

A novel framework for designing a multi-DoF prosthetic wrist control using machine learning

Prosthetic arms can significantly increase the upper limb function of individuals with upper limb loss, however despite the development of various multi-DoF prosthetic arms the rate of prosthesis abandonment is still high. One of the major challenges is to design a multi-DoF controller that has high precision, robustness, and intuitiveness for daily use. The present study demonstrates a novel framework for developing a controller leveraging machine learning algorithms and movement synergies to implement natural control of a 2-DoF prosthetic wrist for activities of daily living (ADL). The data was collected during ADL tasks of ten individuals with a wrist brace emulating the absence of wrist function. Using this data, the neural network classifies the movement and then random forest regression computes the desired velocity of the prosthetic wrist. The models were trained/tested with ADLs where their robustness was tested using cross-validation and holdout data sets. The proposed framework demonstrated high accuracy (F-1 score of 99% for the classifier and Pearson's correlation of 0.98 for the regression). Additionally, the interpretable nature of random forest regression was used to verify the targeted movement synergies. The present work provides a novel and effective framework to develop an intuitive control for multi-DoF prosthetic devices.

Localization accuracy of multiple magnets in a myokinetic control interface

Magnetic localizers have been widely investigated in the biomedical field, especially for intra-body applications, because they don't require a free line-of-sight between the implanted magnets and the magnetic field sensors. However, while researchers have focused on narrow and specific aspects of the localization problem, no one has comprehensively searched for general design rules for accurately localizing multiple magnetic objectives. In this study, we sought to systematically analyse the effects of remanent magnetization, number of sensors, and geometrical configuration (i.e. distance among magnets-L_inter-MM-and between magnets and sensors-L_MM-sensor) on the accuracy of the localizer in order to unveil the basic principles of the localization problem. Specifically, through simulations validated with a physical system, we observed that the accuracy of the localization was mainly affected by a specific angle ([Formula: see text] = tan^-1(L_inter-MM / L_MM-sensor)), descriptive of the system geometry. In particular, while tracking nine magnets, errors below ~ 1 mm (10% of the length of the simulated trajectory) and around 9° were obtained if θ ≥  ~ 31°. The latter proved a general rule across all tested conditions, also when the number of magnets was doubled. Our results are interesting for a whole range of biomedical engineering applications exploiting multiple-magnets tracking, such as human-machine interfaces, capsule endoscopy, ventriculostomy interventions, and endovascular catheter navigation.

Stability and in vivo safety of gold, titanium nitride and parylene C coatings on NdFeB magnets implanted in muscles towards a new generation of myokinetic prosthetic limbs

Rare earth magnets are the elective choice when high magnetic field density is required and they are particularly intriguing for inclusion in implantable devices. A safe implantation of NdFeB magnets in muscles would enable the control of limb prostheses using a myokinetic interface i.e., direct control of artificial limb movements by means of magnetic tracking of residual muscle contractions. However, myokinetic prosthesis control is prevented by NdFeB magnets poor biocompatibility, at present. Here we investigated three biocompatible materials as NdFeB magnet coating candidates, namely gold, titanium nitride and parylene C, which have not been analyzed in a systematic way for this purpose, so far. In vitro testing in a tissue-mimicking environment and upon contact with C2C12 myoblasts enabled assessment of the superiority of parylene C coated magnets in terms of corrosion prevention and lack of cytotoxicity. In addition, parylene C coated magnets implanted in rabbit muscles for 28 days confirmed, both locally and systemically, their biocompatibility, with a lack of irritation and toxicity associated with the implant. These findings pave the way towards the development of implantable devices based on permanent magnets and of a new generation of limb prostheses.

The Myokinetic Control Interface: How Many Magnets Can be Implanted in an Amputated Forearm? Evidence From a Simulated Environment

We recently introduced the concept of a new human-machine interface (the myokinetic control interface) to control hand prostheses. The interface tracks muscle contractions via permanent magnets implanted in the muscles and magnetic field sensors hosted in the prosthetic socket. Previously we showed the feasibility of localizing several magnets in non-realistic workspaces. Here, aided by a 3D CAD model of the forearm, we computed the localization accuracy simulated for three different below-elbow amputation levels, following general guidelines identified in early work. To this aim we first identified the number of magnets that could fit and be tracked in a proximal (T1), middle (T2) and distal (T3) representative amputation, starting from 18, 20 and 23 eligible muscles, respectively. Then we ran a localization algorithm to estimate the poses of the magnets based on the sensor readings. A sensor selection strategy (from an initial grid of 840 sensors) was also implemented to optimize the computational cost of the localization process. Results showed that the localizer was able to accurately track up to 11 (T1), 13 (T2) and 19 (T3) magnetic markers (MMs) with an array of 154, 205 and 260 sensors, respectively. Localization errors lower than 7% the trajectory travelled by the magnets during muscle contraction were always achieved. This work not only answers the question: "how many magnets could be implanted in a forearm and successfully tracked with a the myokinetic control approach?", but also provides interesting insights for a wide range of bioengineering applications exploiting magnetic tracking.

Control Methods for Transradial Prostheses Based on Remnant Muscle Activity and Its Relationship with Proprioceptive Feedback

The loss of a hand can significantly affect one's work and social life. For many patients, an artificial limb can improve their mobility and ability to manage everyday activities, as well as provide the means to remain independent. This paper provides an extensive review of available biosensing methods to implement the control system for transradial prostheses based on the measured activity in remnant muscles. Covered techniques include electromyography, magnetomyography, electrical impedance tomography, capacitance sensing, near-infrared spectroscopy, sonomyography, optical myography, force myography, phonomyography, myokinetic control, and modern approaches to cineplasty. The paper also covers combinations of these approaches, which, in many cases, achieve better accuracy while mitigating the weaknesses of individual methods. The work is focused on the practical applicability of the approaches, and analyses present challenges associated with each technique along with their relationship with proprioceptive feedback, which is an important factor for intuitive control over the prosthetic device, especially for high dexterity prosthetic hands.

Technological Advances in Prosthesis Design and Rehabilitation Following Upper Extremity Limb Loss

PURPOSE OF REVIEW

The complexity of the human extremity, particularly the upper extremity and the hand, allows us to interact with the world. Prosthetists have struggled to recreate the intuitive motor control, light touch sensation, and proprioception of the innate limb in a manner that reflects the complexity of its native form and function. Nevertheless, recent advances in prosthesis technology, surgical innovations, and enhanced rehabilitation appear promising for patients with limb loss who hope to return to their pre-injury level of function. The purpose of this review is to illustrate recent technological advances that are moving us one step closer to the goal of multi-functional, self-identifiable, durable, and intuitive prostheses.

RECENT FINDINGS

Surgical advances such as targeted muscle reinnervation, regenerative peripheral nerve interfaces, agonist-antagonist myoneural interfaces, and targeted sensory reinnervation; development of technology designed to restore sensation, such as implanted sensors and haptic devices; and evolution of osseointegrated (bone-anchored) prostheses show great promise. Augmented and virtual reality platforms have the potential to enhance prosthesis design, pre-prosthetic training, incorporation, and use. Emerging technologies move surgeons, rehabilitation physicians, therapists, and prosthetists closer to the goal of creating highly functional prostheses with elevated sensory and motor control. Collaboration between medical teams, scientists, and industry stakeholders will be required to keep pace with patients who require durable, high-functioning prostheses.

Feasibility of Tracking Multiple Implanted Magnets With a Myokinetic Control Interface: Simulation and Experimental Evidence Based on the Point Dipole Model

OBJECTIVE

The quest for an intuitive and physiologically appropriate human-machine interface for the control of dexterous prostheses is far from being completed. To control a hand prosthesis, a possible approach could consist in using information related to the displacement of forearm muscles of an amputee during contraction. We recently proposed that muscle displacement could be monitored by implanting passive magnetic markers (MMs- i.e., permanent magnets) in them. We dubbed this the myokinetic interface. However, besides the system feasibility, how much its accuracy, precision and computation time are affected by the number and distribution of both the MMs and the sensors used to record the MF was not quantified.

METHODS

Here we investigated, through simulations validated with a physical system, the performance of a system capable to track position and orientation of up to 9 MMs using information from up to 112 sensors in a volume resembling the dimensions of the human forearm.

RESULTS

The system was able to track up to 7 MMs in 450 ms, demonstrating position/orientation accuracies in the range of 1 mm/5°. The comparison with the experimental recordings demonstrated a median difference with the simulations in the order of 0.45 mm.

CONCLUSION

We were able to formulate general guidelines for the implementation of magnetic tracking systems.

SIGNIFICANCE

Our results pave the way towards the development of new human-machine interfaces for the control of artificial limbs, but they are also interesting for the whole range of biomedical engineering applications exploiting magnetic tracking.

Development of an Embedded Myokinetic Prosthetic Hand Controller

The quest for an intuitive and physiologically appropriate human machine interface for the control of dexterous prostheses is far from being completed. In the last decade, much effort has been dedicated to explore innovative control strategies based on the electrical signals generated by the muscles during contraction. In contrast, a novel approach, dubbed myokinetic interface, derives the control signals from the localization of multiple magnetic markers (MMs) directly implanted into the residual muscles of the amputee. Building on this idea, here we present an embedded system based on 32 magnetic field sensors and a real time computation platform. We demonstrate that the platform can simultaneously localize in real-time up to five MMs in an anatomically relevant workspace. The system proved highly linear (R2 = 0.99) and precise (1% repeatability), yet exhibiting short computation times (4 ms) and limited cross talk errors (10% the mean stroke of the magnets). Compared to a previous PC implementation, the system exhibited similar precision and accuracy, while being ~75% faster. These results proved for the first time the viability of using an embedded system for magnet localization. They also suggest that, by using an adequate number of sensors, it is possible to increase the number of simultaneously tracked MMs while introducing delays that are not perceivable by the human operator. This could allow to control more degrees of freedom than those controllable with current technologies.