Using a high spatial resolution tactile sensor for intention detection

Channel Selection for Gesture Recognition Using Force Myography: A Universal Model for Gesture Measurement Points

Gesture recognition has emerged as a significant research domain in computer vision and human-computer interaction. One of the key challenges in gesture recognition is how to select the most useful channels that can effectively represent gesture movements. In this study, we have developed a channel selection algorithm that determines the number and placement of sensors that are critical to gesture classification. To validate this algorithm, we constructed a Force Myography (FMG)-based signal acquisition system. The algorithm considers each sensor as a distinct channel, with the most effective channel combinations and recognition accuracy determined through assessing the correlation between each channel and the target gesture, as well as the redundant correlation between different channels. The database was created by collecting experimental data from 10 healthy individuals who wore 16 sensors to perform 13 unique hand gestures. The results indicate that the average number of channels across the 10 participants was 3, corresponding to an 75% decrease in the initial channel count, with an average recognition accuracy of 94.46%. This outperforms four widely adopted feature selection algorithms, including Relief-F, mRMR, CFS, and ILFS. Moreover, we have established a universal model for the position of gesture measurement points and verified it with an additional five participants, resulting in an average recognition accuracy of 96.3%. This study provides a sound basis for identifying the optimal and minimum number and location of channels on the forearm and designing specialized arm rings with unique shapes.

A survey on the state of the art of force myography technique (FMG): analysis and assessment

Precise feedback assures precise control commands especially for assistive or rehabilitation devices. Biofeedback systems integrated with assistive or rehabilitative robotic exoskeletons tend to increase its performance and effectiveness. Therefore, there has been plenty of research in the field of biofeedback covering different aspects such as signal acquisition, conditioning, feature extraction and integration with the control system. Among several types of biofeedback systems, Force myography (FMG) technique is a promising one in terms of affordability, high classification accuracies, ease to use, and low computational cost. Compared to traditional biofeedback systems such as electromyography (EMG) which offers some invasive techniques, FMG offers a completely non-invasive solution with much less effort for preprocessing with high accuracies. This work covers the whole aspects of FMG technique in terms of signal acquisition, feature extraction, signal processing, developing the machine learning model, evaluating tools for the performance of the model. Stating the difference between real-time and offline assessment, also highlighting the main uncovered points for further study, and thus enhancing the development of this technique.

Design of Fuzzy Logic Motion Detection Algorithm for the Bracelet Type Sensor Consisting of Conductive Layer-Polymer Composite Film

To improve the motion detection performance of a bracelet-type sensor that uses only two tiny sensor modules developed using carbon-based conductive polymer composite films, a fuzzy-logic algorithm was developed in this study. A polyethylene terephthalate polymer film with a conductive layer composed of carbon paste was used as the integral material utilized for the composite film; a small sensor module composed of mechanical parts mounted on the film was developed to effectively detect the surface resistance variations of the film. A participant wore a bracelet sensor, which consisted of two sensor modules, on their forearm, and the resistance variations of the contact area between the forearm and the sensor modules corresponding to the flexion changes of the surface of the body due to muscle contraction and relaxation were detected. The surface resistance variations of the film were converted to voltage signals, which were used as inputs to the fuzzy logic algorithm to detect four consecutive motions of the forearm. The results demonstrated that the fuzzy-logic algorithm attained an accuracy of 94%. The fuzzy algorithm successfully detected four motions and the resting state of the forearms; moreover, it showed improved performance compared to previous research.

Wearable Sensor for Forearm Motion Detection Using a Carbon-Based Conductive Layer-Polymer Composite Film

In this study, we developed a fabrication method for a bracelet-type wearable sensor to detect four motions of the forearm by using a carbon-based conductive layer-polymer composite film. The integral material used for the composite film is a polyethylene terephthalate polymer film with a conductive layer composed of a carbon paste. It is capable of detecting the resistance variations corresponding to the flexion changes of the surface of the body due to muscle contraction and relaxation. To effectively detect the surface resistance variations of the film, a small sensor module composed of mechanical parts mounted on the film was designed and fabricated. A subject wore the bracelet sensor, consisting of three such sensor modules, on their forearm. The surface resistance of the film varied corresponding to the flexion change of the contact area between the forearm and the sensor modules. The surface resistance variations of the film were converted to voltage signals and used for motion detection. The results demonstrate that the thin bracelet-type wearable sensor, which is comfortable to wear and easily applicable, successfully detected each motion with high accuracy.

Brain-machine and muscle-machine bio-sensing methods for gesture intent acquisition in upper-limb prosthesis control: a review

This paper presents a review of a number of bio-sensing methods for gesture intent signal acquisition in control tasks for upper-limb prosthesis. The paper specifically provides a breakdown of the control task in myoelectric prosthesis, and in addition, highlights and describes the importance of the acquisition of a high-quality bio-signal. The paper also describes commonly used invasive and non-invasive brain and muscle machine interfaces such as electroencephalography, electrocorticography, electroneurography, surface electromyography, sonomyography, mechanomyography, near infra-red, force sensitive resistance/pressure, and magnetoencephalography. Each modality is reviewed based on its operating principle and limitations in gesture recognition, followed by respective advantages and disadvantages. Also described within this paper, are multimodal sensing approaches, which involve data fusion of information from various sensing modalities for an enhanced neuromuscular bio-sensing source. Using a semi-systematic review methodology, we are able to derive a novel tabular approach towards contrasting the various strengths and weaknesses of the reviewed bio-sensing methods towards gesture recognition in a prosthesis interface. This would allow for a streamlined method of down selection of an appropriate bio-sensor given specific prosthesis design criteria and requirements. The paper concludes by highlighting a number of research areas that require more work for strides to be made towards improving and enhancing the connection between man and machine as it concerns upper-limb prosthesis. Such areas include classifier augmentation for gesture recognition, filtering techniques for sensor disturbance rejection, feeling of tactile sensations with an artificial limb.

Investigation of Channel Selection for Gesture Classification for Prosthesis Control Using Force Myography: A Case Study

Background: Various human machine interfaces (HMIs) are used to control prostheses, such as robotic hands. One of the promising HMIs is Force Myography (FMG). Previous research has shown the potential for the use of high density FMG (HD-FMG) that can lead to higher accuracy of prosthesis control. Motivation: The more sensors used in an FMG controlled system, the more complicated and costlier the system becomes. This study proposes a design method that can produce powered prostheses with performance comparable to that of HD-FMG controlled systems using a fewer number of sensors. An HD-FMG apparatus would be used to collect information from the user only in the design phase. Channel selection would then be applied to the collected data to determine the number and location of sensors that are vital to performance of the device. This study assessed the use of multiple channel selection (CS) methods for this purpose. Methods: In this case study, three datasets were used. These datasets were collected from force sensitive resistors embedded in the inner socket of a subject with transradial amputation. Sensor data were collected as the subject carried out five repetitions of six gestures. Collected data were then used to asses five CS methods: Sequential forward selection (SFS) with two different stopping criteria, minimum redundancy-maximum relevance (mRMR), genetic algorithm (GA), and Boruta. Results: Three out of the five methods (mRMR, GA, and Boruta) were able to decrease channel numbers significantly while maintaining classification accuracy in all datasets. Neither of them outperformed the other two in all datasets. However, GA resulted in the smallest channel subset in all three of the datasets. The three selected methods were also compared in terms of stability [i.e., consistency of the channel subset chosen by the method as new training data were introduced or some training data were removed (Chandrashekar and Sahin, 2014)]. Boruta and mRMR resulted in less instability compared to GA when applied to the datasets of this study. Conclusion: This study shows feasibility of using the proposed design method that can produce prosthetic systems that are simpler than HD-FMG systems but have performance comparable to theirs.

Tactile Signatures and Hand Motion Intent Recognition for Wearable Assistive Devices

Within the field of robotics and autonomous systems where there is a human in the loop, intent recognition plays an important role. This is especially true for wearable assistive devices used for rehabilitation, particularly post-stroke recovery. This paper reports results on the use of tactile patterns to detect weak muscle contractions in the forearm while at the same time associating these patterns with the muscle synergies during different grips. To investigate this concept, a series of experiments with healthy participants were carried out using a tactile arm brace (TAB) on the forearm while performing four different types of grip. The expected force patterns were established by analysing the muscle synergies of the four grip types and the forearm physiology. The results showed that the tactile signatures of the forearm recorded on the TAB align with the anticipated force patterns. Furthermore, a linear separability of the data across all four grip types was identified. Using the TAB data, machine learning algorithms achieved a 99% classification accuracy. The TAB results were highly comparable to a similar commercial intent recognition system based on a surface electromyography (sEMG) sensing.

FMG Versus EMG: A Comparison of Usability for Real-Time Pattern Recognition Based Control

OBJECTIVE

Force myography (FMG), which measures the surface pressure profile exerted by contracting muscles, has been proposed as an alternative to electromyography (EMG) for human-machine interfaces. Although FMG pattern recognition-based control systems have yielded higher offline classification accuracy, comparatively few works have examined the usability of FMG for real-time control. In this work, we conduct a comprehensive comparison of EMG- and FMG-based schemes using both classification and regression controllers.

METHODS

A total of 20 participants performed a two-degree-of-freedom Fitts' Law-style virtual target acquisition task using both FMG- and EMG-based classification and regression control schemes. Performance was evaluated based on the standard Fitts' law testing metrics throughput, path efficiency, average speed, number of timeouts, overshoot, stopping distance, and simultaneity.

RESULTS

The FMG-based classification system significantly outperformed the EMG-based classification system in both throughput (0.902 ± 0.270) versus (0.751 ± 0.309), (ρ < 0.001) and path efficiency (87.2 ± 8.7) versus (83.2 ± 7.8), (ρ < 0.001). Similarly, FMG-based regression significantly outperformed EMG-based regression in throughput (0.871 ± 0.2) versus (0.69 ± 0.3), (ρ < 0.001) and path efficiency (64.8 ± 5.3) versus (58.8 ± 7.1), (ρ < 0.001).

CONCLUSIONS

The FMG-based schemes outperformed the EMG-based schemes regardless of which controller was used. This provides further evidence for FMG as a viable alternative to EMG for human-machine interfaces.

SIGNIFICANCE

This work describes a comprehensive evaluation of the online usability of FMG- and EMG-based control using both sequential classification and simultaneous regression control.

A proportional control scheme for high density force myography

OBJECTIVE

Force myography (FMG) has been shown to be a potentially higher accuracy alternative to electromyography for pattern recognition based prosthetic control. Classification accuracy, however, is just one factor that affects the usability of a control system. Others, like the ability to start and stop, to coordinate dynamic movements, and to control the velocity of the device through some proportional control scheme can be of equal importance. To impart effective fine control using FMG-based pattern recognition, it is important that a method of controlling the velocity of each motion be developed.

METHODS

In this work force myography data were collected from 14 able bodied participants and one amputee participant as they performed a set of wrist and hand motions. The offline proportional control performance of a standard mean signal amplitude approach and a proposed regression-based alternative was compared. The impact of providing feedback during training, as well as the use of constrained or unconstrained hand and wrist contractions, were also evaluated.

RESULTS

It is shown that the commonly used mean of rectified channel amplitudes approach commonly employed with electromyography does not translate to force myography. The proposed class-based regression proportional control approach is shown significantly outperform this standard approach (ρ  <  0.001), yielding a R² correlation coefficients of 0.837 and 0.830 for constrained and unconstrained forearm contractions, respectively for able bodied participants. No significant difference (ρ  =  0.693) was found in R² performance when feedback was provided during training or not. The amputee subject achieved a classification accuracy of 83.4%  ±  3.47% demonstrating the ability to distinguish contractions well with FMG. In proportional control the amputee participant achieved an R² of of 0.375 for regression based proportional control during unconstrained contractions. This is lower than the unconstrained case for able-bodied subjects for this particular amputee, possibly due to difficultly in visualizing contraction level modulation without feedback. This may be remedied in the use of a prosthetic limb that would provide real-time feedback in the form of device speed.

CONCLUSION

A novel class-specific regression-based approach is proposed for multi-class control is described and shown to provide an effective means of providing FMG-based proportional control.

Non-invasive control interfaces for intention detection in active movement-assistive devices

Active movement-assistive devices aim to increase the quality of life for patients with neuromusculoskeletal disorders. This technology requires interaction between the user and the device through a control interface that detects the user’s movement intention. Researchers have explored a wide variety of invasive and non-invasive control interfaces. To summarize the wide spectrum of strategies, this paper presents a comprehensive review focused on non-invasive control interfaces used to operate active movement-assistive devices. A novel systematic classification method is proposed to categorize the control interfaces based on: (I) the source of the physiological signal, (II) the physiological phenomena responsible for generating the signal, and (III) the sensors used to measure the physiological signal. The proposed classification method can successfully categorize all the existing control interfaces providing a comprehensive overview of the state of the art. Each sensing modality is briefly described in the body of the paper following the same structure used in the classification method. Furthermore, we discuss several design considerations, challenges, and future directions of non-invasive control interfaces for active movement-assistive devices.

Driving interface based on tactile sensors for electric wheelchairs or trolleys

This paper introduces a novel device based on a tactile interface to replace the attendant joystick in electric wheelchairs. It can also be used in other vehicles such as shopping trolleys. Its use allows intuitive driving that requires little or no training, so its usability is high. This is achieved by a tactile sensor located on the handlebar of the chair or trolley and the processing of the information provided by it. When the user interacts with the handle of the chair or trolley, he or she exerts a pressure pattern that depends on the intention to accelerate, brake or turn to the left or right. The electronics within the device then perform the signal conditioning and processing of the information received, identifying the intention of the user on the basis of this pattern using an algorithm, and translating it into control signals for the control module of the wheelchair. These signals are equivalent to those provided by a joystick. This proposal aims to help disabled people and their attendees and prolong the personal autonomy in a context of aging populations.