Machine Learning Approaches for Activity Recognition and/or Activity Prediction in Locomotion Assistive Devices-A Systematic Review

Social media for palliative and end-of-life care research: a systematic review

BACKGROUND

Social media with real-time content and a wide-reaching user network opens up more possibilities for palliative and end-of-life care (PEoLC) researchers who have begun to embrace it as a complementary research tool. This review aims to identify the uses of social media in PEoLC studies and to examine the ethical considerations and data collection approaches raised by this research approach.

METHODS

Nine online databases were searched for PEoLC research using social media published before December 2022. Thematic analysis and narrative synthesis approach were used to categorise social media applications.

RESULTS

21 studies were included. 16 studies used social media to conduct secondary analysis and five studies used social media as a platform for information sharing. Ethical considerations relevant to social media studies varied while 15 studies discussed ethical considerations, only 6 studies obtained ethical approval and 5 studies confirmed participant consent. Among studies that used social media data, most of them manually collected social media data, and other studies relied on Twitter application programming interface or third-party analytical tools. A total of 1 520 329 posts, 325 videos and 33 articles related to PEoLC from 2008 to 2022 were collected and analysed.

CONCLUSIONS

Social media has emerged as a promising complementary research tool with demonstrated feasibility in various applications. However, we identified the absence of standardised ethical handling and data collection approaches which pose an ongoing challenge. We provided practical recommendations to bridge these pressing gaps for researchers wishing to use social media in future PEoLC-related studies.

A Locomotion Mode Recognition Algorithm Using Adaptive Dynamic Movement Primitives

Control systems of robotic prostheses should be designed to decode the users' intent to start, stop, or change locomotion; and to select the suitable control strategy, accordingly. This paper describes a locomotion mode recognition algorithm based on adaptive Dynamic Movement Primitive models used as locomotion templates. The models take foot-ground contact information and thigh roll angle, measured by an inertial measurement unit, for generating continuous model variables to extract features for a set of Support Vector Machines. The proposed algorithm was tested offline on data acquired from 10 intact subjects and 1 subject with transtibial amputation, in ground-level walking and stair ascending/descending activities. Following subject-specific training, results on intact subjects showed that the algorithm can classify initiatory and steady-state steps with up to 100.00% median accuracy medially at 28.45% and 27.40% of the swing phase, respectively. While the transitory steps were classified with up to 87.30% median accuracy medially at 90.54% of the swing phase. Results with data of the transtibial amputee showed that the algorithm classified initiatory, steady-state, and transitory steps with up to 92.59%, 100%, and 93.10% median accuracies medially at 19.48%, 51.47%, and 93.33% of the swing phase, respectively. The results support the feasibility of this approach in robotic prosthesis control.

Prediction of Gait Kinematics and Kinetics: A Systematic Review of EMG and EEG Signal Use and Their Contribution to Prediction Accuracy

Human-machine interfaces hold promise in enhancing rehabilitation by predicting and responding to subjects' movement intent. In gait rehabilitation, neural network architectures utilize lower-limb muscle and brain activity to predict continuous kinematics and kinetics during stepping and walking. This systematic review, spanning five databases, assessed 16 papers meeting inclusion criteria. Studies predicted lower-limb kinematics and kinetics using electroencephalograms (EEGs), electromyograms (EMGs), or a combination with kinematic data and anthropological parameters. Long short-term memory (LSTM) and convolutional neural network (CNN) tools demonstrated highest accuracies. EEG focused on joint angles, while EMG predicted moments and torque joints. Useful EEG electrode locations included C3, C4, Cz, P3, F4, and F8. Vastus Lateralis, Rectus Femoris, and Gastrocnemius were the most commonly accessed muscles for kinematic and kinetic prediction using EMGs. No studies combining EEGs and EMGs to predict lower-limb kinematics and kinetics during stepping or walking were found, suggesting a potential avenue for future development in this technology.

Unsupervised Cluster Analysis of Walking Activity Data for Healthy Individuals and Individuals with Lower Limb Amputation

This is the first investigation to perform an unsupervised cluster analysis of activities performed by individuals with lower limb amputation (ILLAs) and individuals without gait impairment, in free-living conditions. Eight individuals with no gait impairments and four ILLAs wore a thigh-based accelerometer and walked on an improvised route across a variety of terrains in the vicinity of their homes. Their physical activity data were clustered to extract 'unique' groupings in a low-dimension feature space in an unsupervised learning approach, and an algorithm was created to automatically distinguish such activities. After testing three dimensionality reduction methods-namely, principal component analysis (PCA), t-distributed stochastic neighbor embedding (tSNE), and uniform manifold approximation and projection (UMAP)-we selected tSNE due to its performance and stable outputs. Cluster formation of activities via DBSCAN only occurred after the data were reduced to two dimensions via tSNE and contained only samples for a single individual. Additionally, through analysis of the t-SNE plots, appreciable clusters in walking-based activities were only apparent with ground walking and stair ambulation. Through a combination of density-based clustering and analysis of cluster distance and density, a novel algorithm inspired by the t-SNE plots, resulting in three proposed and validated hypotheses, was able to identify cluster formations that arose from ground walking and stair ambulation. Low dimensional clustering of activities has thus been found feasible when analyzing individual sets of data and can currently recognize stair and ground walking ambulation.

Improving performance of human action intent recognition: Analysis of gait recognition machine learning algorithms and optimal combination with inertial measurement units

Human action intent recognition has become increasingly dependent on computational accuracy, real-time responsiveness, and model lightness. Model selection, data filtering, and experimental design are three critical factors for the recognition of human intention in research. However, the performance of machine learning algorithms can vary depending on factors such as sensor location, the number of sensors used, channel selection, and dimensional combinations. Moreover, the collection of adequate and balanced data in such scenarios can be challenging. To address this issue, we present a comparative analysis of 12 commonly used machine learning algorithms for human action intention recognition. The synthetic minority oversampling technique is applied to fill in missing data. Traversing all possible combinations would require conducting 686 experiments, which is a daunting task in terms of both cost and efficiency. To tackle this challenge, we employ an orthogonal experiment design based on the Quasi-horizontal method. Our analysis indicates that lightGBM outperforms other algorithms in recognizing eight human daily activities. Furthermore, we conduct a polar difference and variance analysis based on a comprehensive balanced multi-metric orthogonal experiment for lightGBM using various sensor combinations and dimensions. The optimal combinations of different sensor numbers in terms of position, channel, and dimension are derived using this approach. Notably, our experimental design reduces the number of experiments required to only 49 times.

Too much information is no information: how machine learning and feature selection could help in understanding the motor control of pointing

The aim of this study was to develop the use of Machine Learning techniques as a means of multivariate analysis in studies of motor control. These studies generate a huge amount of data, the analysis of which continues to be largely univariate. We propose the use of machine learning classification and feature selection as a means of uncovering feature combinations that are altered between conditions. High dimensional electromyogram (EMG) vectors were generated as several arm and trunk muscles were recorded while subjects pointed at various angles above and below the gravity neutral horizontal plane. We used Linear Discriminant Analysis (LDA) to carry out binary classifications between the EMG vectors for pointing at a particular angle, vs. pointing at the gravity neutral direction. Classification success provided a composite index of muscular adjustments for various task constraints-in this case, pointing angles. In order to find the combination of features that were significantly altered between task conditions, we conducted a post classification feature selection i.e., investigated which combination of features had allowed for the classification. Feature selection was done by comparing the representations of each category created by LDA for the classification. In other words computing the difference between the representations of each class. We propose that this approach will help with comparing high dimensional EMG patterns in two ways; (i) quantifying the effects of the entire pattern rather than using single arbitrarily defined variables and (ii) identifying the parts of the patterns that convey the most information regarding the investigated effects.

Database of lower limb kinematics and electromyography during gait-related activities in able-bodied subjects

This data descriptor describes the Roessingh Research & Development-MyLeg database for activity prediction (MyPredict), containing three data sets. These data sets contain data from 55 able-bodied subjects, mean age 24 ± 2 years, measured in 85 measurement sessions. Measurement sessions consisted of trials containing sitting, standing, overground walking, stair ascent, stair descent, ramp ascent, ramp descent, walking on uneven terrain and walking in simulated confined spaces. Subjects were measured using eight inertial measurement units in combination with different types of sEMG. Recorded kinematics consisted of joint angles, sensor accelerations, angular velocity, orientation and virtual marker positions. sEMG was recorded using bipolar sEMG, multi-array sEMG or a combination of both. All data showed excellent correlation with other online available data sets. The data reported in this descriptor forms a solid basis for research into myoelectric pattern recognition, myoelectric control development and electromyography to be used in data-driven applications.

Ten steps to becoming a musculoskeletal simulation expert: A half-century of progress and outlook for the future

Over the past half-century, musculoskeletal simulations have deepened our knowledge of human and animal movement. This article outlines ten steps to becoming a musculoskeletal simulation expert so you can contribute to the next half-century of technical innovation and scientific discovery. We advocate looking to the past, present, and future to harness the power of simulations that seek to understand and improve mobility. Instead of presenting a comprehensive literature review, we articulate a set of ideas intended to help researchers use simulations effectively and responsibly by understanding the work on which today's musculoskeletal simulations are built, following established modeling and simulation principles, and branching out in new directions.

Ambulation Mode Classification of Individuals with Transfemoral Amputation through A-Mode Sonomyography and Convolutional Neural Networks

Many people struggle with mobility impairments due to lower limb amputations. To participate in society, they need to be able to walk on a wide variety of terrains, such as stairs, ramps, and level ground. Current lower limb powered prostheses require different control strategies for varying ambulation modes, and use data from mechanical sensors within the prosthesis to determine which ambulation mode the user is in. However, it can be challenging to distinguish between ambulation modes. Efforts have been made to improve classification accuracy by adding electromyography information, but this requires a large number of sensors, has a low signal-to-noise ratio, and cannot distinguish between superficial and deep muscle activations. An alternative sensing modality, A-mode ultrasound, can detect and distinguish between changes in superficial and deep muscles. It has also shown promising results in upper limb gesture classification. Despite these advantages, A-mode ultrasound has yet to be employed for lower limb activity classification. Here we show that A- mode ultrasound can classify ambulation mode with comparable, and in some cases, superior accuracy to mechanical sensing. In this study, seven transfemoral amputee subjects walked on an ambulation circuit while wearing A-mode ultrasound transducers, IMU sensors, and their passive prosthesis. The circuit consisted of sitting, standing, level-ground walking, ramp ascent, ramp descent, stair ascent, and stair descent, and a spatial-temporal convolutional network was trained to continuously classify these seven activities. Offline continuous classification with A-mode ultrasound alone was able to achieve an accuracy of 91.8±3.4%, compared with 93.8±3.0%, when using kinematic data alone. Combined kinematic and ultrasound produced 95.8±2.3% accuracy. This suggests that A-mode ultrasound provides additional useful information about the user's gait beyond what is provided by mechanical sensors, and that it may be able to improve ambulation mode classification. By incorporating these sensors into powered prostheses, users may enjoy higher reliability for their prostheses, and more seamless transitions between ambulation modes.

Comparison of machine learning and deep learning-based methods for locomotion mode recognition using a single inertial measurement unit

Locomotion mode recognition provides the prosthesis control with the information on when to switch between different walking modes, whereas the gait phase detection indicates where we are in the gait cycle. But powered prostheses often implement a different control strategy for each locomotion mode to improve the functionality of the prosthesis. Existing studies employed several classical machine learning methods for locomotion mode recognition. However, these methods were less effective for data with complex decision boundaries and resulted in misclassifications of motion recognition. Deep learning-based methods potentially resolve these limitations as it is a special type of machine learning method with more sophistication. Therefore, this study evaluated three deep learning-based models for locomotion mode recognition, namely recurrent neural network (RNN), long short-term memory (LSTM) neural network, and convolutional neural network (CNN), and compared the recognition performance of deep learning models to the machine learning model with random forest classifier (RFC). The models are trained from data of one inertial measurement unit (IMU) placed on the lower shanks of four able-bodied subjects to perform four walking modes, including level ground walking (LW), standing (ST), and stair ascent/stair descent (SA/SD). The results indicated that CNN and LSTM models outperformed other models, and these models were promising for applying locomotion mode recognition in real-time for robotic prostheses.

A Review on Locomotion Mode Recognition and Prediction When Using Active Orthoses and Exoskeletons

Understanding how to seamlessly adapt the assistance of lower-limb wearable assistive devices (active orthosis (AOs) and exoskeletons) to human locomotion modes (LMs) is challenging. Several algorithms and sensors have been explored to recognize and predict the users' LMs. Nevertheless, it is not yet clear which are the most used and effective sensor and classifier configurations in AOs/exoskeletons and how these devices' control is adapted according to the decoded LMs. To explore these aspects, we performed a systematic review by electronic search in Scopus and Web of Science databases, including published studies from 1 January 2010 to 31 August 2022. Sixteen studies were included and scored with 84.7 ± 8.7% quality. Decoding focused on level-ground walking along with ascent/descent stairs tasks performed by healthy subjects. Time-domain raw data from inertial measurement unit sensors were the most used data. Different classifiers were employed considering the LMs to decode (accuracy above 90% for all tasks). Five studies have adapted the assistance of AOs/exoskeletons attending to the decoded LM, in which only one study predicted the new LM before its occurrence. Future research is encouraged to develop decoding tools considering data from people with lower-limb impairments walking at self-selected speeds while performing daily LMs with AOs/exoskeletons.

Adaptive Lower Limb Pattern Recognition for Multi-Day Control

Pattern recognition in EMG-based control systems suffer from increase in error rate over time, which could lead to unwanted behavior. This so-called concept drift in myoelectric control systems could be caused by fatigue, sensor replacement and varying skin conditions. To circumvent concept drift, adaptation strategies could be used to retrain a pattern recognition system, which could lead to comparable error rates over multiple days. In this study, we investigated the error rate development over one week and compared three adaptation strategies to reduce the error rate increase. The three adaptation strategies were based on entropy, on backward prediction and a combination of backward prediction and entropy. Ten able-bodied subjects were measured on four measurement days while performing gait-related activities. During the measurement electromyography and kinematics were recorded. The three adaptation strategies were implemented and compared against the baseline error rate and against adaptation using the ground truth labels. It can be concluded that without adaptation the baseline error rate increases significantly from day 1 to 2, but plateaus on day 2, 3 and 7. Of the three tested adaptation strategies, entropy based adaptation showed the smallest increase in error rate over time. It can be concluded that entropy based adaptation is simple to implement and can be considered a feasible adaptation strategy for lower limb pattern recognition.

Multilabel Classification Methods for Human Activity Recognition: A Comparison of Algorithms

As the world’s population is aging, and since access to ambient sensors has become easier over the past years, activity recognition in smart home installations has gained increased scientific interest. The majority of published papers in the literature focus on single-resident activity recognition. While this is an important area, especially when focusing on elderly people living alone, multi-resident activity recognition has potentially more applications in smart homes. Activity recognition for multiple residents acting concurrently can be treated as a multilabel classification problem (MLC). In this study, an experimental comparison between different MLC algorithms is attempted. Three different techniques were implemented: RAkELd, classifier chains, and binary relevance. These methods are evaluated using the ARAS and CASAS public datasets. Results obtained from experiments have shown that using MLC can recognize activities performed by multiple people with high accuracy. While RAkELd had the best performance, the rest of the methods had on-par results.

Machine-Learning-Aided Self-Powered Assistive Physical Therapy Devices

An expanding elderly population and people with disabilities pose considerable challenges to the current healthcare system. As a practical technology that integrates systems and services, assistive physical therapy devices are essential to maintain or to improve an individual's functioning and independence, thus promoting their well-being. Given technological advancements, core components of self-powered sensors and optimized machine-learning algorithms will play innovative roles in providing assistive services for unmet global needs. In this Perspective, we provide an overview of the latest developments in machine-learning-aided assistive physical therapy devices based on emerging self-powered sensing systems and a discussion of the challenges and opportunities in this field.

Human Activity Recognition of Individuals with Lower Limb Amputation in Free-Living Conditions: A Pilot Study

This pilot study aimed to investigate the implementation of supervised classifiers and a neural network for the recognition of activities carried out by Individuals with Lower Limb Amputation (ILLAs), as well as individuals without gait impairment, in free living conditions. Eight individuals with no gait impairments and four ILLAs wore a thigh-based accelerometer and walked on an improvised route in the vicinity of their homes across a variety of terrains. Various machine learning classifiers were trained and tested for recognition of walking activities. Additional investigations were made regarding the detail of the activity label versus classifier accuracy and whether the classifiers were capable of being trained exclusively on non-impaired individuals’ data and could recognize physical activities carried out by ILLAs. At a basic level of label detail, Support Vector Machines (SVM) and Long-Short Term Memory (LSTM) networks were able to acquire 77–78% mean classification accuracy, which fell with increased label detail. Classifiers trained on individuals without gait impairment could not recognize activities carried out by ILLAs. This investigation presents the groundwork for a HAR system capable of recognizing a variety of walking activities, both for individuals with no gait impairments and ILLAs.

A Machine Learning Classification Model for Monitoring the Daily Physical Behaviour of Lower-Limb Amputees

There are currently limited data on how prosthetic devices are used to support lower-limb prosthesis users in their free-living environment. Possessing the ability to monitor a patient’s physical behaviour while using these devices would enhance our understanding of the impact of different prosthetic products. The current approaches for monitoring human physical behaviour use a single thigh or wrist-worn accelerometer, but in a lower-limb amputee population, we have the unique opportunity to embed a device within the prosthesis, eliminating compliance issues. This study aimed to develop a model capable of accurately classifying postures (sitting, standing, stepping, and lying) by using data from a single shank-worn accelerometer. Free-living posture data were collected from 14 anatomically intact participants and one amputee over three days. A thigh worn activity monitor collected labelled posture data, while a shank worn accelerometer collected 3-axis acceleration data. Postures and the corresponding shank accelerations were extracted in window lengths of 5–180 s and used to train several machine learning classifiers which were assessed by using stratified cross-validation. A random forest classifier with a 15 s window length provided the highest classification accuracy of 93% weighted average F-score and between 88 and 98% classification accuracy across all four posture classes, which is the best performance achieved to date with a shank-worn device. The results of this study show that data from a single shank-worn accelerometer with a machine learning classification model can be used to accurately identify postures that make up an individual’s daily physical behaviour. This opens up the possibility of embedding an accelerometer-based activity monitor into the shank component of a prosthesis to capture physical behaviour information in both above and below-knee amputees. The models and software used in this study have been made open source in order to overcome the current restrictions of applying activity monitoring methods to lower-limb prosthesis users.

Machine Learning in Agriculture: A Comprehensive Updated Review

The digital transformation of agriculture has evolved various aspects of management into artificial intelligent systems for the sake of making value from the ever-increasing data originated from numerous sources. A subset of artificial intelligence, namely machine learning, has a considerable potential to handle numerous challenges in the establishment of knowledge-based farming systems. The present study aims at shedding light on machine learning in agriculture by thoroughly reviewing the recent scholarly literature based on keywords' combinations of "machine learning" along with "crop management", "water management", "soil management", and "livestock management", and in accordance with PRISMA guidelines. Only journal papers were considered eligible that were published within 2018-2020. The results indicated that this topic pertains to different disciplines that favour convergence research at the international level. Furthermore, crop management was observed to be at the centre of attention. A plethora of machine learning algorithms were used, with those belonging to Artificial Neural Networks being more efficient. In addition, maize and wheat as well as cattle and sheep were the most investigated crops and animals, respectively. Finally, a variety of sensors, attached on satellites and unmanned ground and aerial vehicles, have been utilized as a means of getting reliable input data for the data analyses. It is anticipated that this study will constitute a beneficial guide to all stakeholders towards enhancing awareness of the potential advantages of using machine learning in agriculture and contributing to a more systematic research on this topic.

Understanding LSTM Network Behaviour of IMU-Based Locomotion Mode Recognition for Applications in Prostheses and Wearables

Human Locomotion Mode Recognition (LMR) has the potential to be used as a control mechanism for lower-limb active prostheses. Active prostheses can assist and restore a more natural gait for amputees, but as a medical device it must minimize user risks, such as falls and trips. As such, any control system must have high accuracy and robustness, with a detailed understanding of its internal operation. Long Short-Term Memory (LSTM) machine-learning networks can perform LMR with high accuracy levels. However, the internal behavior during classification is unknown, and they struggle to generalize when presented with novel users. The target problem addressed in this paper is understanding the LSTM classification behavior for LMR. A dataset of six locomotive activities (walking, stopped, stairs and ramps) from 22 non-amputee subjects is collected, capturing both steady-state and transitions between activities in natural environments. Non-amputees are used as a substitute for amputees to provide a larger dataset. The dataset is used to analyze the internal behavior of a reduced complexity LSTM network. This analysis identifies that the model primarily classifies activity type based on data around early stance. Evaluation of generalization for unseen subjects reveals low sensitivity to hyper-parameters and over-fitting to individuals’ gait traits. Investigating the differences between individual subjects showed that gait variations between users primarily occur in early stance, potentially explaining the poor generalization. Adjustment of hyper-parameters alone could not solve this, demonstrating the need for individual personalization of models. The main achievements of the paper are (i) the better understanding of LSTM for LMR, (ii) demonstration of its low sensitivity to learning hyper-parameters when evaluating novel user generalization, and (iii) demonstration of the need for personalization of ML models to achieve acceptable accuracy.