Evaluation of classification performance in human lower limb jump phases of signal correlation information and LSTM models

Use of Artificial Intelligence Techniques to Assist Individuals with Physical Disabilities

Assistive technologies (AT) enable people with disabilities to perform activities of daily living more independently, have greater access to community and healthcare services, and be more productive performing educational and/or employment tasks. Integrating artificial intelligence (AI) with various agents, including electronics, robotics, and software, has revolutionized AT, resulting in groundbreaking technologies such as mind-controlled exoskeletons, bionic limbs, intelligent wheelchairs, and smart home assistants. This article provides a review of various AI techniques that have helped those with physical disabilities, including brain-computer interfaces, computer vision, natural language processing, and human-computer interaction. The current challenges and future directions for AI-powered advanced technologies are also addressed.

Multimodal Gait Abnormality Recognition Using a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) Network Based on Multi-Sensor Data Fusion

Global aging leads to a surge in neurological diseases. Quantitative gait analysis for the early detection of neurological diseases can effectively reduce the impact of the diseases. Recently, extensive research has focused on gait-abnormality-recognition algorithms using a single type of portable sensor. However, these studies are limited by the sensor's type and the task specificity, constraining the widespread application of quantitative gait recognition. In this study, we propose a multimodal gait-abnormality-recognition framework based on a Convolutional Neural Network-Bidirectional Long Short-Term Memory (CNN-BiLSTM) network. The as-established framework effectively addresses the challenges arising from smooth data interference and lengthy time series by employing an adaptive sliding window technique. Then, we convert the time series into time-frequency plots to capture the characteristic variations in different abnormality gaits and achieve a unified representation of the multiple data types. This makes our signal processing method adaptable to several types of sensors. Additionally, we use a pre-trained Deep Convolutional Neural Network (DCNN) for feature extraction, and the consequently established CNN-BiLSTM network can achieve high-accuracy recognition by fusing and classifying the multi-sensor input data. To validate the proposed method, we conducted diversified experiments to recognize the gait abnormalities caused by different neuropathic diseases, such as amyotrophic lateral sclerosis (ALS), Parkinson's disease (PD), and Huntington's disease (HD). In the PDgait dataset, the framework achieved an accuracy of 98.89% in the classification of Parkinson's disease severity, surpassing DCLSTM's 96.71%. Moreover, the recognition accuracy of ALS, PD, and HD on the PDgait dataset was 100%, 96.97%, and 95.43% respectively, surpassing the majority of previously reported methods. These experimental results strongly demonstrate the potential of the proposed multimodal framework for gait abnormality identification. Due to the advantages of the framework, such as its suitability for different types of sensors and fewer training parameters, it is more suitable for gait monitoring in daily life and the customization of medical rehabilitation schedules, which will help more patients alleviate the harm caused by their diseases.

Improving performance of human action intent recognition: Analysis of gait recognition machine learning algorithms and optimal combination with inertial measurement units

Human action intent recognition has become increasingly dependent on computational accuracy, real-time responsiveness, and model lightness. Model selection, data filtering, and experimental design are three critical factors for the recognition of human intention in research. However, the performance of machine learning algorithms can vary depending on factors such as sensor location, the number of sensors used, channel selection, and dimensional combinations. Moreover, the collection of adequate and balanced data in such scenarios can be challenging. To address this issue, we present a comparative analysis of 12 commonly used machine learning algorithms for human action intention recognition. The synthetic minority oversampling technique is applied to fill in missing data. Traversing all possible combinations would require conducting 686 experiments, which is a daunting task in terms of both cost and efficiency. To tackle this challenge, we employ an orthogonal experiment design based on the Quasi-horizontal method. Our analysis indicates that lightGBM outperforms other algorithms in recognizing eight human daily activities. Furthermore, we conduct a polar difference and variance analysis based on a comprehensive balanced multi-metric orthogonal experiment for lightGBM using various sensor combinations and dimensions. The optimal combinations of different sensor numbers in terms of position, channel, and dimension are derived using this approach. Notably, our experimental design reduces the number of experiments required to only 49 times.

A Novel Gait Phase Recognition Method Based on DPF-LSTM-CNN Using Wearable Inertial Sensors

Gait phase recognition is of great importance in the development of rehabilitation devices. The advantages of Long Short-Term Memory (LSTM) and Convolutional Neural Network (CNN) are combined (LSTM-CNN) in this paper, then a gait phase recognition method based on LSTM-CNN neural network model is proposed. In the LSTM-CNN model, the LSTM layer is used to process temporal sequences and the CNN layer is used to extract features A wireless sensor system including six inertial measurement units (IMU) fixed on the six positions of the lower limbs was developed. The difference in the gait recognition performance of the LSTM-CNN model was estimated using different groups of input data collected by seven different IMU grouping methods. Four phases in a complete gait were considered in this paper including the supporting phase with the right hill strike (SU-RHS), left leg swimming phase (SW-L), the supporting phase with the left hill strike (SU-LHS), and right leg swimming phase (SW-R). The results show that the best performance of the model in gait recognition appeared based on the group of data from all the six IMUs, with the recognition precision and macro-F1 unto 95.03% and 95.29%, respectively. At the same time, the best phase recognition accuracy for SU-RHS and SW-R appeared and up to 96.49% and 95.64%, respectively. The results also showed the best phase recognition accuracy (97.22%) for SW-L was acquired based on the group of data from four IMUs located at the left and right thighs and shanks. Comparably, the best phase recognition accuracy (97.86%) for SU-LHS was acquired based on the group of data from four IMUs located at left and right shanks and feet. Ulteriorly, a novel gait recognition method based on Data Pre-Filtering Long Short-Term Memory and Convolutional Neural Network (DPF-LSTM-CNN) model was proposed and its performance for gait phase recognition was evaluated. The experiment results showed that the recognition accuracy reached 97.21%, which was the highest compared to Deep convolutional neural networks (DCNN) and CNN-LSTM.

Observer-Based Output Feedback Event-Triggered Adaptive Control for Linear Multiagent Systems Under Switching Topologies

The consensus problem of general linear multiagent systems (MASs) is studied under switching topologies by using observer-based event-triggered control method in this article. On the basis of the output information of agents, two kinds of novel event-triggered adaptive control schemes are designed to achieve the leaderless and leader-follower consensus problems, which do not need to utilize the global information of the communication networks. Finally, two simulation examples are introduced to show that the consensus error converges to zero and Zeno behavior is eliminated in MASs. Compared with the existing output feedback control research, one of the significant advantages of our methods is that the controller protocols and triggering mechanisms do not rely on any global information, are independent of the network scale, and are fully distributed ways.

Non-invasive dual attention TCN for electromyography and motion data fusion in lower limb ambulation prediction

OBJECTIVE

Recent technological advances show the feasibility of fusing surface electromyography (sEMG) signals and movement data to predict lower limb ambulation intentions. However, since the invasive fusion of different signals is a major impediment to improving predictive performance, searching for a non-invasive fusion mechanism for lower limb ambulation pattern recognition based on different modal features is crucial.

APPROACH

We propose an end-to-end sequence prediction model with non-invasive dual attention temporal convolutional networks (NIDA-TCN) as a core to elegantly address the essential deficiencies of traditional decision models with heterogeneous signal fusion. Notably, the NIDA-TCN is a weighted fusion of sEMG and inertial measurement units (IMU) with time-dependent effective hidden information in the temporal and channel dimensions using TCN and self-attentive mechanisms. The new model can better discriminate between walking, jumping, downstairs, and upstairs four lower limb activities of daily living (ADL).

MAIN RESULTS

The results of this study show that the NIDA-TCN models produce predictions that significantly outperform both frame-wise and TCN models in terms of accuracy, sensitivity, precision, F1 score, and stability. Particularly, the NIDA-TCN with sequence decision fusion (NIDA-TCN-SDF) models, have maximum accuracy and stability increments of 3.37% and 4.95% relative to the frame-wise model, respectively, without manual feature-encoding and complex model parameters.

SIGNIFICANCE

It is concluded that the results demonstrate the validity and feasibility of the NIDA-TCN-SDF models to ensure the prediction of daily lower limb ambulation activities, paving the way to the development of fused heterogeneous signal decoding with better prediction performance.

Optimised EMG pipeline for gesture classification

In the expanding field of robotic prosthetics, surface electromyography (sEMG) signals can be decoded to seamlessly control a robotic prosthesis to perform the desired gesture. It is essential to create a pipeline, which can acquire, process, and accurately classify sEMG signals in order to replicate the desired hand gesture in near real-time and in a reliable manner. In this study, an optimised pipeline is proposed. This pipeline encompasses the main stages of sEMG signal processing and hand gesture classification and implements a sliding window approach, which is the main focus of the optimisation. In this study, a range of different parameters and modelling approaches are evaluated. The main contributions of this work are a robust and extensive analysis of sliding window parameter selection and an optimised pipeline that could be implemented in practice with minimal overheads. The optimum pipeline is efficient and achieves accurate prediction of hand gestures with an uninterrupted processing pipeline.

Detection of physical descaling damage in carp based on hyperspectral images and dimension reduction of principal component analysis combined with pixel values

The surface of carp is easily damaged during the descaling process, which jeopardizes the quality and safety of carp products. Damage recognition realized by manual detection is an important factor restricting the automation in the pretreatment. For the commonly used methods of mechanical and water-jet descaling, damage area recognition according to the hyperspectral data was proposed. Two discrimination models, including decision tree (DT) and self-organizing feature mapping (SOM), were established to recognize the damaged and normal descaling area with the average spectral value. The damage-discrimination model based on DT was determined to be the optimal one, which possessed the best model performance (accuracy = 96.7%, sensitivity = 96.7%, specificity = 96.7%, F1-score = 96.7%). Considering the efficiency and precision of damage-area recognition and visualization, the principal component analysis (PCA) combined with pixel values statistical analysis was used to reduce the dimension of hyperspectral images at the image level. Through statistical analysis, the value 0 was used as the threshold to distinguish the normal area and the damaged area in the PC image to achieve preliminary segmentation. Then, the spectral values of the initially discriminated damage area were input into the DT discrimination model to realize the final discriminant of damaged area. On this basis, the position information of the damaged area could be used to realize the visualization. The final visualization maps for mechanical and water-jet descaling damage were obtained by image morphology processing. The average recognition accuracy can reach 94.9% and 90.3%, respectively. The results revealed that the hyperspectral imaging technique has great potential to recognize the carp damage area nondestructively and accurately under descaling processing. PRACTICAL APPLICATION: This study demonstrated that hyperspectral imaging technique can realize the carp damage area detection nondestructively and accurately under descaling processing. With the advantages of nondestructive and rapid, hyperspectral imaging system and the method can be widely expanded and applied to the quality detection of other freshwater fish pretreatment.

Deep learning analysis based on multi-sensor fusion data for hemiplegia rehabilitation training system for stoke patients

By recognizing the motion of the healthy side, the lower limb exoskeleton robot can provide therapy to the affected side of stroke patients. To improve the accuracy of motion intention recognition based on sensor data, the research based on deep learning was carried out. Eighty healthy subjects performed gait experiments under five different gait environments (flat ground, 10 ${}^\circ$ upslope and downslope, and upstairs and downstairs) by simulating stroke patients. To facilitate the training and classification of the neural network, this paper presents template processing schemes to adapt to different data formats. The novel algorithm model of a hybrid network model based on convolutional neural network (CNN) and Long–short-term memory (LSTM) model is constructed. To mitigate the data-sparse problem, a spatial–temporal-embedded LSTM model (SQLSTM) combining spatial–temporal influence with the LSTM model is proposed. The proposed CNN-SQLSTM model is evaluated on a real trajectory dataset, and the results demonstrate the effectiveness of the proposed model. The proposed method will be used to guide the control strategy design of robot system for active rehabilitation training.

Statistical Features in High-Frequency Bands of Interictal iEEG Work Efficiently in Identifying the Seizure Onset Zone in Patients with Focal Epilepsy

The design of a computer-aided system for identifying the seizure onset zone (SOZ) from interictal and ictal electroencephalograms (EEGs) is desired by epileptologists. This study aims to introduce the statistical features of high-frequency components (HFCs) in interictal intracranial electroencephalograms (iEEGs) to identify the possible seizure onset zone (SOZ) channels. It is known that the activity of HFCs in interictal iEEGs, including ripple and fast ripple bands, is associated with epileptic seizures. This paper proposes to decompose multi-channel interictal iEEG signals into a number of subbands. For every 20 s segment, twelve features are computed from each subband. A mutual information (MI)-based method with grid search was applied to select the most prominent bands and features. A gradient-boosting decision tree-based algorithm called LightGBM was used to score each segment of the channels and these were averaged together to achieve a final score for each channel. The possible SOZ channels were localized based on the higher value channels. The experimental results with eleven epilepsy patients were tested to observe the efficiency of the proposed design compared to the state-of-the-art methods.