Joint segmentation of multivariate time series with hidden process regression for human activity recognition

Identification of recurrent dynamics in distributed neural populations

Large-scale recordings of neural activity over broad anatomical areas with high spatial and temporal resolution are increasingly common in modern experimental neuroscience. Recently, recurrent switching dynamical systems have been used to tackle the scale and complexity of these data. However, an important challenge remains in providing insights into the existence and structure of recurrent linear dynamics in neural time series data. Here we test a scalable approach to time-varying autoregression with low-rank tensors to recover the recurrent dynamics in stochastic neural mass models with multiple stable attractors. We demonstrate that the parsimonious representation of time-varying system matrices in terms of temporal modes can recover the attractor structure of simple systems via clustering. We then consider simulations based on a human brain connectivity matrix in high and low global connection strength regimes, and reveal the hierarchical clustering structure of the dynamics. Finally, we explain the impact of the forecast time delay on the estimation of the underlying rank and temporal variability of the time series dynamics. This study illustrates that prediction error minimization is not sufficient to recover meaningful dynamic structure and that it is crucial to account for the three key timescales arising from dynamics, noise processes, and attractor switching.

A Smartphone-Based sEMG Signal Analysis System for Human Action Recognition

In lower-limb rehabilitation, human action recognition (HAR) technology can be introduced to analyze the surface electromyography (sEMG) signal generated by movements, which can provide an objective and accurate evaluation of the patient's action. To balance the long cycle required for rehabilitation and the inconvenient factors brought by wearing sEMG devices, a portable sEMG signal acquisition device was developed that can be used under daily scenarios. Additionally, a mobile application was developed to meet the demand for real-time monitoring and analysis of sEMG signals. This application can monitor data in real time and has functions such as plotting, filtering, storage, and action capture and recognition. To build the dataset required for the recognition model, six lower-limb motions were developed for rehabilitation (kick, toe off, heel off, toe off and heel up, step back and kick, and full gait). The sEMG segment and action label were combined for training a convolutional neural network (CNN) to achieve high-precision recognition performance for human lower-limb actions (with a maximum accuracy of 97.96% and recognition accuracy for all actions reaching over 97%). The results show that the smartphone-based sEMG analysis system proposed in this paper can provide reliable information for the clinical evaluation of lower-limb rehabilitation.

Online detection of compensatory strategies in human movement with supervised classification: a pilot study

Introduction

Stroke survivors often compensate for the loss of motor function in their distal joints by altered use of more proximal joints and body segments. Since this can be detrimental to the rehabilitation process in the long-term, it is imperative that such movements are indicated to the patients and their caregiver. This is a difficult task since compensation strategies are varied and multi-faceted. Recent works that have focused on supervised machine learning methods for compensation detection often require a large training dataset of motions with compensation location annotations for each time-step of the recorded motion. In contrast, this study proposed a novel approach that learned a linear classifier from energy-based features to discriminate between healthy and compensatory movements and identify the compensating joints without the need for dense and explicit annotations.

Methods

Six healthy physiotherapists performed five different tasks using healthy movements and acted compensations. The resulting motion capture data was transformed into joint kinematic and dynamic trajectories. Inspired by works in bio-mechanics, energy-based features were extracted from this dataset. Support vector machine (SVM) and logistic regression (LR) algorithms were then applied for detection of compensatory movements. For compensating joint identification, an additional condition enforcing the independence of the feature calculation for each observable degree of freedom was imposed.

Results

Using leave-one-out cross validation, low values of mean brier score (<0.15), mis-classification rate (<0.2) and false discovery rate (<0.2) were obtained for both SVM and LR classifiers. These methods were found to outperform deep learning classifiers that did not use energy-based features. Additionally, online classification performance by our methods were also shown to outperform deep learning baselines. Furthermore, qualitative results obtained from the compensation joint identification experiment indicated that the method could successfully identify compensating joints.

Discussion

Results from this study indicated that including prior bio-mechanical information in the form of energy based features can improve classification performance even when linear classifiers are used, both for offline and online classification. Furthermore, evaluation compensation joint identification algorithm indicated that it could potentially provide a straightforward and interpretable way of identifying compensating joints, as well as the degree of compensation being performed.

Gait Recognition for Lower Limb Exoskeletons Based on Interactive Information Fusion

In recent decades, although the research on gait recognition of lower limb exoskeleton robot has been widely developed, there are still limitations in rehabilitation training and clinical practice. The emergence of interactive information fusion technology provides a new research idea for the solution of this problem, and it is also the development trend in the future. In order to better explore the issue, this paper summarizes gait recognition based on interactive information fusion of lower limb exoskeleton robots. This review introduces the current research status, methods, and directions for information acquisition, interaction, fusion, and gait recognition of exoskeleton robots. The content involves the research progress of information acquisition methods, sensor placements, target groups, lower limb sports biomechanics, interactive information fusion, and gait recognition model. Finally, the current challenges, possible solutions, and promising prospects are analysed and discussed, which provides a useful reference resource for the study of interactive information fusion and gait recognition of rehabilitation exoskeleton robots.

Human Motion State Recognition Based on Flexible, Wearable Capacitive Pressure Sensors

Human motion state recognition technology based on flexible, wearable sensor devices has been widely applied in the fields of human-computer interaction and health monitoring. In this study, a new type of flexible capacitive pressure sensor is designed and applied to the recognition of human motion state. The electrode layers use multi-walled carbon nanotubes (MWCNTs) as conductive materials, and polydimethylsiloxane (PDMS) with microstructures is embedded in the surface as a flexible substrate. A composite film of barium titanate (BaTiO3) with a high dielectric constant and low dielectric loss and PDMS is used as the intermediate dielectric layer. The sensor has the advantages of high sensitivity (2.39 kPa-1), wide pressure range (0-120 kPa), low pressure resolution (6.8 Pa), fast response time (16 ms), fast recovery time (8 ms), lower hysteresis, and stability. The human body motion state recognition system is designed based on a multi-layer back propagation neural network, which can collect, process, and recognize the sensor signals of different motion states (sitting, standing, walking, and running). The results indicate that the overall recognition rate of the system for the human motion state reaches 94%. This proves the feasibility of the human motion state recognition system based on the flexible wearable sensor. Furthermore, the system has high application potential in the field of wearable motion detection.

Approximations of conditional probability density functions in Lebesgue spaces via mixture of experts models

Mixture of experts (MoE) models are widely applied for conditional probability density estimation problems. We demonstrate the richness of the class of MoE models by proving denseness results in Lebesgue spaces, when inputs and outputs variables are both compactly supported. We further prove an almost uniform convergence result when the input is univariate. Auxiliary lemmas are proved regarding the richness of the soft-max gating function class, and their relationships to the class of Gaussian gating functions.

Powered Two-Wheeler Riding Profile Clustering for an In-Depth Study of Bend-Taking Practices

The understanding of rider/vehicle interaction modalities remains an issue, specifically in the case of bend-taking. This difficulty results both from the lack of adequate instrumentation to conduct this type of study and from the variety of practices of this population of road users. Riders have numerous explanations of strategies for controlling their motorcycles when taking bends. The objective of this paper is to develop a data-driven methodology in order to identify typical riding behaviors in bends by using clustering methods. The real dataset used for the experiments is collected within the VIROLO++ collaborative project to improve the knowledge of actual PTW riding practices, especially during bend taking, by collecting real data on this riding situation, including data on PTW dynamics (velocity, normal acceleration, and jerk), position on the road (road curvature), and handlebar actions (handlebar steering angle). A detailed analysis of the results is provided for both the Anderson–Darling test and clustering steps. Moreover, the clustering results are compared with the subjective data of subjects to highlight and contextualize typical riding tendencies. Finally, we perform an in-depth analysis of the bend-taking practices of one subject to highlight the differences between different methods of controlling the motorcycle (steering handlebar vs. rider’s lean) using the rider action measurements made by pressure sensors.

Identifying Qualitative Between-Subject and Within-Subject Variability: A Method for Clustering Regime-Switching Dynamics

Technological advancement provides an unprecedented amount of high-frequency data of human dynamic processes. In this paper, we introduce an approach for characterizing qualitative between and within-subject variability from quantitative changes in the multi-subject time-series data. We present the statistical model and examine the strengths and limitations of the approach in potential applications using Monte Carlo simulations. We illustrate its usage in characterizing clusters of dynamics with phase transitions with real-time hand movement data collected on an embodied learning platform designed to foster mathematical learning.

Bayesian Detection of Piecewise Linear Trends in Replicated Time-Series with Application to Growth Data Modelling

We consider the situation where a temporal process is composed of contiguous segments with differing slopes and replicated noise-corrupted time series measurements are observed. The unknown mean of the data generating process is modelled as a piecewise linear function of time with an unknown number of change-points. We develop a Bayesian approach to infer the joint posterior distribution of the number and position of change-points as well as the unknown mean parameters. A-priori, the proposed model uses an overfitting number of mean parameters but, conditionally on a set of change-points, only a subset of them influences the likelihood. An exponentially decreasing prior distribution on the number of change-points gives rise to a posterior distribution concentrating on sparse representations of the underlying sequence. A Metropolis-Hastings Markov chain Monte Carlo (MCMC) sampler is constructed for approximating the posterior distribution. Our method is benchmarked using simulated data and is applied to uncover differences in the dynamics of fungal growth from imaging time course data collected from different strains. The source code is available on CRAN.

Adaptive, locally linear models of complex dynamics

The dynamics of complex systems generally include high-dimensional, nonstationary, and nonlinear behavior, all of which pose fundamental challenges to quantitative understanding. To address these difficulties, we detail an approach based on local linear models within windows determined adaptively from data. While the dynamics within each window are simple, consisting of exponential decay, growth, and oscillations, the collection of local parameters across all windows provides a principled characterization of the full time series. To explore the resulting model space, we develop a likelihood-based hierarchical clustering, and we examine the eigenvalues of the linear dynamics. We demonstrate our analysis with the Lorenz system undergoing stable spiral dynamics and in the standard chaotic regime. Applied to the posture dynamics of the nematode Caenorhabditis elegans, our approach identifies fine-grained behavioral states and model dynamics which fluctuate about an instability boundary, and we detail a bifurcation in a transition from forward to backward crawling. We analyze whole-brain imaging in C. elegans and show that global brain dynamics is damped away from the instability boundary by a decrease in oxygen concentration. We provide additional evidence for such near-critical dynamics from the analysis of electrocorticography in monkey and the imaging of a neural population from mouse visual cortex at single-cell resolution.

Real-Time Human Physical Activity Recognition with Low Latency Prediction Feedback Using Raw IMU Data

In the realm of Human Activity Recognition (HAR), supervised machine learning and deep learning are commonly used. Their training is done using time and frequency features extracted from raw data (inertial and gyroscopic). Nevertheless, raw data are seldom employed. In this paper, a dataset of able-bodied participants is recorded using 3 custom wireless motion sensors providing embedded IMU and sEMG detection and processing and a base station (a Raspberry Pi 3) running a classification algorithm. A Support Vector Machine with Radius Basis Function Kernel (RBF-SVM) is augmented using Spherical Normalization to achieve a motion classification accuracy of 97.35% between 8 body motions. The proposed classifier allows for real-time prediction callback with low latency output.

Deep learning-based classification with improved time resolution for physical activities of children

Background

The proportion of overweight and obese people has increased tremendously in a short period, culminating in a worldwide trend of obesity that is reaching epidemic proportions. Overweight and obesity are serious issues, especially with regard to children. This is because obese children have twice the risk of becoming obese as adults, as compared to non-obese children. Nowadays, many methods for maintaining a caloric balance exist; however, these methods are not applicable to children. In this study, a new approach for helping children monitor their activities using a convolutional neural network (CNN) is proposed, which is applicable for real-time scenarios requiring high accuracy.

Methods

A total of 136 participants (86 boys and 50 girls), aged between 8.5 years and 12.5 years (mean 10.5, standard deviation 1.1), took part in this study. The participants performed various movement while wearing custom-made three-axis accelerometer modules around their waists. The data acquired by the accelerometer module was preprocessed by dividing them into small sets (128 sample points for 2.8 s). Approximately 183,600 data samples were used by the developed CNN for learning to classify ten physical activities : slow walking, fast walking, slow running, fast running, walking up the stairs, walking down the stairs, jumping rope, standing up, sitting down, and remaining still.

Results

The developed CNN classified the ten activities with an overall accuracy of 81.2%. When similar activities were merged, leading to seven merged activities, the CNN classified activities with an overall accuracy of 91.1%. Activity merging also improved performance indicators, for the maximum case of 66.4% in recall, 48.5% in precision, and 57.4% in f1 score . The developed CNN classifier was compared to conventional machine learning algorithms such as the support vector machine, decision tree, and k-nearest neighbor algorithms, and the proposed CNN classifier performed the best: CNN (81.2%) > SVM (64.8%) > DT (63.9%) > kNN (55.4%) (for ten activities); CNN (91.1%) > SVM (74.4%) > DT (73.2%) > kNN (65.3%) (for the merged seven activities).

Discussion

The developed algorithm distinguished physical activities with improved time resolution using short-time acceleration signals from the physical activities performed by children. This study involved algorithm development, participant recruitment, IRB approval, custom-design of a data acquisition module, and data collection. The self-selected moving speeds for walking and running (slow and fast) and the structure of staircase degraded the performance of the algorithm. However, after similar activities were merged, the effects caused by the self-selection of speed were reduced. The experimental results show that the proposed algorithm performed better than conventional algorithms. Owing to its simplicity, the proposed algorithm could be applied to real-time applicaitons.

A Review of Wearable Technologies for Elderly Care that Can Accurately Track Indoor Position, Recognize Physical Activities and Monitor Vital Signs in Real Time

Rapid growth of the aged population has caused an immense increase in the demand for healthcare services. Generally, the elderly are more prone to health problems compared to other age groups. With effective monitoring and alarm systems, the adverse effects of unpredictable events such as sudden illnesses, falls, and so on can be ameliorated to some extent. Recently, advances in wearable and sensor technologies have improved the prospects of these service systems for assisting elderly people. In this article, we review state-of-the-art wearable technologies that can be used for elderly care. These technologies are categorized into three types: indoor positioning, activity recognition and real time vital sign monitoring. Positioning is the process of accurate localization and is particularly important for elderly people so that they can be found in a timely manner. Activity recognition not only helps ensure that sudden events (e.g., falls) will raise alarms but also functions as a feasible way to guide people’s activities so that they avoid dangerous behaviors. Since most elderly people suffer from age-related problems, some vital signs that can be monitored comfortably and continuously via existing techniques are also summarized. Finally, we discussed a series of considerations and future trends with regard to the construction of “smart clothing” system.

Physical Human Activity Recognition Using Wearable Sensors

This paper presents a review of different classification techniques used to recognize human activities from wearable inertial sensor data. Three inertial sensor units were used in this study and were worn by healthy subjects at key points of upper/lower body limbs (chest, right thigh and left ankle). Three main steps describe the activity recognition process: sensors’ placement, data pre-processing and data classification. Four supervised classification techniques namely, k-Nearest Neighbor (k-NN), Support Vector Machines (SVM), Gaussian Mixture Models (GMM), and Random Forest (RF) as well as three unsupervised classification techniques namely, k-Means, Gaussian mixture models (GMM) and Hidden Markov Model (HMM), are compared in terms of correct classification rate, F-measure, recall, precision, and specificity. Raw data and extracted features are used separately as inputs of each classifier. The feature selection is performed using a wrapper approach based on the RF algorithm. Based on our experiments, the results obtained show that the k-NN classifier provides the best performance compared to other supervised classification algorithms, whereas the HMM classifier is the one that gives the best results among unsupervised classification algorithms. This comparison highlights which approach gives better performance in both supervised and unsupervised contexts. It should be noted that the obtained results are limited to the context of this study, which concerns the classification of the main daily living human activities using three wearable accelerometers placed at the chest, right shank and left ankle of the subject.

Segmentation of biological multivariate time-series data

Time-series data from multicomponent systems capture the dynamics of the ongoing processes and reflect the interactions between the components. The progression of processes in such systems usually involves check-points and events at which the relationships between the components are altered in response to stimuli. Detecting these events together with the implicated components can help understand the temporal aspects of complex biological systems. Here we propose a regularized regression-based approach for identifying breakpoints and corresponding segments from multivariate time-series data. In combination with techniques from clustering, the approach also allows estimating the significance of the determined breakpoints as well as the key components implicated in the emergence of the breakpoints. Comparative analysis with the existing alternatives demonstrates the power of the approach to identify biologically meaningful breakpoints in diverse time-resolved transcriptomics data sets from the yeast Saccharomyces cerevisiae and the diatom Thalassiosira pseudonana.

Improving human activity recognition and its application in early stroke diagnosis

The development of efficient stroke-detection methods is of significant importance in today's society due to the effects and impact of stroke on health and economy worldwide. This study focuses on Human Activity Recognition (HAR), which is a key component in developing an early stroke-diagnosis tool. An overview of the proposed global approach able to discriminate normal resting from stroke-related paralysis is detailed. The main contributions include an extension of the Genetic Fuzzy Finite State Machine (GFFSM) method and a new hybrid feature selection (FS) algorithm involving Principal Component Analysis (PCA) and a voting scheme putting the cross-validation results together. Experimental results show that the proposed approach is a well-performing HAR tool that can be successfully embedded in devices.