Evaluation of accelerometer based multi-sensor versus single-sensor activity recognition systems

Inertial measurement data from loose clothing worn on the lower body during everyday activities

Embedding sensors into clothing is promising as a way for people to wear multiple sensors easily, for applications such as long-term activity monitoring. To our knowledge, this is the first published dataset collected from sensors in loose clothing. 6 Inertial Measurement Units (IMUs) were configured as a 'sensor string' and attached to casual trousers such that there were three sensors on each leg near the waist, thigh, and ankle/lower-shank. Participants also wore an Actigraph accelerometer on their dominant wrist. The dataset consists of 15 participant-days worth of data collected from 5 healthy adults (age range: 28-48 years, 3 males and 2 females). Each participant wore the clothes with sensors for between 1 and 4 days for 5-8 hours per day. Each day, data were collected while participants completed a fixed circuit of activities (with a video ground truth) as well as during free day-to-day activities (with a diary). This dataset can be used to analyse human movements, transitional movements, and postural changes based on a range of features.

Fall Recognition Based on an IMU Wearable Device and Fall Verification through a Smart Speaker and the IoT

A fall is one of the most devastating events that aging people can experience. Fall-related physical injuries, hospital admission, or even mortality among the elderly are all critical health issues. As the population continues to age worldwide, there is an imperative need to develop fall detection systems. We propose a system for the recognition and verification of falls based on a chest-worn wearable device, which can be used for elderly health institutions or home care. The wearable device utilizes a built-in three-axis accelerometer and gyroscope in the nine-axis inertial sensor to determine the user's postures, such as standing, sitting, and lying down. The resultant force was obtained by calculation with three-axis acceleration. Integration of three-axis acceleration and a three-axis gyroscope can obtain a pitch angle through the gradient descent algorithm. The height value was converted from a barometer. Integration of the pitch angle with the height value can determine the behavior state including sitting down, standing up, walking, lying down, and falling. In our study, we can clearly determine the direction of the fall. Acceleration changes during the fall can determine the force of the impact. Furthermore, with the IoT (Internet of Things) and smart speakers, we can verify whether the user has fallen by asking from smart speakers. In this study, posture determination is operated directly on the wearable device through the state machine. The ability to recognize and report a fall event in real-time can help to lessen the response time of a caregiver. The family members or care provider monitor, in real-time, the user's current posture via a mobile device app or internet webpage. All collected data supports subsequent medical evaluation and further intervention.

Comparing Loose Clothing-Mounted Sensors with Body-Mounted Sensors in the Analysis of Walking

A person's walking pattern can reveal important information about their health. Mounting multiple sensors onto loose clothing potentially offers a comfortable way of collecting data about walking and other human movement. This research investigates how well the data from three sensors mounted on the lateral side of clothing (on a pair of trousers near the waist, upper thigh and lower shank) correlate with the data from sensors mounted on the frontal side of the body. Data collected from three participants (two male, one female) for two days were analysed. Gait cycles were extracted based on features in the lower-shank accelerometry and analysed in terms of sensor-to-vertical angles (SVA). The correlations in SVA between the clothing- and body-mounted sensor pairs were analysed. Correlation coefficients above 0.76 were found for the waist sensor pairs, while the thigh and lower-shank sensor pairs had correlations above 0.90. The cyclical nature of gait cycles was evident in the clothing data, and it was possible to distinguish the stance and swing phases of walking based on features in the clothing data. Furthermore, simultaneously recording data from the waist, thigh, and shank was helpful in capturing the movement of the whole leg.

Monitoring Arm Movements Post-Stroke for Applications in Rehabilitation and Home Settings

Optimal recovery of arm function following stroke requires patients to perform a large number of functional arm movements in clinical therapy sessions, as well as at home. Technology to monitor adherence to this activity would be helpful to patients and clinicians. Current approaches to monitoring arm movements are limited because of challenges in distinguishing between functional and non-functional movements. Here, we present an Arm Rehabilitation Monitor (ARM), a device intended to make such measurements in an unobtrusive manner. The ARM device is based on a single Inertial Measurement Unit (IMU) worn on the wrist and uses machine learning techniques to interpret the resulting signals. We characterized the ability of the ARM to detect reaching actions in a functional assessment dataset (functional assessment tasks) and an Activities-of-Daily-Living (ADL) dataset (pizza-making and walking task) from 12 participants with stroke. The Convolutional Neural Network (CNN) and Random Forests (RF) classifiers had a Matthews Correlation Coefficient score of 0.59 and 0.58 when trained and tested on the functional dataset, 0.50 and 0.49 when trained and tested on the ADL dataset, and 0.37 and 0.36 when trained on the functional dataset and tested on the ADL dataset, respectively. The latter is the most relevant scenario for the intended application of training during a clinical visit for monitoring movements in the in-home setting. The classifiers showed good performance in estimating the time spent reaching and number of reaching gestures and showed low sensitivity to irrelevant arm movements produced during walking. We conclude that the ARM has sufficient accuracy and robustness to merit being used in preliminary studies to monitor arm activity in rehabilitation or home applications.

Machine Learning-Based Predicted Age of the Elderly on the Instrumented Timed Up and Go Test and Six-Minute Walk Test

A decrease in dynamic balance ability (DBA) in the elderly is closely associated with aging. Various studies have investigated different methods to quantify the DBA in the elderly through DBA evaluation methods such as the timed up and go test (TUG) and the six-minute walk test (6MWT), applying the G-Walk wearable system. However, these methods have generally been difficult for the elderly to intuitively understand. The goal of this study was thus to generate a regression model based on machine learning (ML) to predict the age of the elderly as a familiar indicator. The model was based on inertial measurement unit (IMU) data as part of the DBA evaluation, and the performance of the model was comparatively analyzed with respect to age prediction based on the IMU data of the TUG test and the 6MWT. The DBA evaluation used the TUG test and the 6MWT performed by 136 elderly participants. When performing the TUG test and the 6MWT, a single IMU was attached to the second lumbar spine of the participant, and the three-dimensional linear acceleration and gyroscope data were collected. The features used in the ML-based regression model included the gait symmetry parameters and the harmonic ratio applied in quantifying the DBA, in addition to the features of description statistics for IMU signals. The feature set was differentiated between the TUG test and the 6MWT, and the performance of the regression model was comparatively analyzed based on the feature sets. The XGBoost algorithm was used to train the regression model. Comparison of the regression model performance according to the TUG test and 6MWT feature sets showed that the performance was best for the model using all features of the TUG test and the 6MWT. This indicated that the evaluation of DBA in the elderly should apply the TUG test and the 6MWT concomitantly for more accurate predictions. The findings in this study provide basic data for the development of a DBA monitoring system for the elderly.

A comparative study on recognizing human activities by applying diverse Machine Learning approaches

This paper deals with the problem of identifying and recognizing everyday human activities. The main goal is to compare a variety of implemented classification models founded on diverse machine learning approaches; one that utilizes features extracted from the time and frequency domain and three others that take advantage of the attributes of the symbolic space in order to extract conclusions regarding the performance and the potential usefulness of each of them. To guarantee the impartiality of the comparison, we used the signals contained in a free accessible dataset, which are subjected to the same preprocessing, and divided into equal time-length windows. The Nearest Neighour classifier is applied to compare the four approaches.

Semi-Supervised Adversarial Learning Using LSTM for Human Activity Recognition

The training of Human Activity Recognition (HAR) models requires a substantial amount of labeled data. Unfortunately, despite being trained on enormous datasets, most current models have poor performance rates when evaluated against anonymous data from new users. Furthermore, due to the limits and problems of working with human users, capturing adequate data for each new user is not feasible. This paper presents semi-supervised adversarial learning using the LSTM (Long-short term memory) approach for human activity recognition. This proposed method trains annotated and unannotated data (anonymous data) by adapting the semi-supervised learning paradigms on which adversarial learning capitalizes to improve the learning capabilities in dealing with errors that appear in the process. Moreover, it adapts to the change in human activity routine and new activities, i.e., it does not require prior understanding and historical information. Simultaneously, this method is designed as a temporal interactive model instantiation and shows the capacity to estimate heteroscedastic uncertainty owing to inherent data ambiguity. Our methodology also benefits from multiple parallel input sequential data predicting an output exploiting the synchronized LSTM. The proposed method proved to be the best state-of-the-art method with more than 98% accuracy in implementation utilizing the publicly available datasets collected from the smart home environment facilitated with heterogeneous sensors. This technique is a novel approach for high-level human activity recognition and is likely to be a broad application prospect for HAR.

Validity of trunk acceleration measurement with a chest-worn monitor for assessment of physical activity intensity

Background

Recent advancements in wearable technology have enabled easy measurement of daily activities, potentially applicable in rehabilitation practice for various purposes such as maintaining and increasing patients’ activity levels. In this study, we aimed to examine the validity of trunk acceleration measurement using a chest monitor embedded in a smart clothing system (‘hitoe’ system), an emerging wearable system, in assessing the physical activity in an experimental setting with healthy subjects (Study 1) and in a clinical setting with post-stroke patients (Study 2).

Methods

Study 1 involved the participation of 14 healthy individuals. The trunk acceleration, heart rate (HR), and oxygen consumption were simultaneously measured during treadmill testing with a Bruce protocol. Trunk acceleration and HR were measured using the "hitoe" system, a smart clothing system with embedded chest sensors. Expiratory gas analysis was performed to measure oxygen consumption. Three parameters, moving average (MA), moving standard deviation (MSD), and moving root mean square (RMS), were calculated from the norm of the trunk acceleration. The relationships between these accelerometer-based parameters and oxygen consumption-based physical activity intensity measured with the percent VO2 reserve (%VO₂R) were examined. In Study 2, 48 h of simultaneous measurement of trunk acceleration and heart rate-based physical activity intensity in terms of percent heart rate reserve (%HRR) was conducted with the "hitoe" system in 136 post-stroke patients.

Results

The values of MA, MSD, RMS, and %VO₂R were significantly different between levels 1, 2, 3, and 4 in the Bruce protocol (P < 0.01). The average coefficients of determination for individual regression for %VO₂R versus MA, %VO₂R versus MSD, and %VO₂R versus RMS were 0.89 ± 0.05, 0.96 ± 0.03, and 0.91 ± 0.05, respectively. Among the parameters examined, MSD showed the best correlation with %VO₂R, indicating high validity of the parameter for assessing physical activity intensity. The 48-h measurement of MSD and %HRR in post-stroke patients showed significant within-individual correlation (P < 0.05) in 131 out of 136 patients (correlation coefficient: 0.60 ± 0.16).

Conclusions

The results support the validity of the MSD calculated from the trunk acceleration measured with a smart clothing system in assessing the physical activity intensity.

Trial registration: UMIN000034967. Registered 21 November 2018 (retrospectively registered).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13102-022-00492-4.

LIPSHOK: LIARA Portable Smart Home Kit

Several smart home architecture implementations have been proposed in the last decade. These architectures are mostly deployed in laboratories or inside real habitations built for research purposes to enable the use of ambient intelligence using a wide variety of sensors, actuators and machine learning algorithms. However, the major issues for most related smart home architectures are their price, proprietary hardware requirements and the need for highly specialized personnel to deploy such systems. To tackle these challenges, lighter forms of smart home architectures known as smart homes in a box (SHiB) have been proposed. While SHiB remain an encouraging first step towards lightweight yet affordable solutions, they still suffer from few drawbacks. Indeed, some of these kits lack hardware support for some technologies, and others do not include enough sensors and actuators to cover most smart homes' requirements. Thus, this paper introduces the LIARA Portable Smart Home Kit (LIPSHOK). It has been designed to provide an affordable SHiB solution that anyone is able to install in an existing home. Moreover, LIPSHOK is a generic kit that includes a total of four specialized sensor modules that were introduced independently, as our laboratory has been working on their development over the last few years. This paper first provides a summary of each of these modules and their respective benefits within a smart home context. Then, it mainly focus on the introduction of the LIPSHOK architecture that provides a framework to unify the use of the proposed sensors thanks to a common modular infrastructure capable of managing heterogeneous technologies. Finally, we compare our work to the existing SHiB kit solutions and outline that it offers a more affordable, extensible and scalable solution whose resources are distributed under an open-source license.

Gait Recognition for Lower Limb Exoskeletons Based on Interactive Information Fusion

In recent decades, although the research on gait recognition of lower limb exoskeleton robot has been widely developed, there are still limitations in rehabilitation training and clinical practice. The emergence of interactive information fusion technology provides a new research idea for the solution of this problem, and it is also the development trend in the future. In order to better explore the issue, this paper summarizes gait recognition based on interactive information fusion of lower limb exoskeleton robots. This review introduces the current research status, methods, and directions for information acquisition, interaction, fusion, and gait recognition of exoskeleton robots. The content involves the research progress of information acquisition methods, sensor placements, target groups, lower limb sports biomechanics, interactive information fusion, and gait recognition model. Finally, the current challenges, possible solutions, and promising prospects are analysed and discussed, which provides a useful reference resource for the study of interactive information fusion and gait recognition of rehabilitation exoskeleton robots.

Sensor-Based Gym Physical Exercise Recognition: Data Acquisition and Experiments

Automatic tracking and quantification of exercises not only helps in motivating people but also contributes towards improving health conditions. Weight training, in addition to aerobic exercises, is an important component of a balanced exercise program. Excellent trackers are available for aerobic exercises but, in contrast, tracking free weight exercises is still performed manually. This study presents the details of our data acquisition effort using a single chest-mounted tri-axial accelerometer, followed by a novel method for the recognition of a wide range of gym-based free weight exercises. Exercises are recognized using LSTM neural networks and the reported results confirm the feasibility of the proposed approach. We train and test several LSTM-based gym exercise recognition models. More specifically, in one set of experiments, we experiment with separate models, one for each muscle group. In another experiment, we develop a universal model for all exercises. We believe that the promising results will potentially contribute to the vision of an automated system for comprehensive monitoring and analysis of gym-based exercises and create a new experience for exercising by freeing the exerciser from manual record-keeping.

Real-time motion onset recognition for robot-assisted gait rehabilitation

BACKGROUND

Many patients with neurological movement disorders fear to fall while performing postural transitions without assistance, which prevents them from participating in daily life. To overcome this limitation, multi-directional Body Weight Support (BWS) systems have been developed allowing them to perform training in a safe environment. In addition to overground walking, these innovative/novel systems can assist patients to train many more gait-related tasks needed for daily life under very realistic conditions. The necessary assistance during the users' movements can be provided via task-dependent support designs. One remaining challenge is the manual switching between task-dependent supports. It is error-prone, cumbersome, distracts therapists and patients, and interrupts the training workflow. Hence, we propose a real-time motion onset recognition model that performs automatic support switching between standing-up and sitting-down transitions and other gait-related tasks (8 classes in total).

METHODS

To predict the onsets of the gait-related tasks, three Inertial Measurement Units (IMUs) were attached to the sternum and middle of outer thighs of 19 controls without neurological movement disorders and two individuals with incomplete Spinal Cord Injury (iSCI). The data of IMUs obtained from different gait tasks was sent synchronously to a real-time data acquisition system through a custom-made Bluetooth-EtherCAT gateway. In the first step, data was applied offline for training five different classifiers. The best classifier was chosen based on F1-score results of a Leave-One-Participant-Out Cross-Validation (LOPOCV), which is an unbiased way of testing. In a final step, the chosen classifier was tested in real time with an additional control participant to demonstrate feasibility for real-time classification.

RESULTS

Testing five different classifiers, the best performance was obtained in a single-layer neural network with 25 neurons. The F1-score of [Formula: see text] and [Formula: see text] are achieved on testing using LOPOCV and test data ([Formula: see text], participants = 20), respectively. Furthermore, the results from the implemented real-time classifier were compared with the offline classifier and revealed nearly identical performance (difference = [Formula: see text]).

CONCLUSIONS

A neural network classifier was trained for identifying the onset of gait-related tasks in real time. Test data showed convincing performance for offline and real-time classification. This demonstrates the feasibility and potential for implementing real-time onset recognition in rehabilitation devices in future.

Predicting atrial fibrillation episodes with rapid ventricular rates associated with low levels of activity

BACKGROUND

Rapid and irregular ventricular rates (RVR) are an important consequence of atrial fibrillation (AF). Raw accelerometry data in combination with electrocardiogram (ECG) data have the potential to distinguish inappropriate from appropriate tachycardia in AF. This can allow for the development of a just-in-time intervention for clinical treatments of AF events. The objective of this study is to develop a machine learning algorithm that can distinguish episodes of AF with RVR that are associated with low levels of activity.

METHODS

This study involves 45 patients with persistent or paroxysmal AF. The ECG and accelerometer data were recorded continuously for up to 3 weeks. The prediction of AF episodes with RVR and low activity was achieved using a deterministic probabilistic finite-state automata (DPFA)-based approach. Rapid and irregular ventricular rate (RVR) is defined as having heart rates (HR) greater than 110 beats per minute (BPM) and high activity is defined as greater than 0.75 quantile of the activity level. The AF events were annotated using the FDA-cleared BeatLogic algorithm. Various time intervals prior to the events were used to determine the longest prediction intervals for predicting AF with RVR episodes associated with low levels of activity.

RESULTS

Among the 961 annotated AF events, 292 met the criterion for RVR episode. There were 176 and 116 episodes with low and high activity levels respectively. Out of the 961 AF episodes, 770 (80.1%) were used in the training data set and the remaining 191 intervals were held out for testing. The model was able to predict AF with RVR and low activity up to 4.5 min before the events. The mean prediction performance gradually decreased as the time to events increased. The overall Area under the ROC Curve (AUC) for the model lies within the range of 0.67-0.78.

CONCLUSION

The DPFA algorithm can predict AF with RVR associated with low levels of activity up to 4.5 min before the onset of the event. This would enable the development of just-in-time interventions that could reduce the morbidity and mortality associated with AF and other similar arrhythmias.

HARTH: A Human Activity Recognition Dataset for Machine Learning

Existing accelerometer-based human activity recognition (HAR) benchmark datasets that were recorded during free living suffer from non-fixed sensor placement, the usage of only one sensor, and unreliable annotations. We make two contributions in this work. First, we present the publicly available Human Activity Recognition Trondheim dataset (HARTH). Twenty-two participants were recorded for 90 to 120 min during their regular working hours using two three-axial accelerometers, attached to the thigh and lower back, and a chest-mounted camera. Experts annotated the data independently using the camera's video signal and achieved high inter-rater agreement (Fleiss' Kappa =0.96). They labeled twelve activities. The second contribution of this paper is the training of seven different baseline machine learning models for HAR on our dataset. We used a support vector machine, k-nearest neighbor, random forest, extreme gradient boost, convolutional neural network, bidirectional long short-term memory, and convolutional neural network with multi-resolution blocks. The support vector machine achieved the best results with an F1-score of 0.81 (standard deviation: ±0.18), recall of 0.85±0.13, and precision of 0.79±0.22 in a leave-one-subject-out cross-validation. Our highly professional recordings and annotations provide a promising benchmark dataset for researchers to develop innovative machine learning approaches for precise HAR in free living.

Design of a seniors and Alzheimer's disease caring service platform

Background

To meet the needs of aging and dementia patients in Taiwan, this study designed a nursing system that includes communication, location tracking, and fall detection, and early warning services. The main purpose of this research is to provide timely services to the elderly and patients and hope to reduce the burden when the number of nursing staff decreases. This article is a remote disease care service platform with the Internet of Things (IoT) devices to monitor the location of the elderly and whether they have dropped warning alerts.

Results

The device is connected to the patient's waist and chest, monitors the patient's movement and behavior, and transmits messages to the back-end system, and informs caregivers through mobile phone applications when unexpected or shocking events occur. The system can identify whether the patient has fallen, accidentally, or long-term inactivity. The device is equipped with sensors that enable it to monitor the patient's location and behavior data through Bluetooth and GPS technology. Finally, we proposed a basic model and an integrated model that will industrialize the system and is expected to play a role in a larger patient population.

Conclusions

The system developed in this research has passed the Activities of Daily Living (ADL) test and verification, and is expected to provide appropriate safety care services for nursing homes and elderly residences.

Human Activity Recognition: A Dynamic Inductive Bias Selection Perspective

In this article, we study activity recognition in the context of sensor-rich environments. In these environments, many different constraints arise at various levels during the data generation process, such as the intrinsic characteristics of the sensing devices, their energy and computational constraints, and their collective (collaborative) dimension. These constraints have a fundamental impact on the final activity recognition models as the quality of the data, its availability, and its reliability, among other things, are not ensured during model deployment in real-world configurations. Current approaches for activity recognition rely on the activity recognition chain which defines several steps that the sensed data undergo: This is an inductive process that involves exploring a hypothesis space to find a theory able to explain the observations. For activity recognition to be effective and robust, this inductive process must consider the constraints at all levels and model them explicitly. Whether it is a bias related to sensor measurement, transmission protocol, sensor deployment topology, heterogeneity, dynamicity, or stochastic effects, it is essential to understand their substantial impact on the quality of the data and ultimately on activity recognition models. This study highlights the need to exhibit the different types of biases arising in real situations so that machine learning models, e.g., can adapt to the dynamicity of these environments, resist sensor failures, and follow the evolution of the sensors’ topology. We propose a metamodeling approach in which these biases are specified as hyperparameters that can control the structure of the activity recognition models. Via these hyperparameters, it becomes easier to optimize the inductive processes, reason about them, and incorporate additional knowledge. It also provides a principled strategy to adapt the models to the evolutions of the environment. We illustrate our approach on the SHL dataset, which features motion sensor data for a set of human activities collected in real conditions. The obtained results make a case for the proposed metamodeling approach; noticeably, the robustness gains achieved when the deployed models are confronted with the evolution of the initial sensing configurations. The trade-offs exhibited and the broader implications of the proposed approach are discussed with alternative techniques to encode and incorporate knowledge into activity recognition models.

OpiTrack: A Wearable-based Clinical Opioid Use Tracker with Temporal Convolutional Attention Networks

Opioid use disorder is a medical condition with major social and economic consequences. While ubiquitous physiological sensing technologies have been widely adopted and extensively used to monitor day-to-day activities and deliver targeted interventions to improve human health, the use of these technologies to detect drug use in natural environments has been largely underexplored. The long-term goal of our work is to develop a mobile technology system that can identify high-risk opioid-related events (i.e., development of tolerance in the setting of prescription opioid use, return-to-use events in the setting of opioid use disorder) and deploy just-in-time interventions to mitigate the risk of overdose morbidity and mortality. In the current paper, we take an initial step by asking a crucial question: Can opioid use be detected using physiological signals obtained from a wrist-mounted sensor? Thirty-six individuals who were admitted to the hospital for an acute painful condition and received opioid analgesics as part of their clinical care were enrolled. Subjects wore a noninvasive wrist sensor during this time (1-14 days) that continuously measured physiological signals (heart rate, skin temperature, accelerometry, electrodermal activity, and interbeat interval). We collected a total of 2070 hours (≈ 86 days) of physiological data and observed a total of 339 opioid administrations. Our results are encouraging and show that using a Channel-Temporal Attention TCN (CTA-TCN) model, we can detect an opioid administration in a time-window with an F1-score of 0.80, a specificity of 0.77, sensitivity of 0.80, and an AUC of 0.77. We also predict the exact moment of administration in this time-window with a normalized mean absolute error of 8.6% and R ² coefficient of 0.85.

Using a Stacked Autoencoder for Mobility and Fall Risk Assessment via Time-Frequency Representations of the Timed Up and Go Test

Fall risk assessment is very important for the graying societies of developed countries. A major contributor to the fall risk of the elderly is mobility impairment. Timely detection of the fall risk can facilitate early intervention to avoid preventable falls. However, continuous fall risk monitoring requires extensive healthcare and clinical resources. Our objective is to develop a method suitable for remote and long-term health monitoring of the elderly for mobility impairment and fall risk without the need for an expert. We employed time–frequency analysis (TFA) and a stacked autoencoder (SAE), which is a deep neural network (DNN)-based learning algorithm, to assess the mobility and fall risk of the elderly according to the criteria of the timed up and go test (TUG). The time series signal of the triaxial accelerometer can be transformed by TFA to obtain richer image information. On the basis of the TUG criteria, the semi-supervised SAE model was able to achieve high predictive accuracies of 89.1, 93.4, and 94.1% for the vertical, mediolateral and anteroposterior axes, respectively. We believe that deep learning can be used to analyze triaxial acceleration data, and our work demonstrates its applicability to assessing the mobility and fall risk of the elderly.

The Effect of Sensor Placement and Number on Physical Activity Recognition and Energy Expenditure Estimation in Older Adults: Validation Study

BACKGROUND

Research has shown the feasibility of human activity recognition using wearable accelerometer devices. Different studies have used varying numbers and placements for data collection using sensors.

OBJECTIVE

This study aims to compare accuracy performance between multiple and variable placements of accelerometer devices in categorizing the type of physical activity and corresponding energy expenditure in older adults.

METHODS

In total, 93 participants (mean age 72.2 years, SD 7.1) completed a total of 32 activities of daily life in a laboratory setting. Activities were classified as sedentary versus nonsedentary, locomotion versus nonlocomotion, and lifestyle versus nonlifestyle activities (eg, leisure walk vs computer work). A portable metabolic unit was worn during each activity to measure metabolic equivalents (METs). Accelerometers were placed on 5 different body positions: wrist, hip, ankle, upper arm, and thigh. Accelerometer data from each body position and combinations of positions were used to develop random forest models to assess activity category recognition accuracy and MET estimation.

RESULTS

Model performance for both MET estimation and activity category recognition were strengthened with the use of additional accelerometer devices. However, a single accelerometer on the ankle, upper arm, hip, thigh, or wrist had only a 0.03-0.09 MET increase in prediction error compared with wearing all 5 devices. Balanced accuracy showed similar trends with slight decreases in balanced accuracy for the detection of locomotion (balanced accuracy decrease range 0-0.01), sedentary (balanced accuracy decrease range 0.05-0.13), and lifestyle activities (balanced accuracy decrease range 0.04-0.08) compared with all 5 placements. The accuracy of recognizing activity categories increased with additional placements (accuracy decrease range 0.15-0.29). Notably, the hip was the best single body position for MET estimation and activity category recognition.

CONCLUSIONS

Additional accelerometer devices slightly enhance activity recognition accuracy and MET estimation in older adults. However, given the extra burden of wearing additional devices, single accelerometers with appropriate placement appear to be sufficient for estimating energy expenditure and activity category recognition in older adults.

Chest-Worn Inertial Sensors: A Survey of Applications and Methods

Inertial Measurement Units (IMUs) are frequently implemented in wearable devices. Thanks to advances in signal processing and machine learning, applications of IMUs are not limited to those explicitly addressing body movements such as Activity Recognition (AR). On the other hand, wearing IMUs on the chest offers a few advantages over other body positions. AR and posture analysis, cardiopulmonary parameters estimation, voice and swallowing activity detection and other measurements can be approached through chest-worn inertial sensors. This survey tries to introduce the applications that come with the chest-worn IMUs and summarizes the existing methods, current challenges and future directions associated with them. In this regard, this paper references a total number of 57 relevant studies from the last 10 years and categorizes them into seven application areas. We discuss the inertial sensors used as well as their placement on the body and their associated validation methods based on the application categories. Our investigations show meaningful correlations among the studies within the same application categories. Then, we investigate the data processing architectures of the studies from the hardware point of view, indicating a lack of effort on handling the main processing through on-body units. Finally, we propose combining the discussed applications in a single platform, finding robust ways for artifact cancellation, and planning optimized sensing/processing architectures for them, to be taken more seriously in future research.

Posture and Physical Activity Detection: Impact of Number of Sensors and Feature Type

Studies using wearable sensors to measure posture, physical activity (PA), and sedentary behavior typically use a single sensor worn on the ankle, thigh, wrist, or hip. Although the use of single sensors may be convenient, using multiple sensors is becoming more practical as sensors miniaturize.

PURPOSE

We evaluated the effect of single-site versus multisite motion sensing at seven body locations (both ankles, wrists, hips, and dominant thigh) on the detection of physical behavior recognition using a machine learning algorithm. We also explored the effect of using orientation versus orientation-invariant features on performance.

METHODS

Performance (F1 score) of PA and posture recognition was evaluated using leave-one-subject-out cross-validation on a 42-participant data set containing 22 physical activities with three postures (lying, sitting, and upright).

RESULTS

Posture and PA recognition models using two sensors had higher F1 scores (posture, 0.89 ± 0.06; PA, 0.53 ± 0.08) than did models using a single sensor (posture, 0.78 ± 0.11; PA, 0.43 ± 0.03). Models using two nonwrist sensors for posture recognition (F1 score, 0.93 ± 0.03) outperformed two-sensor models including one or two wrist sensors (F1 score, 0.85 ± 0.06). However, two-sensor models for PA recognition with at least one wrist sensor (F1 score, 0.60 ± 0.05) outperformed other two-sensor models (F1 score, 0.47 ± 0.02). Both posture and PA recognition F1 scores improved with more sensors (up to seven; 0.99 for posture and 0.70 for PA), but with diminishing performance returns. Models performed best when including orientation-based features.

CONCLUSIONS

Researchers measuring posture should consider multisite sensing using at least two nonwrist sensors, and researchers measuring PA should consider multisite sensing using at least one wrist sensor and one nonwrist sensor. Including orientation-based features improved both posture and PA recognition.

Exploration of Human Activity Recognition Using a Single Sensor for Stroke Survivors and Able-Bodied People

Commonly used sensors like accelerometers, gyroscopes, surface electromyography sensors, etc., which provide a convenient and practical solution for human activity recognition (HAR), have gained extensive attention. However, which kind of sensor can provide adequate information in achieving a satisfactory performance, or whether the position of a single sensor would play a significant effect on the performance in HAR are sparsely studied. In this paper, a comparative study to fully investigate the performance of the aforementioned sensors for classifying four activities (walking, tooth brushing, face washing, drinking) is explored. Sensors are spatially distributed over the human body, and subjects are categorized into three groups (able-bodied people, stroke survivors, and the union of both). Performances of using accelerometer, gyroscope, sEMG, and their combination in each group are evaluated by adopting the Support Vector Machine classifier with the Leave-One-Subject-Out Cross-Validation technique, and the optimal sensor position for each kind of sensor is presented based on the accuracy. Experimental results show that using the accelerometer could obtain the best performance in each group. The highest accuracy of HAR involving stroke survivors was 95.84 ± 1.75% (mean ± standard error), achieved by the accelerometer attached to the extensor carpi ulnaris. Furthermore, taking the practical application of HAR into consideration, a novel approach to distinguish various activities of stroke survivors based on a pre-trained HAR model built on healthy subjects is proposed, the highest accuracy of which is 77.89 ± 4.81% (mean ± standard error) with the accelerometer attached to the extensor carpi ulnaris.

Violent activity recognition by E-textile sensors based on machine learning methods

In the new era of technology with the development of wearable sensors, it is possible to collect data and analyze the same for recognition of different human activities. Activity recognition is used to monitor humans’ activity in various applications like assistance for an elderly and disabled person, Health care, physical activity monitoring, and also to identify a physical attack on a person etc. This paper presents the techniques of classifying the data from normal activity and violent attack on a victim. To solve this problem, the paper emphasis on classifying different activities using machine learning (supervised) techniques. Various experiments have been conducted using wearable inertial fabric sensors for different activities. These wearable e-textile sensors were woven onto the jacket worn by a healthy subject. The main steps which outline the process of activity recognition: location of sensors, pre-processing of the statistical data and activity. Three supervised algorithmic techniques were used namely Decision tree, k-NN classifier and Support Vector Machine (SVM). Based on the experimental work, the results obtained show that the SVM algorithm offers an overall good performance matched in terms of accuracy i.e. 97.6% and computation time of 0.85 seconds for k-NN and Decision Tree for all activities.

Data Mining and Fusion of Unobtrusive Sensing Solutions for Indoor Activity Recognition

This paper proposes the fusion of data from unobtrusive sensing solutions for the recognition and classification of activities in home environments. The ability to recognize and classify activities can help in the objective monitoring of health and wellness trends in ageing adults. While the use of video and stereo cameras for monitoring activities provides an adequate insight, the privacy of users is not fully protected (i.e., users can easily be recognized from the images). Another concern is that widely used wearable sensors, such as accelerometers, have some disadvantages, such as limited battery life, adoption issues and wearability. This study investigates the use of low-cost thermal sensing solutions capable of generating distinct thermal blobs with timestamps to recognize the activities of study participants. More than 11,000 thermal blobs were recorded from 10 healthy participants with two thermal sensors placed in a laboratory kitchen: (i) one mounted on the ceiling, and (ii) the other positioned on a mini tripod stand in the corner of the room. Furthermore, data from the ceiling thermal sensor were fused with data gleaned from the lateral thermal sensor. Contact sensors were used at each stage as the gold standard for timestamp approximation during data acquisition, which allowed the attainment of: (i) the time at which each activity took place, (ii) the type of activity performed, and (iii) the location of each participant. Experimental results demonstrated successful cluster-based activity recognition and classification with an average regression co-efficient of 0.95 for tested clusters and features. Also, an average accuracy of 95% was obtained for data mining models such as k-nearest neighbor, logistic regression, neural network and random forest on Evaluation Test.Clinical Relevance-This study presents an unobtrusive (i.e., privacy-friendly) solution for activity recognition and classification, for the purposes of profiling trends in health and wellbeing.

The Use of Inertial Measurement Units for the Study of Free Living Environment Activity Assessment: A Literature Review

This article presents an overview of fifty-eight articles dedicated to the evaluation of physical activity in free-living conditions using wearable motion sensors. This review provides a comprehensive summary of the technical aspects linked to sensors (types, number, body positions, and technical characteristics) as well as a deep discussion on the protocols implemented in free-living conditions (environment, duration, instructions, activities, and annotation). Finally, it presents a description and a comparison of the main algorithms and processing tools used for assessing physical activity from raw signals.

A Dual-Accelerometer System for Detecting Human Movement in a Free-living Environment

PURPOSE

Accurate measurement of various human movement behaviors is essential in developing 24-h movement profiles. A dual-accelerometer system recently showed promising results for accurately classifying a broad range of behaviors in a controlled laboratory environment. As a progressive step, the aim of this study is to validate the same dual-accelerometer system in semi free-living conditions in children and adults. The efficacy of several placement sites (e.g., wrist, thigh, back) was evaluated for comparison.

METHODS

Thirty participants (15 children) wore three Axivity AX3 accelerometers alongside an automated clip camera (clipped to the lapel) that recorded video of their free-living environment (ground truth criterion measure of physical activity). Participants were encouraged to complete a range of daily-living activities within a 2-h timeframe. A random forest machine-learning classifier was trained using features generated from the raw accelerometer data. Three different placement combinations were examined (thigh-back, thigh-wrist, back-wrist), and their performance was evaluated using leave-one-out cross-validation for the child and adult samples separately.

RESULTS

Machine learning models developed using the thigh-back accelerometer combination performed the best in distinguishing seven distinct activity classes with an overall accuracy of 95.6% in the adult sample, and eight activity classes with an overall accuracy of 92.0% in the child sample. There was a drop in accuracy (at least 11.0%) when other placement combinations were evaluated.

CONCLUSIONS

This validation study demonstrated that a dual-accelerometer system previously validated in a laboratory setting also performs well in semi free-living conditions. Although these results are promising and progressive, further work is needed to expand the scope of this measurement system to detect other components of behavior (e.g., activity intensity and sleep) that are related to health.

Evaluation of Machine Learning Models for Classifying Upper Extremity Exercises Using Inertial Measurement Unit-Based Kinematic Data

The amount of home-based exercise prescribed by a physical therapist is difficult to monitor. However, the integration of wearable inertial measurement unit (IMU) devices can aid in monitoring home exercise by analyzing exercise biomechanics. The objective of this study is to evaluate machine learning models for classifying nine different upper extremity exercises, based upon kinematic data captured from an IMU-based device. Fifty participants performed one compound and eight isolation exercises with their right arm. Each exercise was performed ten times for a total of 4500 trials. Joint angles were calculated using IMUs that were placed on the hand, forearm, upper arm, and torso. Various machine learning models were developed with different algorithms and train-test splits. Random forest models with flattened kinematic data as a feature had the greatest accuracy (98.6%). Using triaxial joint range of motion as the feature set resulted in decreased accuracy (91.9%) with faster speeds. Accuracy did not decrease below 90% until training size was decreased to 5% from 50%. Accuracy decreased (88.7%) when splitting data by participant. Upper extremity exercises can be classified accurately using kinematic data from a wearable IMU device. A random forest classification model was developed that quickly and accurately classified exercises. Sampling frequency and lower training splits had a modest effect on performance. When the data were split by subject stratification, larger training sizes were required for acceptable algorithm performance. These findings set the basis for more objective and accurate measurements of home-based exercise using emerging healthcare technologies.

A multiple motion sensors index for motor state quantification in Parkinson's disease

AIM

To construct a Treatment Response Index from Multiple Sensors (TRIMS) for quantification of motor state in patients with Parkinson's disease (PD) during a single levodopa dose. Another aim was to compare TRIMS to sensor indexes derived from individual motor tasks.

METHOD

Nineteen PD patients performed three motor tests including leg agility, pronation-supination movement of hands, and walking in a clinic while wearing inertial measurement unit sensors on their wrists and ankles. They performed the tests repeatedly before and after taking 150% of their individual oral levodopa-carbidopa equivalent morning dose.Three neurologists blinded to treatment status, viewed patients' videos and rated their motor symptoms, dyskinesia, overall motor state based on selected items of Unified PD Rating Scale (UPDRS) part III, Dyskinesia scale, and Treatment Response Scale (TRS). To build TRIMS, out of initially 178 extracted features from upper- and lower-limbs data, 39 features were selected by stepwise regression method and were used as input to support vector machines to be mapped to mean reference TRS scores using 10-fold cross-validation method. Test-retest reliability, responsiveness to medication, and correlation to TRS as well as other UPDRS items were evaluated for TRIMS.

RESULTS

The correlation of TRIMS with TRS was 0.93. TRIMS had good test-retest reliability (ICC = 0.83). Responsiveness of the TRIMS to medication was good compared to TRS indicating its power in capturing the treatment effects. TRIMS was highly correlated to dyskinesia (R = 0.85), bradykinesia (R = 0.84) and gait (R = 0.79) UPDRS items. Correlation of sensor index from the upper-limb to TRS was 0.89.

CONCLUSION

Using the fusion of upper- and lower-limbs sensor data to construct TRIMS provided accurate PD motor states estimation and responsive to treatment. In addition, quantification of upper-limb sensor data during walking test provided strong results.

Design and Validation of a Minimal Complexity Algorithm for Stair Step Counting

Wearable sensors play a significant role for monitoring the functional ability of the elderly and in general, promoting active ageing. One of the relevant variables to be tracked is the number of stair steps (single stair steps) performed daily, which is more challenging than counting flight of stairs and detecting stair climbing. In this study, we proposed a minimal complexity algorithm composed of a hierarchical classifier and a linear model to estimate the number of stair steps performed during everyday activities. The algorithm was calibrated on accelerometer and barometer recordings measured using a sensor platform worn at the wrist from 20 healthy subjects. It was then tested on 10 older people, specifically enrolled for the study. The algorithm was then compared with other three state-of-the-art methods, which used the accelerometer, the barometer or both. The experiments showed the good performance of our algorithm (stair step counting error: 13.8%), comparable with the best state-of-the-art (p > 0.05), but using a lower computational load and model complexity. Finally, the algorithm was successfully implemented in a low-power smartwatch prototype with a memory footprint of about 4 kB.

Human activity recognition using magnetic induction-based motion signals and deep recurrent neural networks

Recognizing human physical activities using wireless sensor networks has attracted significant research interest due to its broad range of applications, such as healthcare, rehabilitation, athletics, and senior monitoring. There are critical challenges inherent in designing a sensor-based activity recognition system operating in and around a lossy medium such as the human body to gain a trade-off among power consumption, cost, computational complexity, and accuracy. We introduce an innovative wireless system based on magnetic induction for human activity recognition to tackle these challenges and constraints. The magnetic induction system is integrated with machine learning techniques to detect a wide range of human motions. This approach is successfully evaluated using synthesized datasets, laboratory measurements, and deep recurrent neural networks.

Application of Raw Accelerometer Data and Machine-Learning Techniques to Characterize Human Movement Behavior: A Systematic Scoping Review

BACKGROUND

Application of machine learning for classifying human behavior is increasingly common as access to raw accelerometer data improves. The aims of this scoping review are (1) to examine if machine-learning techniques can accurately identify human activity behaviors from raw accelerometer data and (2) to summarize the practical implications of these machine-learning techniques for future work.

METHODS

Keyword searches were performed in Scopus, Web of Science, and EBSCO databases in 2018. Studies that applied supervised machine-learning techniques to raw accelerometer data and estimated components of physical activity were included. Information on study characteristics, machine-learning techniques, and key study findings were extracted from included studies.

RESULTS

Of the 53 studies included in the review, 75% were published in the last 5 years. Most studies predicted postures and activity type, rather than intensity, and were conducted in controlled environments using 1 or 2 devices. The most common models were support vector machine, random forest, and artificial neural network. Overall, classification accuracy ranged from 62% to 99.8%, although nearly 80% of studies achieved an overall accuracy above 85%.

CONCLUSIONS

Machine-learning algorithms demonstrate good accuracy when predicting physical activity components; however, their application to free-living settings is currently uncertain.

A Hybrid Kinematic-Acoustic System for Automated Activity Detection of Construction Equipment

Automatically recognizing and tracking construction equipment activities is the first step towards performance monitoring of a job site. Recognizing equipment activities helps construction managers to detect the equipment downtime/idle time in a real-time framework, estimate the productivity rate of each equipment based on its progress, and efficiently evaluate the cycle time of each activity. Thus, it leads to project cost reduction and time schedule improvement. Previous studies on this topic have been based on single sources of data (e.g., kinematic, audio, video signals) for automated activity-detection purposes. However, relying on only one source of data is not appropriate, as the selected data source may not be applicable under certain conditions and fails to provide accurate results. To tackle this issue, the authors propose a hybrid system for recognizing multiple activities of construction equipment. The system integrates two major sources of data-audio and kinematic-through implementing a robust data fusion procedure. The presented system includes recording audio and kinematic signals, preprocessing data, extracting several features, as well as dimension reduction, feature fusion, equipment activity classification using Support Vector Machines (SVM), and smoothing labels. The proposed system was implemented in several case studies (i.e., ten different types and equipment models operating at various construction job sites) and the results indicate that a hybrid system is capable of providing up to 20% more accurate results, compared to cases using individual sources of data.

Non-Intrusive Presence Detection and Position Tracking for Multiple People Using Low-Resolution Thermal Sensors

This paper presents a framework to accurately and non-intrusively detect the number of people in an environment and track their positions. Different from most of the previous studies, our system setup uses only ambient thermal sensors with low-resolution, using no multimedia resources or wearable sensors. This preserves user privacy in the environment, and requires no active participation by the users, causing no discomfort. We first develop multiple methods to estimate the number of people in the environment. Our experiments demonstrate that algorithm selection is very important, but with careful selection, we can obtain up to 100% accuracy when detecting user presence. In addition, we prove that sensor placement plays a crucial role in the system performance, where placing the sensor on the room ceiling yields to the best results. After accurately finding the number of people in the environment, we perform position tracking on the collected ambient data, which are thermal images of the space where there are multiple people. We consider position tracking as static activity detection, where the user’s position does not change while performing activities, such as sitting, standing, etc. We perform efficient pre-processing on the data, including normalization and resizing, and then feed the data into well-known machine learning methods. We tested the efficiency of our framework (including the hardware and software setup) by detecting four static activities. Our results show that we can achieved up to 97.5% accuracy when detecting these static activities, with up to 100% class-wise precision and recall rates. Our framework can be very beneficial to several applications such as health-care, surveillance, and home automation, without causing any discomfort or privacy issues for the users.

Multi-Sensor Activity Monitoring: Combination of Models with Class-Specific Voting

This paper presents a multi-sensor model combination system with class-specific voting for physical activity monitoring, which combines multiple classifiers obtained by splicing sensor data from different nodes into new data frames to improve the diversity of model inputs. Data obtained from a wearable multi-sensor wireless integrated measurement system (WIMS) consisting of two accelerometers and one ventilation sensor have been analysed to identify 10 different activity types of varying intensities performed by 110 voluntary participants. It is noted that each classifier shows better performance on some specific activity classes. Through class-specific weighted majority voting, the recognition accuracy of 10 PA types has been improved from 86% to 92% compared with the non-combination approach. Furthermore, the combination method has shown to be effective in reducing the subject-to-subject variability (standard deviation of recognition accuracies across subjects) in activity recognition and has better performance in monitoring physical activities of varying intensities than traditional homogeneous classifiers.

Using a smart textile system for classifying occupational manual material handling tasks: evidence from lab-based simulations

Physical monitoring systems represent potentially powerful assessment devices to detect and describe occupational physical activities. A promising technology for such use is smart textile systems (STSs). Our goal in this exploratory study was to assess the feasibility and accuracy of using two STSs to classify several manual material handling (MMH) tasks. Specifically, commercially-available 'smart' socks and a custom 'smart' shirt were used individually and in combination. Eleven participants simulated nine separate MMH tasks while wearing the STSs, and task classification accuracy was quantified subsequently using several common models. The shirt and socks, both individually and in combination, could classify the simulated tasks with greater than 97% accuracy. Thus, using STSs appears to have potential utility for discriminating occupational physical tasks in the work environment. Practitioner summary: A smart textile system could classify diverse MMH tasks with high accuracy. This technology may help in developing future ergonomic exposure assessment systems, with the goal of preventing occupational injuries.

Wearable Embedded Intelligence for Detection of Falls Independently of on-Body Location

Falls are one of the most common problems in the elderly population. Therefore, each year more solutions for automatic fall detection are emerging. This paper proposes a single accelerometer algorithm for wearable devices that works for three different body locations: chest, waist and pocket, without a calibration step being required. This algorithm is able to be fully executed on a wearable device and no external devices are necessary for data processing. Additionally, a study of the accelerometer sampling rate, that allows the algorithm to achieve a better performance, was performed. The algorithm was validated with a continuous dataset with daily living activities and 272 simulated falls. Considering the trade-off between sensitivity and the number of false alarms the most suitable sampling rate found was 50 Hz. The proposed algorithm was able to achieve a trade-off of no false alarms and 89.5% of fall detection rate when wearing the sensor on the user's waist with a medium sensitivity level of the algorithm. In conclusion, this paper presents a reliable solution for automatic fall detection that can be adapted to different usages and conditions, since it can be used in different body locations and its sensitivity can be adapted to different subjects according to their physical activity level.

Would a thermal sensor improve arm motion classification accuracy of a single wrist-mounted inertial device?

Background

Inertial Measurement Unit (IMU)-based wearable sensors have found common use to track arm activity in daily life. However, classifying a high number of arm motions with single IMU-based systems still remains a challenging task. This paper explores the possibility to increase the classification accuracy of these systems by incorporating a thermal sensor. Increasing the number of arm motions that can be classified is relevant to increasing applicability of single-device wearable systems for a variety of applications, including activity monitoring for athletes, gesture control for video games, and motion classification for physical rehabilitation patients. This study explores whether a thermal sensor can increase the classification accuracy of a single-device motion classification system when evaluated with healthy participants. The motions performed are reproductions of exercises described in established rehabilitation protocols.

Methods

A single wrist-mounted device was built with an inertial sensor and a thermal sensor. This device was worn on the wrist, was battery powered, and transmitted data over Bluetooth to computer during recording. A LabVIEW Graphical User Interface (GUI) instructed the user to complete 24 different arm motions in a pre-randomized order. The received data were pre-processed, and secondary features were calculated on these data. These features were processed with Principal Component Analysis (PCA) for dimensionality reduction and then several machine learning models were applied to select the optimal model based on speed and accuracy. To test the effectiveness of the scheme, 11 healthy subjects participated in the trials.

Results

Average personalized classification model accuracies of 93.55% were obtained for 11 healthy participants. Generalized model accuracies of 82.5% indicated that the device can classify arm motions on a user without prior training. The addition of a thermal sensor significantly increased classification accuracy of a single wrist-mounted inertial device, from 75 to 93.55%, (F(1,20) = 90.53, p = 7.25e−09).

Conclusion

This study found that the addition of the thermal sensor improved the classification accuracy of 24 arm motions from 75 to 93.55% for a single-device system. Our results provide evidence that a single device can be used to classify a relatively large number of arm motions from arm rehabilitation protocols. While this study provides a conceptual proof-of-concept with a healthy population, additional investigation is required to evaluate the performance of this system for specific applications, such as activity classification for physically affected stroke survivors undergoing home-based rehabilitation.

Electronic supplementary material

The online version of this article (10.1186/s12938-019-0677-7) contains supplementary material, which is available to authorized users.

Static and Dynamic Activity Detection with Ambient Sensors in Smart Spaces

Convergence of Machine Learning, Internet of Things, and computationally powerful single-board computers has boosted research and implementation of smart spaces. Smart spaces make predictions based on historical data to enhance user experience. In this paper, we present a low-cost, low-energy smart space implementation to detect static and dynamic human activities that require simple motions. We use low-resolution (4 × 16) and non-intrusive thermal sensors to collect data. We train six machine learning algorithms, namely logistic regression, naive Bayes, support vector machine, decision tree, random forest and artificial neural network (vanilla feed-forward) on the dataset collected in our lab. Our experiments reveal a very high static activity detection rate with all algorithms, where the feed-forward neural network method gives the best accuracy of 99.96%. We also show how data collection methods and sensor placement plays an important role in the resulting accuracy of different machine learning algorithms. To detect dynamic activities in real time, we use cross-correlation and connected components of thermal images. Our smart space implementation, with its real-time properties, can be used in various domains and applications, such as conference room automation, elderly health-care, etc.

Windows Into Human Health Through Wearables Data Analytics

Background

Wearable sensors (wearables) have been commonly integrated into a wide variety of commercial products and are increasingly being used to collect and process raw physiological parameters into salient digital health information. The data collected by wearables are currently being investigated across a broad set of clinical domains and patient populations. There is significant research occurring in the domain of algorithm development, with the aim of translating raw sensor data into fitness- or health-related outcomes of interest for users, patients, and health care providers.

Objectives

The aim of this review is to highlight a selected group of fitness- and health-related indicators from wearables data and to describe several algorithmic approaches used to generate these higher order indicators.

Methods

A systematic search of the Pubmed database was performed with the following search terms (number of records in parentheses): Fitbit algorithm (18), Apple Watch algorithm (3), Garmin algorithm (5), Microsoft Band algorithm (8), Samsung Gear algorithm (2), Xiaomi MiBand algorithm (1), Huawei Band (Watch) algorithm (2), photoplethysmography algorithm (465), accelerometry algorithm (966), ECG algorithm (8287), continuous glucose monitor algorithm (343). The search terms chosen for this review are focused on algorithms for wearable devices that dominated the commercial wearables market between 2014-2017 and that were highly represented in the biomedical literature. A second set of search terms included categories of algorithms for fitness-related and health-related indicators that are commonly used in wearable devices (e.g. accelerometry, PPG, ECG). These papers covered the following domain areas: fitness; exercise; movement; physical activity; step count; walking; running; swimming; energy expenditure; atrial fibrillation; arrhythmia; cardiovascular; autonomic nervous system; neuropathy; heart rate variability; fall detection; trauma; behavior change; diet; eating; stress detection; serum glucose monitoring; continuous glucose monitoring; diabetes mellitus type 1; diabetes mellitus type 2. All studies uncovered through this search on commercially available device algorithms and pivotal studies on sensor algorithm development were summarized, and a summary table was constructed using references generated by the literature review as described (Table 1).

Conclusions

Wearable health technologies aim to collect and process raw physiological or environmental parameters into salient digital health information. Much of the current and future utility of wearables lies in the signal processing steps and algorithms used to analyze large volumes of data. Continued algorithmic development and advances in machine learning techniques will further increase analytic capabilities. In the context of these advances, our review aims to highlight a range of advances in fitness- and other health-related indicators provided by current wearable technologies.

A smartwatch-based framework for real-time and online assessment and mobility monitoring

Smartphone and smartwatch technology is changing the transmission and monitoring landscape for patients and research participants to communicate their healthcare information in real time. Flexible, bidirectional and real-time control of communication allows development of a rich set of healthcare applications that can provide interactivity with the participant and adapt dynamically to their changing environment. Additionally, smartwatches have a variety of sensors suitable for collecting physical activity and location data. The combination of all these features makes it possible to transmit the collected data to a remote server, and thus, to monitor physical activity and potentially social activity in real time. As smartwatches exhibit high user acceptability and increasing popularity, they are ideal devices for monitoring activities for extended periods of time to investigate the physical activity patterns in free-living condition and their relationship with the seemingly random occurring illnesses, which have remained a challenge in the current literature. Therefore, the purpose of this study was to develop a smartwatch-based framework for real-time and online assessment and mobility monitoring (ROAMM). The proposed ROAMM framework will include a smartwatch application and server. The smartwatch application will be used to collect and preprocess data. The server will be used to store and retrieve data, remote monitor, and for other administrative purposes. With the integration of sensor-based and user-reported data collection, the ROAMM framework allows for data visualization and summary statistics in real-time.

Classification of Human Daily Activities Using Ensemble Methods Based on Smartphone Inertial Sensors

Increasing interest in analyzing human gait using various wearable sensors, which is known as Human Activity Recognition (HAR), can be found in recent research. Sensors such as accelerometers and gyroscopes are widely used in HAR. Recently, high interest has been shown in the use of wearable sensors in numerous applications such as rehabilitation, computer games, animation, filmmaking, and biomechanics. In this paper, classification of human daily activities using Ensemble Methods based on data acquired from smartphone inertial sensors involving about 30 subjects with six different activities is discussed. The six daily activities are walking, walking upstairs, walking downstairs, sitting, standing and lying. It involved three stages of activity recognition; namely, data signal processing (filtering and segmentation), feature extraction and classification. Five types of ensemble classifiers utilized are Bagging, Adaboost, Rotation forest, Ensembles of nested dichotomies (END) and Random subspace. These ensemble classifiers employed Support vector machine (SVM) and Random forest (RF) as the base learners of the ensemble classifiers. The data classification is evaluated with the holdout and 10-fold cross-validation evaluation methods. The performance of each human daily activity was measured in terms of precision, recall, F-measure, and receiver operating characteristic (ROC) curve. In addition, the performance is also measured based on the comparison of overall accuracy rate of classification between different ensemble classifiers and base learners. It was observed that overall, SVM produced better accuracy rate with 99.22% compared to RF with 97.91% based on a random subspace ensemble classifier.

Deep learning-based classification with improved time resolution for physical activities of children

Background

The proportion of overweight and obese people has increased tremendously in a short period, culminating in a worldwide trend of obesity that is reaching epidemic proportions. Overweight and obesity are serious issues, especially with regard to children. This is because obese children have twice the risk of becoming obese as adults, as compared to non-obese children. Nowadays, many methods for maintaining a caloric balance exist; however, these methods are not applicable to children. In this study, a new approach for helping children monitor their activities using a convolutional neural network (CNN) is proposed, which is applicable for real-time scenarios requiring high accuracy.

Methods

A total of 136 participants (86 boys and 50 girls), aged between 8.5 years and 12.5 years (mean 10.5, standard deviation 1.1), took part in this study. The participants performed various movement while wearing custom-made three-axis accelerometer modules around their waists. The data acquired by the accelerometer module was preprocessed by dividing them into small sets (128 sample points for 2.8 s). Approximately 183,600 data samples were used by the developed CNN for learning to classify ten physical activities : slow walking, fast walking, slow running, fast running, walking up the stairs, walking down the stairs, jumping rope, standing up, sitting down, and remaining still.

Results

The developed CNN classified the ten activities with an overall accuracy of 81.2%. When similar activities were merged, leading to seven merged activities, the CNN classified activities with an overall accuracy of 91.1%. Activity merging also improved performance indicators, for the maximum case of 66.4% in recall, 48.5% in precision, and 57.4% in f1 score . The developed CNN classifier was compared to conventional machine learning algorithms such as the support vector machine, decision tree, and k-nearest neighbor algorithms, and the proposed CNN classifier performed the best: CNN (81.2%) > SVM (64.8%) > DT (63.9%) > kNN (55.4%) (for ten activities); CNN (91.1%) > SVM (74.4%) > DT (73.2%) > kNN (65.3%) (for the merged seven activities).

Discussion

The developed algorithm distinguished physical activities with improved time resolution using short-time acceleration signals from the physical activities performed by children. This study involved algorithm development, participant recruitment, IRB approval, custom-design of a data acquisition module, and data collection. The self-selected moving speeds for walking and running (slow and fast) and the structure of staircase degraded the performance of the algorithm. However, after similar activities were merged, the effects caused by the self-selection of speed were reduced. The experimental results show that the proposed algorithm performed better than conventional algorithms. Owing to its simplicity, the proposed algorithm could be applied to real-time applicaitons.

Activity Monitoring with a Wrist-Worn, Accelerometer-Based Device

This study condenses huge amount of raw data measured from a MEMS accelerometer-based, wrist-worn device on different levels of physical activities (PAs) for subjects wearing the device 24 h a day continuously. In this study, we have employed the device to build up assessment models for quantifying activities, to develop an algorithm for sleep duration detection and to assess the regularity of activity of daily living (ADL) quantitatively. A new parameter, the activity index (AI), has been proposed to represent the quantity of activities and can be used to categorize different PAs into 5 levels, namely, rest/sleep, sedentary, light, moderate, and vigorous activity states. Another new parameter, the regularity index (RI), was calculated to represent the degree of regularity for ADL. The methods proposed in this study have been used to monitor a subject’s daily PA status and to access sleep quality, along with the quantitative assessment of the regularity of activity of daily living (ADL) with the 24-h continuously recorded data over several months to develop activity-based evaluation models for different medical-care applications. This work provides simple models for activity monitoring based on the accelerometer-based, wrist-worn device without trying to identify the details of types of activity and that are suitable for further applications combined with cloud computing services.