Prediction of Relative Physical Activity Intensity Using Multimodal Sensing of Physiological Data

Electrodermal Activity (EDA) Morphologies and Prediction of Engagement with Simple Moving Average Crossover: A Mixed-Method Study

Electrodermal Activity (EDA), which primarily indicates arousal through sympathetic nervous system activity, serves as a tool to measure constructs like engagement, cognitive load, performance, and stress. Despite its potential, empirical studies have often yielded mixed results and found it of limited use. To better understand EDA, we conducted a mixed-methods study in which quantitative EDA profiles and survey data were investigated using qualitative interviews. This study furnishes an EDA dataset measuring the engagement levels of seven participants who watched three videos for 4-10 min. The subsequent interviews revealed five EDA morphologies with varying short-term signatures and long-term trends. We used this dataset to demonstrate the moving average crossover, a novel metric for EDA analysis, in predicting engagement-disengagement dynamics in such data. Our contributions include the creation of the detailed dataset, comprising EDA profiles annotated with qualitative data, the identification of five distinct EDA morphologies, and the proposition of the moving average crossover as an indicator of the beginning of engagement or disengagement in an individual.

Human Activity Recognition Algorithm with Physiological and Inertial Signals Fusion: Photoplethysmography, Electrodermal Activity, and Accelerometry

Inertial signals are the most widely used signals in human activity recognition (HAR) applications, and extensive research has been performed on developing HAR classifiers using accelerometer and gyroscope data. This study aimed to investigate the potential enhancement of HAR models through the fusion of biological signals with inertial signals. The classification of eight common low-, medium-, and high-intensity activities was assessed using machine learning (ML) algorithms, trained on accelerometer (ACC), blood volume pulse (BVP), and electrodermal activity (EDA) data obtained from a wrist-worn sensor. Two types of ML algorithms were employed: a random forest (RF) trained on features; and a pre-trained deep learning (DL) network (ResNet-18) trained on spectrogram images. Evaluation was conducted on both individual activities and more generalized activity groups, based on similar intensity. Results indicated that RF classifiers outperformed corresponding DL classifiers at both individual and grouped levels. However, the fusion of EDA and BVP signals with ACC data improved DL classifier performance compared to a baseline DL model with ACC-only data. The best performance was achieved by a classifier trained on a combination of ACC, EDA, and BVP images, yielding F1-scores of 69 and 87 for individual and grouped activity classifications, respectively. For DL models trained with additional biological signals, almost all individual activity classifications showed improvement (p-value < 0.05). In grouped activity classifications, DL model performance was enhanced for low- and medium-intensity activities. Exploring the classification of two specific activities, ascending/descending stairs and cycling, revealed significantly improved results using a DL model trained on combined ACC, BVP, and EDA spectrogram images (p-value < 0.05).

Transforming the cardiometabolic disease landscape: Multimodal AI-powered approaches in prevention and management

The rise of artificial intelligence (AI) has revolutionized various scientific fields, particularly in medicine, where it has enabled the modeling of complex relationships from massive datasets. Initially, AI algorithms focused on improved interpretation of diagnostic studies such as chest X-rays and electrocardiograms in addition to predicting patient outcomes and future disease onset. However, AI has evolved with the introduction of transformer models, allowing analysis of the diverse, multimodal data sources existing in medicine today. Multimodal AI holds great promise in more accurate disease risk assessment and stratification as well as optimizing the key driving factors in cardiometabolic disease: blood pressure, sleep, stress, glucose control, weight, nutrition, and physical activity. In this article we outline the current state of medical AI in cardiometabolic disease, highlighting the potential of multimodal AI to augment personalized prevention and treatment strategies in cardiometabolic disease.

Generalizability of Human Activity Recognition Machine Learning Models from non-Parkinson's to Parkinson's Disease Patients

Recent evidence shows that high-intensity exercises reduce tremors and stiffness in Parkinson's disease (PD). However, there is insufficient evidence on the types of exercises; in effect, high-intensity may be a personalized measure. Recent progress in automated Human Activity Recognition using machine learning (ML) models shows potential for better monitoring of PD patients. However, ML models must be calibrated to ignore tremors and accurately identify activity and its intensity. We report findings from a study where we trained ML models using data from medically validated triple synchronous sensors connected to 8 non-PD subjects performing 32 exercises. We then tested the models to identify exercises performed by 8 PD patients at different stages of the disease. Our analysis shows that better data preprocessing before modeling can provide some model generalizability. However, it is extremely challenging, as the models work with high accuracy on one group (Healthy or PD patients) (F1=0.88-0.94) but not on both groups.Clinical relevance-Patients with Parkinson's and other motor-generative diseases can now accurately measure physical activity with machine learning approaches. Clinicians, caregivers, and apps can make accurate, personalized exercise recommendations to augment medications that reduce tremors and stiffness.

Machine Learning on Prediction of Relative Physical Activity Intensity Using Medical Radar Sensor and 3D Accelerometer

BACKGROUND

One of the most critical topics in sports safety today is the reduction in injury risks through controlled fatigue using non-invasive athlete monitoring. Due to the risk of injuries, it is prohibited to use accelerometer-based smart trackers, activity measurement bracelets, and smart watches for recording health parameters during performance sports activities. This study analyzes the synergy feasibility of medical radar sensors and tri-axial acceleration sensor data to predict physical activity key performance indexes in performance sports by using machine learning (ML). The novelty of this method is that it uses a 24 GHz Doppler radar sensor to detect vital signs such as the heartbeat and breathing without touching the person and to predict the intensity of physical activity, combined with the acceleration data from 3D accelerometers.

METHODS

This study is based on the data collected from professional athletes and freely available datasets created for research purposes. A combination of sensor data management was used: a medical radar sensor with no-contact remote sensing to measure the heart rate (HR) and 3D acceleration to measure the velocity of the activity. Various advanced ML methods and models were employed on the top of sensors to analyze the vital parameters and predict the health activity key performance indexes. three-axial acceleration, heart rate data, age, as well as activity level variances.

RESULTS

The ML models recognized the physical activity intensity and estimated the energy expenditure on a realistic level. Leave-one-out (LOO) cross-validation (CV), as well as out-of-sample testing (OST) methods, have been used to evaluate the level of accuracy in activity intensity prediction. The energy expenditure prediction with three-axial accelerometer sensors by using linear regression provided 97-99% accuracy on selected sports (cycling, running, and soccer). The ML-based RPE results using medical radar sensors on a time-series heart rate (HR) dataset varied between 90 and 96% accuracy. The expected level of accuracy was examined with different models. The average accuracy for all the models (RPE and METs) and setups was higher than 90%.

CONCLUSIONS

The ML models that classify the rating of the perceived exertion and the metabolic equivalent of tasks perform consistently.

PERSIST: A Multimodal Dataset for the Prediction of Perceived Exertion during Resistance Training

Measuring and adjusting the training load is essential in resistance training, as training overload can increase the risk of injuries. At the same time, too little load does not deliver the desired training effects. Usually, external load is quantified using objective measurements, such as lifted weight distributed across sets and repetitions per exercise. Internal training load is usually assessed using questionnaires or ratings of perceived exertion (RPE). A standard RPE scale is the Borg scale, which ranges from 6 (no exertion) to 20 (the highest exertion ever experienced). Researchers have investigated predicting RPE for different sports using sensor modalities and machine learning methods, such as Support Vector Regression or Random Forests. This paper presents PERSIST, a novel dataset for predicting PERceived exertion during reSIStance Training. We recorded multiple sensor modalities simultaneously, including inertial measurement units (IMU), electrocardiography (ECG), and motion capture (MoCap). The MoCap data has been synchronized to the IMU and ECG data. We also provide heart rate variability (HRV) parameters obtained from the ECG signal. Our dataset contains data from twelve young and healthy male participants with at least one year of resistance training experience. Subjects performed twelve sets of squats on a Flywheel platform with twelve repetitions per set. After each set, subjects reported their current RPE. We chose the squat exercise as it involves the largest muscle group. This paper demonstrates how to access the dataset. We further present an exploratory data analysis and show how researchers can use IMU and ECG data to predict perceived exertion.

Machine Learning Techniques for Arousal Classification from Electrodermal Activity: A Systematic Review

This article introduces a systematic review on arousal classification based on electrodermal activity (EDA) and machine learning (ML). From a first set of 284 articles searched for in six scientific databases, fifty-nine were finally selected according to various criteria established. The systematic review has made it possible to analyse all the steps to which the EDA signals are subjected: acquisition, pre-processing, processing and feature extraction. Finally, all ML techniques applied to the features of these signals for arousal classification have been studied. It has been found that support vector machines and artificial neural networks stand out within the supervised learning methods given their high-performance values. In contrast, it has been shown that unsupervised learning is not present in the detection of arousal through EDA. This systematic review concludes that the use of EDA for the detection of arousal is widely spread, with particularly good results in classification with the ML methods found.

Entwicklung einer KI-gestützten Bewegungstherapie bei onkologischen Palliativpatienten

HintergrundDie wechselnde Symptomlast ist eine große Hürde in der Sporttherapie von onkologischen Palliativpatienten. Die täglich variierende Symptomstärke erschwert die Einstellung einer optimalen Trainingsbelastung und stellt neben der Motivation eine große Barriere für die Teilnahme an bewegungstherapeutischen Interventionen dar. Ein durch Künstliche Intelligenz (KI) gesteuertes Training könnte helfen, die Trainingseinheiten individuell anzupassen und die Autonomie von Palliativpatienten zu erhalten.

Methoden Fünf Patienten mit fortgeschrittener unheilbarer Krebsdiagnose haben im Rahmen der Routineversorgung eine supervidierte Bewegungstherapie absolviert. Dabei wurde ein Elektrokardiogramm über einen Polar H10 Brustgurt aufgezeichnet und daraus kardiale und respiratorische Vitalparameter extrahiert. Eine Klassifikation in drei Intensitätsstufen über KI erfolgte anhand von neuronalen Netzen.

Ergebnisse Das KI-gesteuerte Training hat eine sehr hohe Klassifikationsgüte (F1-Score: 0,95±0,05) durch die Vereinigung von respiratorischen und kardialen Vitalparametern. Diese Kombination erzielt genauere Klassifikationsergebnisse als die einzelnen Datensätze für kardiale Parameter (0,93±0,06) und respiratorische Parameter (0,72±0,06). Die Berücksichtigung einer Baselinemessung hat eine positive Wirkung auf die Klassifikationsgenauigkeit.

Diskussion Diese Studie stellt die erste Untersuchung zum Einsatz von KI zur Klassifizierung von trainingswissenschaftlichen Inhalten bei onkologischen Palliativpatienten dar. Diese vulnerable Patientengruppe kann von einer objektiven Erfassung des Belastungsniveaus anhand von Parametern des kardiovaskulären Systems profitieren. Mit nur fünf Patienten wird die Aussagekraft dieser explorativen Studie über Kreuzvalidierung hergestellt. Zukünftig sollen weitere Parameter wie ein subjektives Empfinden, Alter, Größe und Geschlecht die Klassifikation weiter verbessern. In einem integrierten System ist eine individuelle Trainingssteuerung in Echtzeit möglich.

Are Machine Learning Models on Wrist Accelerometry Robust against Differences in Physical Performance among Older Adults?

Sufficient physical activity (PA) reduces the risk of a myriad of diseases and preserves physical capabilities in later life. While there have been significant achievements in mapping accelerations to real-life movements using machine learning (ML), errors continue to be common, particularly for wrist-worn devices. It remains unknown whether ML models are robust for estimating age-related loss of physical function. In this study, we evaluated the performance of ML models (XGBoost and LASSO) to estimate the hallmark measures of PA in low physical performance (LPP) and high physical performance (HPP) groups. Our models were built to recognize PA types and intensities, identify each individual activity, and estimate energy expenditure (EE) using wrist-worn accelerometer data (33 activities per participant) from a large sample of participants (n = 247, 57% females, aged 60+ years). Results indicated that the ML models were accurate in recognizing PA by type and intensity while also estimating EE accurately. However, the models built to recognize individual activities were less robust. Across all tasks, XGBoost outperformed LASSO. XGBoost obtained F1-Scores for sedentary (0.932 ± 0.005), locomotion (0.946 ± 0.003), lifestyle (0.927 ± 0.006), and strength flexibility exercise (0.915 ± 0.017) activity type recognition tasks. The F1-Scores for recognizing low, light, and moderate activity intensity were (0.932 ± 0.005), (0.840 ± 0.004), and (0.869 ± 0.005), respectively. The root mean square error for EE estimation was 0.836 ± 0.059 METs. There was no evidence showing that splitting the participants into the LPP and HPP groups improved the models’ performance on estimating the hallmark measures of physical activities. In conclusion, using features derived from wrist-worn accelerometer data, machine learning models can accurately recognize PA types and intensities and estimate EE for older adults with high and low physical function.

The Concept of Advanced Multi-Sensor Monitoring of Human Stress

Many people live under stressful conditions which has an adverse effect on their health. Human stress, especially long-term one, can lead to a serious illness. Therefore, monitoring of human stress influence can be very useful. We can monitor stress in strictly controlled laboratory conditions, but it is time-consuming and does not capture reactions, on everyday stressors or in natural environment using wearable sensors, but with limited accuracy. Therefore, we began to analyze the current state of promising wearable stress-meters and the latest advances in the record of related physiological variables. Based on these results, we present the concept of an accurate, reliable and easier to use telemedicine device for long-term monitoring of people in a real life. In our concept, we ratify with two synchronized devices, one on the finger and the second on the chest. The results will be obtained from several physiological variables including electrodermal activity, heart rate and respiration, body temperature, blood pressure and others. All these variables will be measured using a coherent multi-sensors device. Our goal is to show possibilities and trends towards the production of new telemedicine equipment and thus, opening the door to a widespread application of human stress-meters.

Age Differences in Estimating Physical Activity by Wrist Accelerometry Using Machine Learning

Accelerometer-based fitness trackers and smartwatches are proliferating with incessant attention towards health tracking. Despite their growing popularity, accurately measuring hallmark measures of physical activities has yet to be accomplished in adults of all ages. In this work, we evaluated the performance of four machine learning models: decision tree, random forest, extreme gradient boosting (XGBoost) and least absolute shrinkage and selection operator (LASSO), to estimate the hallmark measures of physical activities in young (20-50 years), middle-aged (50-70 years], and older adults (70-89 years]. Our models were built to recognize physical activity types, recognize physical activity intensities, estimate energy expenditure (EE) and recognize individual physical activities using wrist-worn tri-axial accelerometer data (33 activities per participant) from a large sample of participants (n = 253, 62% women, aged 20-89 years old). Results showed that the machine learning models were quite accurate at recognizing physical activity type and intensity and estimating energy expenditure. However, models performed less optimally when recognizing individual physical activities. F1-Scores derived from XGBoost's models were high for sedentary (0.955-0.973), locomotion (0.942-0.964) and lifestyle (0.913-0.949) activity types with no apparent difference across age groups. Low (0.919-0.947), light (0.813-0.828) and moderate (0.846-0.875) physical activity intensities were also recognized accurately. The root mean square error range for EE was approximately 1 equivalent of resting EE [0.835-1.009 METs]. Generally, random forest and XGBoost models outperformed other models. In conclusion, machine learning models to label physical activity types, activity intensity and energy expenditure are accurate and there are minimal differences in their performance across young, middle-aged and older adults.

Precision nutrition: A systematic literature review

Precision Nutrition research aims to use personal information about individuals or groups of individuals to deliver nutritional advice that, theoretically, would be more suitable than generic advice. Machine learning, a subbranch of Artificial Intelligence, has promise to aid in the development of predictive models that are suitable for Precision Nutrition. As such, recent research has applied machine learning algorithms, tools, and techniques in precision nutrition for different purposes. However, a systematic overview of the state-of-the-art on the use of machine learning in Precision Nutrition is lacking. Therefore, we carried out a Systematic Literature Review (SLR) to provide an overview of where and how machine learning has been used in Precision Nutrition from various aspects, what such machine learning models use as input features, what the availability status of the data used in the literature is, and how the models are evaluated. Nine research questions were defined in this study. We retrieved 4930 papers from electronic databases and 60 primary studies were selected to respond to the research questions. All of the selected primary studies were also briefly discussed in this article. Our results show that fifteen problems spread across seven domains of nutrition and health are present. Four machine learning tasks are seen in the form of regression, classification, recommendation and clustering, with most of these utilizing a supervised approach. In total, 30 algorithms were used, with 19 appearing more than once. Models were through the use of four groups of approaches and 23 evaluation metrics. Personalized approaches are promising to reduce the burden of these current problems in nutrition research, and the current review shows Machine Learning can be incorporated into Precision Nutrition research with high performance. Precision Nutrition researchers should consider incorporating Machine Learning into their methods to facilitate the integration of many complex features, allowing for the development of high-performance Precision Nutrition approaches.

Eyeblink Detection in the Field: A Proof of Concept Study of Two Mobile Optical Eye-Trackers

Introduction

High physical and cognitive strain, high pressure, and sleep deficit are part of daily life for military professionals and civilians working in physiologically demanding environments. As a result, cognitive and physical capacities decline and the risk of illness, injury, or accidents increases. Such unfortunate outcomes could be prevented by tracking real-time physiological information, revealing individuals’ objective fatigue levels. Oculometrics, and especially eyeblinks, have been shown to be promising biomarkers that reflect fatigue development. Head-mounted optical eye-trackers are a common method to monitor these oculometrics. However, studies measuring eyeblink detection in real-life settings have been lacking in the literature. Therefore, this study aims to validate two current mobile optical eye-trackers in an unrestrained military training environment.

Materials and Method

Three male participants (age 20.0 ± 1.0) of the Swiss Armed Forces participated in this study by wearing three optical eye-trackers, two VPS16s (Viewpointsystem GmbH, Vienna, Austria) and one Pupil Core (Pupil Labs GmbH, Berlin, Germany), during four military training events: Healthcare education, orienteering, shooting, and military marching. Software outputs were analyzed against a visual inspection (VI) of the video recordings of participants’ eyes via the respective software. Absolute and relative blink numbers were provided. Each blink detected by the software was classified as a “true blink” (TB) when it occurred in the software output and the VI at the same time, as a “false blink” (FB) when it occurred in the software but not in the VI, and as a “missed blink” (MB) when the software failed to detect a blink that occurred in the VI. The FBs were further examined for causes of the incorrect recordings, and they were divided into four categories: “sunlight,” “movements,” “lost pupil,” and “double-counted”. Blink frequency (i.e., blinks per minute) was also analyzed.

Results

Overall, 49.3% and 72.5% of registered eyeblinks were classified as TBs for the VPS16 and Pupil Core, respectively. The VPS16 recorded 50.7% of FBs and accounted for 8.5% of MBs, while the Pupil Core recorded 27.5% of FBs and accounted for 55.5% of MBs. The majority of FBs—45.5% and 73.9% for the VPS16 and Pupil Core, respectively—were erroneously recorded due to participants’ eye movements while looking up, down, or to one side. For blink frequency analysis, systematic biases (±limits of agreement) stood at 23.3 (±43.5) and −4.87 (±14.1) blinks per minute for the VPS16 and Pupil Core, respectively. Significant differences in systematic bias between devices and the respective VIs were found for nearly all activities (P < .05).

Conclusion

An objective physiological monitoring of fatigue is necessary for soldiers as well as civil professionals who are exposed to higher risks when their cognitive or physical capacities weaken. However, optical eye-trackers’ accuracy has not been specified under field conditions—especially not in monitoring fatigue. The significant overestimation and underestimation of the VPS16 and Pupil Core, respectively, demonstrate the general difficulty of blink detection in the field.

Determining Physical Activity Characteristics from Wristband Data for Use in Automated Insulin Delivery Systems

Algorithms that can determine the type of physical activity (PA) and quantify the intensity can allow precision medicine approaches, such as automated insulin delivery systems that modulate insulin administration in response to PA. In this work, data from a multi-sensor wristband is used to design classifiers to distinguish among five different physical states (PS) (resting, activities of daily living, running, biking, and resistance training), and to develop models to estimate the energy expenditure (EE) of the PA for diabetes therapy. The data collected are filtered, features are extracted from the reconciled signals, and the extracted features are used by machine learning algorithms, including deep-learning techniques, to obtain accurate PS classification and EE estimation. The various machine learning techniques have different success rates ranging from 75.7% to 94.8% in classifying the five different PS. The deep neural network model with long short-term memory has 94.8% classification accuracy. We achieved 0.5 MET (Metabolic Equivalent of Task) root-mean-square error for EE estimation accuracy, relative to indirect calorimetry with randomly selected testing data (10% of collected data). We also demonstrate a 5% improvement in PS classification accuracy and a 0.34 MET decrease in the mean absolute error when using multi-sensor approach relative to using only accelerometer data.