cStress: Towards a Gold Standard for Continuous Stress Assessment in the Mobile Environment

EmoWear: Wearable Physiological and Motion Dataset for Emotion Recognition and Context Awareness

The EmoWear dataset provides a bridge to explore Emotion Recognition (ER) via Seismocardiography (SCG), the measurement of small cardio-respiratory induced vibrations on the chest wall through Inertial Measurement Units (IMUs). We recorded Accelerometer (ACC), Gyroscope (GYRO), Electrocardiography (ECG), Blood Volume Pulse (BVP), Respiration (RSP), Electrodermal Activity (EDA), and Skin Temperature (SKT) data from 49 participants who watched validated emotionally stimulating video clips. They self-assessed their emotional valence, arousal, and dominance, as well as extra questions about the video clips. Also, we asked the participants to walk, talk, and drink, so that researchers can detect gait, voice, and swallowing using the same IMU. We demonstrate the effectiveness of emotion stimulation with statistical methods and verify the quality of the collected signals through signal-to-noise ratio and correlation analysis. EmoWear can be used for ER via SCG, ER during gait, multi-modal ER, and the study of IMUs for context-awareness. Targeted contextual information include emotions, gait, voice activity, and drinking, all having the potential to be sensed via a single IMU.

Improved method for stress detection using bio-sensor technology and machine learning algorithms

Maintaining an optimal stress level is vital in our lives, yet many individuals struggle to identify the sources of their stress. As emotional stability and mental awareness become increasingly important, wearable medical technology has gained popularity in recent years. This technology enables real-time monitoring, providing medical professionals with crucial physiological data to enhance patient care. Current stress-detection methods, such as ECG, BVP, and body movement analysis, are limited by their rigidity and susceptibility to noise interference. To overcome these limitations, we introduce STRESS-CARE, a versatile stress detection sensor employing a hybrid approach. This innovative system utilizes a sweat sensor, cutting-edge context identification methods, and machine learning algorithms. STRESS-CARE processes sensor data and models environmental fluctuations using an XG Boost classifier. By combining these advanced techniques, we aim to revolutionize stress detection, offering a more adaptive and robust solution for improved stress management and overall well-being.•In the proposed method, we introduce a state-of-the-art stress detection device with Galvanic Skin Response (GSR) sweat sensors, outperforming traditional Electrocardiogram (ECG) methods while remaining non-invasive•Integrating machine learning, particularly XG-Boost algorithms, enhances detection accuracy and reliability.•This study sheds light on noise context comprehension for various wearable devices, offering crucial guidance for optimizing stress detection in multiple contexts and applications.

Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study

BACKGROUND

As mobile health (mHealth) studies become increasingly productive owing to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. Many health constructs are dependent on subjective responses, and without such responses, researchers are left with little to no ground truth to accompany our ever-growing biobehavioral data. This issue can significantly impact the quality of a study, particularly for populations known to exhibit lower compliance rates. To address this challenge, researchers have proposed innovative approaches that use machine learning (ML) and sensor data to modify the timing and delivery of surveys. However, an overarching concern is the potential introduction of biases or unintended influences on participants' responses when implementing new survey delivery methods.

OBJECTIVE

This study aims to demonstrate the potential impact of an ML-based ecological momentary assessment (EMA) delivery system (using receptivity as the predictor variable) on the participants' reported emotional state. We examine the factors that affect participants' receptivity to EMAs in a 10-day wearable and EMA-based emotional state-sensing mHealth study. We study the physiological relationships indicative of receptivity and affect while also analyzing the interaction between the 2 constructs.

METHODS

We collected data from 45 healthy participants wearing 2 devices measuring electrodermal activity, accelerometer, electrocardiography, and skin temperature while answering 10 EMAs daily, containing questions about perceived mood. Owing to the nature of our constructs, we can only obtain ground truth measures for both affect and receptivity during responses. Therefore, we used unsupervised and supervised ML methods to infer affect when a participant did not respond. Our unsupervised method used k-means clustering to determine the relationship between physiology and receptivity and then inferred the emotional state during nonresponses. For the supervised learning method, we primarily used random forest and neural networks to predict the affect of unlabeled data points as well as receptivity.

RESULTS

Our findings showed that using a receptivity model to trigger EMAs decreased the reported negative affect by >3 points or 0.29 SDs in our self-reported affect measure, scored between 13 and 91. The findings also showed a bimodal distribution of our predicted affect during nonresponses. This indicates that this system initiates EMAs more commonly during states of higher positive emotions.

CONCLUSIONS

Our results showed a clear relationship between affect and receptivity. This relationship can affect the efficacy of an mHealth study, particularly those that use an ML algorithm to trigger EMAs. Therefore, we propose that future work should focus on a smart trigger that promotes EMA receptivity without influencing affect during sampled time points.

Relieving the burden of intensive labeling for stress monitoring in the wild by using semi-supervised learning

Stress, a natural process affecting individuals' wellbeing, has a profound impact on overall quality of life. Researchers from diverse fields employ various technologies and methodologies to investigate it and alleviate the negative effects of this phenomenon. Wearable devices, such as smart bands, capture physiological data, including heart rate variability, motions, and electrodermal activity, enabling stress level monitoring through machine learning models. However, labeling data for model accuracy assessment poses a significant challenge in stress-related research due to incomplete or inaccurate labels provided by individuals in their daily lives. To address this labeling predicament, our study proposes implementing Semi-Supervised Learning (SSL) models. Through comparisons with deep learning-based supervised models and clustering-based unsupervised models, we evaluate the performance of our SSL models. Our experiments show that our SSL models achieve 77% accuracy with a classifier trained on an augmented dataset prepared using the label propagation (LP) algorithm. Additionally, our deep autoencoder network achieves 76% accuracy. These results highlight the superiority of SSL models over unsupervised learning techniques and their comparable performance to supervised learning models, even with limited labeled data. By relieving the burden of labeling in daily life stress recognition, our study advances stress-related research, recognizing stress as a natural process rather than a disease. This facilitates the development of more efficient and accurate stress monitoring methods in the wild.

Application of a mobile health data platform for public health surveillance: A case study in stress monitoring and prediction

Background

Public health surveillance involves the collection, analysis and dissemination of data to improve population health. The main sources of data for public health decision-making are surveys, typically comprised of self-report which may be subject to biases, costs and delays. To complement subjective data, objective measures from sensors could potentially be used. Specifically, advancements in personal mobile and wearable technologies enable the collection of real-time and continuous health data.

Objective

In this context, the goal of this work is to apply a mobile health platform (MHP) that extracts health data from the Apple Health repository to collect data in daily-life scenarios and use it for the prediction of stress, a major public health issue.

Methods

A pilot study was conducted with 45 participants over 2 weeks, using the MHP to collect stress-related data from Apple Health and perceived stress self-reports. Apple, Withings and Empatica devices were distributed to participants and collected a wide range of data, including heart rate, sleep, blood pressure, temperature, and weight. These were used to train random forests and support vector machines. The SMOTE technique was used to handle imbalanced datasets.

Results

Accuracy and f1-macro scores were in line with state-of-the-art models for stress prediction above 60% for the majority of analyses and samples analysed. Apple Watch sleep features were particularly good predictors, with most models with these data achieving results around 70%.

Conclusions

A system such as the MHP could be used for public health data collection, complementing traditional self-reporting methods when possible. The data collected with the system was promising for monitoring and predicting stress in a population.

Predicting Obsessive-Compulsive Disorder Events in Children and Adolescents in the Wild using a Wearable Biosensor (Wrist Angel): Protocol for the Analysis Plan of a Nonrandomized Pilot Study

BACKGROUND

Physiological signals such as heart rate and electrodermal activity can provide insight into an individual's mental state, which are invaluable information for mental health care. Using recordings of physiological signals from wearable devices in the wild can facilitate objective monitoring of symptom severity and evaluation of treatment progress.

OBJECTIVE

We designed a study to evaluate the feasibility of predicting obsessive-compulsive disorder (OCD) events from physiological signals recorded using wrist-worn devices in the wild. Here, we present an analysis plan for the study to document our a priori hypotheses and increase the robustness of the findings of our planned study.

METHODS

In total, 18 children and adolescents aged between 8 and 16 years were included in this study. Nine outpatients with an OCD diagnosis were recruited from a child and adolescent mental health center. Nine youths without a psychiatric diagnosis were recruited from the catchment area. Patients completed a clinical interview to assess OCD severity, types of OCD, and number of OCD symptoms in the clinic. Participants wore a biosensor on their wrist for up to 8 weeks in their everyday lives. Patients were asked to press an event tag button on the biosensor when they were stressed by OCD symptoms. Participants without a psychiatric diagnosis were asked to press this button whenever they felt really scared. Before and after the 8-week observation period, participants wore the biosensor under controlled conditions of rest and stress in the clinic. Features are extracted from 4 different physiological signals within sliding windows to predict the distress event logged by participants during data collection. We will test the prediction models within participants across time and multiple participants. Model selection and estimation using 2-layer cross-validation are outlined for both scenarios.

RESULTS

Participants were included between December 2021 and December 2022. Participants included 10 female and 8 male participants with an even sex distribution between groups. Patients were aged between 10 and 16 years, and adolescents without a psychiatric diagnosis were between the ages of 8 and 16 years. Most patients had moderate to moderate to severe OCD, except for 1 patient with mild OCD.

CONCLUSIONS

The strength of the planned study is the investigation of predictions of OCD events in the wild. Major challenges of the study are the inherent noise of in-the-wild data and the lack of contextual knowledge associated with the recorded signals. This preregistered analysis plan discusses in detail how we plan to address these challenges and may help reduce interpretation bias of the upcoming results. If the obtained results from this study are promising, we will be closer to automated detection of OCD events outside of clinical experiments. This is an important tool for the assessment and treatment of OCD in youth.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05064527; https://clinicaltrials.gov/study/NCT05064527.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/48571.

Detecting Prolonged Stress in Real Life Using Wearable Biosensors and Ecological Momentary Assessments: Naturalistic Experimental Study

BACKGROUND

Increasing efforts toward the prevention of stress-related mental disorders have created a need for unobtrusive real-life monitoring of stress-related symptoms. Wearable devices have emerged as a possible solution to aid in this process, but their use in real-life stress detection has not been systematically investigated.

OBJECTIVE

We aimed to determine the utility of ecological momentary assessments (EMA) and physiological arousal measured through wearable devices in detecting ecologically relevant stress states.

METHODS

Using EMA combined with wearable biosensors for ecological physiological assessments (EPA), we investigated the impact of an ecological stressor (ie, a high-stakes examination week) on physiological arousal and affect compared to a control week without examinations in first-year medical and biomedical science students (51/83, 61.4% female). We first used generalized linear mixed-effects models with maximal fitting approaches to investigate the impact of examination periods on subjective stress exposure, mood, and physiological arousal. We then used machine learning models to investigate whether we could use EMA, wearable biosensors, or the combination of both to classify momentary data (ie, beeps) as belonging to examination or control weeks. We tested both individualized models using a leave-one-beep-out approach and group-based models using a leave-one-subject-out approach.

RESULTS

During stressful high-stakes examination (versus control) weeks, participants reported increased negative affect and decreased positive affect. Intriguingly, physiological arousal decreased on average during the examination week. Time-resolved analyses revealed peaks in physiological arousal associated with both momentary self-reported stress exposure and self-reported positive affect. Mediation models revealed that the decreased physiological arousal in the examination week was mediated by lower positive affect during the same period. We then used machine learning to show that while individualized EMA outperformed EPA in its ability to classify beeps as originating from examinations or from control weeks (1603/4793, 33.45% and 1648/4565, 36.11% error rates, respectively), a combination of EMA and EPA yields optimal classification (1363/4565, 29.87% error rate). Finally, when comparing individualized models to group-based models, we found that the individualized models significantly outperformed the group-based models across all 3 inputs (EMA, EPA, and the combination).

CONCLUSIONS

This study underscores the potential of wearable biosensors for stress-related mental health monitoring. However, it emphasizes the necessity of psychological context in interpreting physiological arousal captured by these devices, as arousal can be related to both positive and negative contexts. Moreover, our findings support a personalized approach in which momentary stress is optimally detected when referenced against an individual's own data.

Detecting Social Contexts from Mobile Sensing Indicators in Virtual Interactions with Socially Anxious Individuals

Mobile sensing is a ubiquitous and useful tool to make inferences about individuals' mental health based on physiology and behavior patterns. Along with sensing features directly associated with mental health, it can be valuable to detect different features of social contexts to learn about social interaction patterns over time and across different environments. This can provide insight into diverse communities' academic, work and social lives, and their social networks. We posit that passively detecting social contexts can be particularly useful for social anxiety research, as it may ultimately help identify changes in social anxiety status and patterns of social avoidance and withdrawal. To this end, we recruited a sample of highly socially anxious undergraduate students (N=46) to examine whether we could detect the presence of experimentally manipulated virtual social contexts via wristband sensors. Using a multitask machine learning pipeline, we leveraged passively sensed biobehavioral streams to detect contexts relevant to social anxiety, including (1) whether people were in a social situation, (2) size of the social group, (3) degree of social evaluation, and (4) phase of social situation (anticipating, actively experiencing, or had just participated in an experience). Results demonstrated the feasibility of detecting most virtual social contexts, with stronger predictive accuracy when detecting whether individuals were in a social situation or not and the phase of the situation, and weaker predictive accuracy when detecting the level of social evaluation. They also indicated that sensing streams are differentially important to prediction based on the context being predicted. Our findings also provide useful information regarding design elements relevant to passive context detection, including optimal sensing duration, the utility of different sensing modalities, and the need for personalization. We discuss implications of these findings for future work on context detection (e.g., just-in-time adaptive intervention development).

Wearables for Stress Management: A Scoping Review

In recent years, wearable devices have been increasingly used to monitor people's health. This has helped healthcare professionals provide timely interventions to support their patients. In this study, we investigated how wearables help people manage stress. We conducted a scoping review following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) standard to address this question. We searched studies in Scopus, IEEE Explore, and Pubmed databases. We included studies reporting user evaluations of wearable-based strategies, reporting their impact on health or usability outcomes. A total of 6259 studies were identified, of which 40 met the inclusion criteria. Based on our findings, we identified that 21 studies report using commercial wearable devices; the most common are smartwatches and smart bands. Thirty-one studies report significant stress reduction using different interventions and interaction modalities. Finally, we identified that the interventions are designed with the following aims: (1) to self-regulate during stress episodes, (2) to support self-regulation therapies for long-term goals, and (3) to provide stress awareness for prevention, consisting of people's ability to recall, recognize and understand their stress.

Assessing the Impact of Context Inference Error and Partial Observability on RL Methods for Just-In-Time Adaptive Interventions

Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized health interventions developed within the behavioral science community. JITAIs aim to provide the right type and amount of support by iteratively selecting a sequence of intervention options from a pre-defined set of components in response to each individual's time varying state. In this work, we explore the application of reinforcement learning methods to the problem of learning intervention option selection policies. We study the effect of context inference error and partial observability on the ability to learn effective policies. Our results show that the propagation of uncertainty from context inferences is critical to improving intervention efficacy as context uncertainty increases, while policy gradient algorithms can provide remarkable robustness to partially observed behavioral state information.

On the Reliability of Wearable Technology: A Tutorial on Measuring Heart Rate and Heart Rate Variability in the Wild

Wearable sensors are quickly making their way into psychophysiological research, as they allow collecting data outside of a laboratory and for an extended period of time. The present tutorial considers fidelity of physiological measurement with wearable sensors, focusing on reliability. We elaborate on why ensuring reliability for wearables is important and offer statistical tools for assessing wearable reliability for between participants and within-participant designs. The framework offered here is illustrated using several brands of commercially available heart rate sensors. Measurement reliability varied across sensors and, more importantly, across the situations tested, and was highest during sleep. Our hope is that by systematically quantifying measurement reliability, researchers will be able to make informed choices about specific wearable devices and measurement procedures that meet their research goals.

K-EmoPhone: A Mobile and Wearable Dataset with In-Situ Emotion, Stress, and Attention Labels

With the popularization of low-cost mobile and wearable sensors, several studies have used them to track and analyze mental well-being, productivity, and behavioral patterns. However, there is still a lack of open datasets collected in real-world contexts with affective and cognitive state labels such as emotion, stress, and attention; the lack of such datasets limits research advances in affective computing and human-computer interaction. This study presents K-EmoPhone, a real-world multimodal dataset collected from 77 students over seven days. This dataset contains (1) continuous probing of peripheral physiological signals and mobility data measured by commercial off-the-shelf devices, (2) context and interaction data collected from individuals' smartphones, and (3) 5,582 self-reported affect states, including emotions, stress, attention, and task disturbance, acquired by the experience sampling method. We anticipate the dataset will contribute to advancements in affective computing, emotion intelligence technologies, and attention management based on mobile and wearable sensor data.

Stress and Workload Assessment in Aviation-A Narrative Review

In aviation, any detail can have massive consequences. Among the potential sources of failure, human error is still the most troublesome to handle. Therefore, research concerning the management of mental workload, attention, and stress is of special interest in aviation. Recognizing conditions in which a pilot is over-challenged or cannot act lucidly could avoid serious outcomes. Furthermore, knowing in depth a pilot's neurophysiological and cognitive-behavioral responses could allow for the optimization of equipment and procedures to minimize risk and increase safety. In addition, it could translate into a general enhancement of both the physical and mental well-being of pilots, producing a healthier and more ergonomic work environment. This review brings together literature on the study of stress and workload in the specific case of pilots of both civil and military aircraft. The most common approaches for studying these phenomena in the avionic context are explored in this review, with a focus on objective methodologies (e.g., the collection and analysis of neurophysiological signals). This review aims to identify the pros, cons, and applicability of the various approaches, to enable the design of an optimal protocol for a comprehensive study of these issues.

Common Mental Disorders in Smart City Settings and Use of Multimodal Medical Sensor Fusion to Detect Them

Cities have undergone numerous permanent transformations at times of severe disruption. The Lisbon earthquake of 1755, for example, sparked the development of seismic construction rules. In 1848, when cholera spread through London, the first health law in the United Kingdom was passed. The Chicago fire of 1871 led to stricter building rules, which led to taller skyscrapers that were less likely to catch fire. Along similar lines, the COVID-19 epidemic may have a lasting effect, having pushed the global shift towards greener, more digital, and more inclusive cities. The pandemic highlighted the significance of smart/remote healthcare. Specifically, the elderly delayed seeking medical help for fear of contracting the infection. As a result, remote medical services were seen as a key way to keep healthcare services running smoothly. When it comes to both human and environmental health, cities play a critical role. By concentrating people and resources in a single location, the urban environment generates both health risks and opportunities to improve health. In this manuscript, we have identified the most common mental disorders and their prevalence rates in cities. We have also identified the factors that contribute to the development of mental health issues in urban spaces. Through careful analysis, we have found that multimodal feature fusion is the best method for measuring and analysing multiple signal types in real time. However, when utilizing multimodal signals, the most important issue is how we might combine them; this is an area of burgeoning research interest. To this end, we have highlighted ways to combine multimodal features for detecting and predicting mental issues such as anxiety, mood state recognition, suicidal tendencies, and substance abuse.

AI-Enabled Smart Wristband Providing Real-Time Vital Signs and Stress Monitoring

This work introduces the design, architecture, implementation, and testing of a low-cost and machine-learning-enabled device to be worn on the wrist. The suggested wearable device has been developed for use during emergency incidents of large passenger ship evacuations, and enables the real-time monitoring of the passengers' physiological state, and stress detection. Based on a properly preprocessed PPG signal, the device provides essential biometric data (pulse rate and oxygen saturation level) and an efficient unimodal machine learning pipeline. The stress detecting machine learning pipeline is based on ultra-short-term pulse rate variability, and has been successfully integrated into the microcontroller of the developed embedded device. As a result, the presented smart wristband is able to provide real-time stress detection. The stress detection system has been trained with the use of the publicly available WESAD dataset, and its performance has been tested through a two-stage process. Initially, evaluation of the lightweight machine learning pipeline on a previously unseen subset of the WESAD dataset was performed, reaching an accuracy score equal to 91%. Subsequently, external validation was conducted, through a dedicated laboratory study of 15 volunteers subjected to well-acknowledged cognitive stressors while wearing the smart wristband, which yielded an accuracy score equal to 76%.

Cross Dataset Analysis for Generalizability of HRV-Based Stress Detection Models

Stress is an increasingly prevalent mental health condition across the world. In Europe, for example, stress is considered one of the most common health problems, and over USD 300 billion are spent on stress treatments annually. Therefore, monitoring, identification and prevention of stress are of the utmost importance. While most stress monitoring is carried out through self-reporting, there are now several studies on stress detection from physiological signals using Artificial Intelligence algorithms. However, the generalizability of these models is only rarely discussed. The main goal of this work is to provide a monitoring proof-of-concept tool exploring the generalization capabilities of Heart Rate Variability-based machine learning models. To this end, two Machine Learning models are used, Logistic Regression and Random Forest to analyze and classify stress in two datasets differing in terms of protocol, stressors and recording devices. First, the models are evaluated using leave-one-subject-out cross-validation with train and test samples from the same dataset. Next, a cross-dataset validation of the models is performed, that is, leave-one-subject-out models trained on a Multi-modal Dataset for Real-time, Continuous Stress Detection from Physiological Signals dataset and validated using the University of Waterloo stress dataset. While both logistic regression and random forest models achieve good classification results in the independent dataset analysis, the random forest model demonstrates better generalization capabilities with a stable F1 score of 61%. This indicates that the random forest can be used to generalize HRV-based stress detection models, which can lead to better analyses in the mental health and medical research field through training and integrating different models.

Aziz NA, Abd. Aziz A. Augmenting ECG Data with Multiple Filters for a Better Emotion Recognition System

A physiological-based emotion recognition system (ERS) with a unimodal approach such as an electrocardiogram (ECG) is not as popular compared to a multimodal approach. However, a single modality has the advantage of lower development and computational cost. Therefore, this study focuses on a unimodal ECG-based ERS. The ECG-based ERS has the potential to become the next mass-adopted consumer application due to the wide availability of wearable and mobile ECG devices in the market. Currently, ECG-inclusive affective datasets are limited, and many of the existing datasets have small sample sizes. Hence, ECG-based ERS studies are stunted by the lack of quality data. A novel multi-filtering augmentation technique is proposed here to increase the sample size of the ECG data. This technique augments the ECG signals by cleaning the data in different ways. Three small ECG datasets labelled according to emotion state are used in this study. The benefit of the proposed augmentation techniques is measured using the classification accuracy of five machine learning algorithms; k-nearest neighbours (KNN), support vector machine, decision tree, random forest and multilayer perceptron. The results show that with the proposed technique, there is a significant improvement in performance for all the datasets and classifiers. KNN classifier improved the most with the augmented data and the reported classification accuracies of over 90%.

A Just-In-Time Adaptive intervention (JITAI) for smoking cessation: Feasibility and acceptability findings

Smoking cessation treatments that are easily accessible and deliver intervention content at vulnerable moments (e.g., high negative affect) have great potential to impact tobacco abstinence. The current study examined the feasibility and acceptability of a multi-component Just-In-Time Adaptive Intervention (JITAI) for smoking cessation. Daily smokers interested in quitting were consented to participate in a 6-week cessation study. Visit 1 occurred 4 days pre-quit, Visit 2 was on the quit day, Visit 3 occurred 3 days post-quit, Visit 4 was 10 days post-quit, and Visit 5 was 28 days post-quit. During the first 2 weeks (Visits 1-4), the JITAI delivered brief mindfulness/motivational strategies via smartphone in real-time based on negative affect or smoking behavior detected by wearable sensors. Participants also attended 5 in-person visits, where brief cessation counseling (Visits 1-4) and nicotine replacement therapy (Visits 2-5) were provided. Outcomes were feasibility and acceptability; biochemically-confirmed abstinence was also measured. Participants (N = 43) were 58.1 % female (AgeMean = 49.1, mean cigarettes per day = 15.4). Retention through follow-up was high (83.7 %). For participants with available data (n = 38), 24 (63 %) met the benchmark for sensor wearing, among whom 16 (67 %) completed at least 60 % of strategies. Perceived ease of wearing sensors (Mean = 5.1 out of 6) and treatment satisfaction (Mean = 3.6 out of 4) were high. Biochemically-confirmed abstinence was 34 % at Visit 4 and 21 % at Visit 5. Overall, the feasibility of this novel multi-component intervention for smoking cessation was mixed but acceptability was high. Future studies with improved technology will decrease participant burden and better detect key intervention moments.

Feasibility of Using an Armband Optical Heart Rate Sensor in Naturalistic Environment

Consumer-grade heart rate (HR) sensors including chest straps, wrist-worn watches and rings have become very popular in recent years for tracking individual physiological state, training for sports and even measuring stress levels and emotional changes. While the majority of these consumer sensors are not medical devices, they can still offer insights for consumers and researchers if used correctly taking into account their limitations. Multiple previous studies have been done using a large variety of consumer sensors including Polar® devices, Apple® watches, and Fitbit® wrist bands. The vast majority of prior studies have been done in laboratory settings where collecting data is relatively straightforward. However, using consumer sensors in naturalistic settings that present significant challenges, including noise artefacts and missing data, has not been as extensively investigated. Additionally, the majority of prior studies focused on wrist-worn optical HR sensors. Arm-worn sensors have not been extensively investigated either. In the present study, we validate HR measurements obtained with an arm-worn optical sensor (Polar OH1) against those obtained with a chest-strap electrical sensor (Polar H10) from 16 participants over a 2-week study period in naturalistic settings. We also investigated the impact of physical activity measured with 3-D accelerometers embedded in the H10 chest strap and OH1 armband sensors on the agreement between the two sensors. Overall, we find that the arm-worn optical Polar OH1 sensor provides a good estimate of HR (Pearson r = 0.90, p <0.01). Filtering the signal that corresponds to physical activity further improves the HR estimates but only slightly (Pearson r = 0.91, p <0.01). Based on these preliminary findings, we conclude that the arm-worn Polar OH1 sensor provides usable HR measurements in daily living conditions, with some caveats discussed in the paper.

ADABase: A Multimodal Dataset for Cognitive Load Estimation

Driver monitoring systems play an important role in lower to mid-level autonomous vehicles. Our work focuses on the detection of cognitive load as a component of driver-state estimation to improve traffic safety. By inducing single and dual-task workloads of increasing intensity on 51 subjects, while continuously measuring signals from multiple modalities, based on physiological measurements such as ECG, EDA, EMG, PPG, respiration rate, skin temperature and eye tracker data, as well as behavioral measurements such as action units extracted from facial videos, performance metrics like reaction time and subjective feedback using questionnaires, we create ADABase (Autonomous Driving Cognitive Load Assessment Database) As a reference method to induce cognitive load onto subjects, we use the well-established n-back test, in addition to our novel simulator-based k-drive test, motivated by real-world semi-autonomously vehicles. We extract expert features of all measurements and find significant changes in multiple modalities. Ultimately we train and evaluate machine learning algorithms using single and multimodal inputs to distinguish cognitive load levels. We carefully evaluate model behavior and study feature importance. In summary, we introduce a novel cognitive load test, create a cognitive load database, validate changes using statistical tests, introduce novel classification and regression tasks for machine learning and train and evaluate machine learning models.

mRisk: Continuous Risk Estimation for Smoking Lapse from Noisy Sensor Data with Incomplete and Positive-Only Labels

Passive detection of risk factors (that may influence unhealthy or adverse behaviors) via wearable and mobile sensors has created new opportunities to improve the effectiveness of behavioral interventions. A key goal is to find opportune moments for intervention by passively detecting rising risk of an imminent adverse behavior. But, it has been difficult due to substantial noise in the data collected by sensors in the natural environment and a lack of reliable label assignment of low- and high-risk states to the continuous stream of sensor data. In this paper, we propose an event-based encoding of sensor data to reduce the effect of noises and then present an approach to efficiently model the historical influence of recent and past sensor-derived contexts on the likelihood of an adverse behavior. Next, to circumvent the lack of any confirmed negative labels (i.e., time periods with no high-risk moment), and only a few positive labels (i.e., detected adverse behavior), we propose a new loss function. We use 1,012 days of sensor and self-report data collected from 92 participants in a smoking cessation field study to train deep learning models to produce a continuous risk estimate for the likelihood of an impending smoking lapse. The risk dynamics produced by the model show that risk peaks an average of 44 minutes before a lapse. Simulations on field study data show that using our model can create intervention opportunities for 85% of lapses with 5.5 interventions per day.

Predicting the Next-Day Perceived and Physiological Stress of Pregnant Women by Using Machine Learning and Explainability: Algorithm Development and Validation

Background

Cognitive behavioral therapy–based interventions are effective in reducing prenatal stress, which can have severe adverse health effects on mothers and newborns if unaddressed. Predicting next-day physiological or perceived stress can help to inform and enable pre-emptive interventions for a likely physiologically and perceptibly stressful day. Machine learning models are useful tools that can be developed to predict next-day physiological and perceived stress by using data collected from the previous day. Such models can improve our understanding of the specific factors that predict physiological and perceived stress and allow researchers to develop systems that collect selected features for assessment in clinical trials to minimize the burden of data collection.

Objective

The aim of this study was to build and evaluate a machine-learned model that predicts next-day physiological and perceived stress by using sensor-based, ecological momentary assessment (EMA)–based, and intervention-based features and to explain the prediction results.

Methods

We enrolled pregnant women into a prospective proof-of-concept study and collected electrocardiography, EMA, and cognitive behavioral therapy intervention data over 12 weeks. We used the data to train and evaluate 6 machine learning models to predict next-day physiological and perceived stress. After selecting the best performing model, Shapley Additive Explanations were used to identify the feature importance and explainability of each feature.

Results

A total of 16 pregnant women enrolled in the study. Overall, 4157.18 hours of data were collected, and participants answered 2838 EMAs. After applying feature selection, 8 and 10 features were found to positively predict next-day physiological and perceived stress, respectively. A random forest classifier performed the best in predicting next-day physiological stress (F1 score of 0.84) and next-day perceived stress (F1 score of 0.74) by using all features. Although any subset of sensor-based, EMA-based, or intervention-based features could reliably predict next-day physiological stress, EMA-based features were necessary to predict next-day perceived stress. The analysis of explainability metrics showed that the prolonged duration of physiological stress was highly predictive of next-day physiological stress and that physiological stress and perceived stress were temporally divergent.

Conclusions

In this study, we were able to build interpretable machine learning models to predict next-day physiological and perceived stress, and we identified unique features that were highly predictive of next-day stress that can help to reduce the burden of data collection.

Detecting Physiological Responses Using Multimodal Earbud Sensors

Continuous stress exposure negatively impacts mental and physical well-being. Physiological arousal due to stress affects heartbeat frequency, changes breathing pattern and peripheral temperature, among several other bodily responses. Traditionally stress detection is performed by collecting signals such as electrocardiogram (ECG), respiration, and skin conductance response using uncomfortable sensors such as a chestband. In this study, we use earbuds that passively measure photoplethysmography (PPG), core body temperature, and inertial measurements. We have conducted a lab study exposing 18 participants to an evaluated speech task and additional tasks aimed at increasing stress or promoting relaxation. We simultaneously collected PPG, ECG, impedance cardiography (ICG), and blood pressure using laboratory grade equipment as reference measurements. We show that the earbud PPG sensor can reliably capture heart rate and heart rate variability. We further show that earbud signals can be used to classify the physiological responses associated with stress with 91.30% recall, 80.52% precision, and 85.12% F1-score using a random forest classifier with leave-one-subject-out cross-validation. The accuracy can further be improved through multi-modal sensing. These findings demonstrate the feasibility of using earbuds for passively monitoring users' physiological responses.

Data-driven Interpretable Policy Construction for Personalized Mobile Health

To promote healthy behaviors, many mobile health applications provide message-based interventions, such as tips, motivational messages, or suggestions for healthy activities. Ideally, the intervention policies should be carefully designed so that users obtain the benefits without being overwhelmed by overly frequent messages. As part of the HeartSteps physical-activity intervention, users receive messages intended to disrupt sedentary behavior. HeartSteps uses an algorithm to uniformly spread out the daily message budget over time, but does not attempt to maximize treatment effects. This limitation motivates constructing a policy to optimize the message delivery decisions for more effective treatments. Moreover, the learned policy needs to be interpretable to enable behavioral scientists to examine it and to inform future theorizing. We address this problem by learning an effective and interpretable policy that reduces sedentary behavior. We propose Optimal Policy Trees + (OPT+), an innovative batch off-policy learning method, that combines a personalized threshold learning and an extension of Optimal Policy Trees under a budget-constrained setting. We implement and test the method using data collected in HeartSteps V2/V3. Computational results demonstrate a significant reduction in sedentary behavior with a lower delivery budget. OPT+ produces a highly interpretable and stable output decision tree thus enabling theoretical insights to guide future research.

Exploring Unsupervised Machine Learning Classification Methods for Physiological Stress Detection

Over the past decade, there has been a significant development in wearable health technologies for diagnosis and monitoring, including application to stress monitoring. Most of the wearable stress monitoring systems are built on a supervised learning classification algorithm. These systems rely on the collection of sensor and reference data during the development phase. One of the most challenging tasks in physiological or pathological stress monitoring is the labeling of the physiological signals collected during an experiment. Commonly, different types of self-reporting questionnaires are used to label the perceived stress instances. These questionnaires only capture stress levels at a specific point in time. Moreover, self-reporting is subjective and prone to inaccuracies. This paper explores the potential feasibility of unsupervised learning clustering classifiers such as Affinity Propagation, Balanced Iterative Reducing and Clustering using Hierarchies (BIRCH), K-mean, Mini-Batch K-mean, Mean Shift, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) and Ordering Points To Identify the Clustering Structure (OPTICS) for implementation in stress monitoring wearable devices. Traditional supervised machine learning (linear, ensembles, trees, and neighboring models) classifiers require hand-crafted features and labels while on the other hand, the unsupervised classifier does not require any labels of perceived stress levels and performs classification based on clustering algorithms. The classification results of unsupervised machine learning classifiers are found comparable to supervised machine learning classifiers on two publicly available datasets. The analysis and results of this comparative study demonstrate the potential of unsupervised learning for the development of non-invasive, continuous, and robust detection and monitoring of physiological and pathological stress.

Individual Decision Model for Using technology in Digital Era

Digital transformation has brought about great social changes, and individuals are constantly facing the challenge of using emerging technologies. This article, for the first time, combines the Diffusion of Innovation Theory and Contract Theory to build a decision model to solve the above challenge. The decision model is constructed according to the key factors that influence the individual decision process, including technological relative advantages, intrinsic motivation, risk-taking, use-cost, technological complexity and compatibility. Through the analysis of the cost utility of each party in Health CrowdSensing technology, the question of whether individuals use the technology is transformed into the question of cost utility. In the experiments, the validity of the decision model is verified by numerical analysis. The decision model proposed in this article provides theoretical basis and experimental verification for further research on how an individual decides whether to use technology or not.

Atrial Fibrillation Detection and Atrial Fibrillation Burden Estimation via Wearables

Atrial Fibrillation (AF) is an important cardiac rhythm disorder, which if left untreated can lead to serious complications such as a stroke. AF can remain asymptomatic, and it can progressively worsen over time; it is thus a disorder that would benefit from detection and continuous monitoring with a wearable sensor. We develop an AF detection algorithm, deploy it on a smartwatch, and prospectively and comprehensively validate its performance on a real-world population that included patients diagnosed with AF. The algorithm showed a sensitivity of 87.8% and a specificity of 97.4% over every 5-minute segment of PPG evaluated. Furthermore, we introduce novel algorithm blocks and system designs to increase the time of coverage and monitor for AF even during periods of motion noise and other artifacts that would be encountered in daily-living scenarios. An average of 67.8% of the entire duration the patients wore the smartwatch produced a valid decision. Finally, we present the ability of our algorithm to function throughout the day and estimate the AF burden, a first-of-this-kind measure using a wearable sensor, showing 98% correlation with the ground truth and an average error of 6.2%.

Dynamic models of stress-smoking responses based on high-frequency sensor data

Self-reports indicate that stress increases the risk for smoking; however, intensive data from sensors can provide a more nuanced understanding of stress in the moments leading up to and following smoking events. Identifying personalized dynamical models of stress-smoking responses can improve characterizations of smoking responses following stress, but techniques used to identify these models require intensive longitudinal data. This study leveraged advances in wearable sensing technology and digital markers of stress and smoking to identify person-specific models of stress and smoking system dynamics by considering stress immediately before, during, and after smoking events. Adult smokers (n = 45) wore the AutoSense chestband (respiration-inductive plethysmograph, electrocardiogram, accelerometer) with MotionSense (accelerometers, gyroscopes) on each wrist for three days prior to a quit attempt. The odds of minute-level smoking events were regressed on minute-level stress probabilities to identify person-specific dynamic models of smoking responses to stress. Simulated pulse responses to a continuous stress episode revealed a consistent pattern of increased odds of smoking either shortly after the beginning of the simulated stress episode or with a delay, for all participants. This pattern is followed by a dramatic reduction in the probability of smoking thereafter, for about half of the participants (49%). Sensor-detected stress probabilities indicate a vulnerability for smoking that may be used as a tailoring variable for just-in-time interventions to support quit attempts.

The association between self-reported stress and cardiovascular measures in daily life: A systematic review

BACKGROUND

Stress plays an important role in the development of mental illness, and an increasing number of studies is trying to detect moments of perceived stress in everyday life based on physiological data gathered using ambulatory devices. However, based on laboratory studies, there is only modest evidence for a relationship between self-reported stress and physiological ambulatory measures. This descriptive systematic review evaluates the evidence for studies investigating an association between self-reported stress and physiological measures under daily life conditions.

METHODS

Three databases were searched for articles assessing an association between self-reported stress and cardiovascular and skin conductance measures simultaneously over the course of at least a day.

RESULTS

We reviewed findings of 36 studies investigating an association between self-reported stress and cardiovascular measures with overall 135 analyses of associations between self-reported stress and cardiovascular measures. Overall, 35% of all analyses showed a significant or marginally significant association in the expected direction. The most consistent results were found for perceived stress, high-arousal negative affect scales, and event-related self-reported stress measures, and for frequency-domain heart rate variability physiological measures. There was much heterogeneity in measures and methods.

CONCLUSION

These findings confirm that daily-life stress-dynamics are complex and require a better understanding. Choices in design and measurement seem to play a role. We provide some guidance for future studies.

Feature Augmented Hybrid CNN for Stress Recognition Using Wrist-based Photoplethysmography Sensor

Stress is a physiological state that hampers mental health and has serious consequences to physical health. More-over, the COVID-19 pandemic has increased stress levels among people across the globe. Therefore, continuous monitoring and detection of stress are necessary. The recent advances in wearable devices have allowed the monitoring of several physiological signals related to stress. Among them, wrist-worn wearable devices like smartwatches are most popular due to their convenient usage. And the photoplethysmography (PPG) sensor is the most prevalent sensor in almost all consumer-grade wrist-worn smartwatches. Therefore, this paper focuses on using a wrist-based PPG sensor that collects Blood Volume Pulse (BVP) signals to detect stress which may be applicable for consumer-grade wristwatches. Moreover, state-of-the-art works have used either classical machine learning algorithms to detect stress using hand-crafted features or have used deep learning algorithms like Convolutional Neural Network (CNN) which automatically extracts features. This paper proposes a novel hybrid CNN (H-CNN) classifier that uses both the hand-crafted features and the automatically extracted features by CNN to detect stress using the BVP signal. Evaluation on the benchmark WESAD dataset shows that, for 3-class classification (Baseline vs. Stress vs. Amusement), our proposed H-CNN outperforms traditional classifiers and normal CNN by ≈5% and ≈7% accuracy, and ≈10% and ≈7% macro F1 score, respectively. Also for 2-class classification (Stress vs. Non-stress), our proposed H-CNN outperforms traditional classifiers and normal CNN by ≈3% and ≈5% accuracy, and ≈3% and ≈7% macro F1score, respectively.

Robotic-Based Well-Being Monitoring and Coaching System for the Elderly in Their Daily Activities

The increasingly ageing population and the tendency to live alone have led science and engineering researchers to search for health care solutions. In the COVID 19 pandemic, the elderly have been seriously affected in addition to suffering from isolation and its associated and psychological consequences. This paper provides an overview of the RobWell (Robotic-based Well-Being Monitoring and Coaching System for the Elderly in their Daily Activities) system. It is a system focused on the field of artificial intelligence for mood prediction and coaching. This paper presents a general overview of the initially proposed system as well as the preliminary results related to the home automation subsystem, autonomous robot navigation and mood estimation through machine learning prior to the final system integration, which will be discussed in future works. The main goal is to improve their mental well-being during their daily household activities. The system is composed of ambient intelligence with intelligent sensors, actuators and a robotic platform that interacts with the user. A test smart home system was set up in which the sensors, actuators and robotic platform were integrated and tested. For artificial intelligence applied to mood prediction, we used machine learning to classify several physiological signals into different moods. In robotics, it was concluded that the ROS autonomous navigation stack and its autodocking algorithm were not reliable enough for this task, while the robot’s autonomy was sufficient. Semantic navigation, artificial intelligence and computer vision alternatives are being sought.

Electrocardiogram-Based Emotion Recognition Systems and Their Applications in Healthcare-A Review

Affective computing is a field of study that integrates human affects and emotions with artificial intelligence into systems or devices. A system or device with affective computing is beneficial for the mental health and wellbeing of individuals that are stressed, anguished, or depressed. Emotion recognition systems are an important technology that enables affective computing. Currently, there are a lot of ways to build an emotion recognition system using various techniques and algorithms. This review paper focuses on emotion recognition research that adopted electrocardiograms (ECGs) as a unimodal approach as well as part of a multimodal approach for emotion recognition systems. Critical observations of data collection, pre-processing, feature extraction, feature selection and dimensionality reduction, classification, and validation are conducted. This paper also highlights the architectures with accuracy of above 90%. The available ECG-inclusive affective databases are also reviewed, and a popularity analysis is presented. Additionally, the benefit of emotion recognition systems towards healthcare systems is also reviewed here. Based on the literature reviewed, a thorough discussion on the subject matter and future works is suggested and concluded. The findings presented here are beneficial for prospective researchers to look into the summary of previous works conducted in the field of ECG-based emotion recognition systems, and for identifying gaps in the area, as well as in developing and designing future applications of emotion recognition systems, especially in improving healthcare.

Detecting Receptivity for mHealth Interventions in the Natural Environment

Just-In-Time Adaptive Intervention (JITAI) is an emerging technique with great potential to support health behavior by providing the right type and amount of support at the right time. A crucial aspect of JITAIs is properly timing the delivery of interventions, to ensure that a user is receptive and ready to process and use the support provided. Some prior works have explored the association of context and some user-specific traits on receptivity, and have built post-study machine-learning models to detect receptivity. For effective intervention delivery, however, a JITAI system needs to make in-the-moment decisions about a user's receptivity. To this end, we conducted a study in which we deployed machine-learning models to detect receptivity in the natural environment, i.e., in free-living conditions. We leveraged prior work regarding receptivity to JITAIs and deployed a chatbot-based digital coach - Ally - that provided physical-activity interventions and motivated participants to achieve their step goals. We extended the original Ally app to include two types of machine-learning model that used contextual information about a person to predict when a person is receptive: a static model that was built before the study started and remained constant for all participants and an adaptive model that continuously learned the receptivity of individual participants and updated itself as the study progressed. For comparison, we included a control model that sent intervention messages at random times. The app randomly selected a delivery model for each intervention message. We observed that the machine-learning models led up to a 40% improvement in receptivity as compared to the control model. Further, we evaluated the temporal dynamics of the different models and observed that receptivity to messages from the adaptive model increased over the course of the study.

Smart Devices and Wearable Technologies to Detect and Monitor Mental Health Conditions and Stress: A Systematic Review

Recently, there has been an increase in the production of devices to monitor mental health and stress as means for expediting detection, and subsequent management of these conditions. The objective of this review is to identify and critically appraise the most recent smart devices and wearable technologies used to identify depression, anxiety, and stress, and the physiological process(es) linked to their detection. The MEDLINE, CINAHL, Cochrane Central, and PsycINFO databases were used to identify studies which utilised smart devices and wearable technologies to detect or monitor anxiety, depression, or stress. The included articles that assessed stress and anxiety unanimously used heart rate variability (HRV) parameters for detection of anxiety and stress, with the latter better detected by HRV and electroencephalogram (EGG) together. Electrodermal activity was used in recent studies, with high accuracy for stress detection; however, with questionable reliability. Depression was found to be largely detected using specific EEG signatures; however, devices detecting depression using EEG are not currently available on the market. This systematic review highlights that average heart rate used by many commercially available smart devices is not as accurate in the detection of stress and anxiety compared with heart rate variability, electrodermal activity, and possibly respiratory rate.

Mobile Health: making the leap to research and clinics

Health applications for mobile and wearable devices continue to experience tremendous growth both in the commercial and research sectors, but their impact on healthcare has yet to be fully realized. This commentary introduces three articles in a special issue that provides guidance on how to successfully address translational barriers to bringing mobile health technologies into clinical research and care. We also discuss how the cross-organizational sharing of data, software, and other digital resources can lower such barriers and accelerate progress across mobile health.

Real-Time Psychological Stress Detection According to ECG Using Deep Learning

Today, excessive psychological stress has become a universal threat to humans. That stress can heavily affect work and study when a person repeatedly is exposed to high stress. If that exposure is long enough, it can even cause cardiovascular disease and cancer. Therefore, both monitoring and managing of stress is imperative to reduce the bad outcomes from excessive psychological stress. Conventional monitoring methods firstly extract the characteristics of the RR interval of an electrocardiogram (ECG) from a time domain and a frequency domain, then use machine learning models, like SVM, random forest, and decision tree, to distinguish the level of that stress. The biggest limitation of using these methods is that at least one minute of ECG data and other signals are indispensable to ensure the high accuracy of the results. This will greatly affect the real-time application of the models. To satisfy real-time detection of stress with high accuracy, we proposed a framework based on deep learning technology. The proposed monitoring framework is based on convolutional neural networks (CNN) and bidirectional long short-term memory (BiLSTM). To evaluate the performance of this network, we conducted the experiments applying conventional methods. The data for the 34 subjects were collected on the server platform created by the group at the Institute of Psychology of the Chinese Academy of Sciences and our group. The accuracy of the proposed framework was up to 0.865 on three levels of stress using a 10 s ECG signal, a 0.228 improvement compared with conventional methods. Therefore, our proposed framework is more suitable for real-time applications

When Do Drivers Interact with In-Vehicle Well-being Interventions? An Exploratory Analysis of a Longitudinal Study on Public Roads

Recent developments of novel in-vehicle interventions show the potential to transform the otherwise routine and mundane task of commuting into opportunities to improve the drivers' health and well-being. Prior research has explored the effectiveness of various in-vehicle interventions and has identified moments in which drivers could be interruptible to interventions. All the previous studies, however, were conducted in either simulated or constrained real-world driving scenarios on a pre-determined route. In this paper, we take a step forward and evaluate when drivers interact with in-vehicle interventions in unconstrained free-living conditions. To this end, we conducted a two-month longitudinal study with 10 participants, in which each participant was provided with a study car for their daily driving needs. We delivered two in-vehicle interventions - each aimed at improving affective well-being - and simultaneously recorded the participants' driving behavior. In our analysis, we found that several pre-trip characteristics (like trip length, traffic flow, and vehicle occupancy) and the pre-trip affective state of the participants had significant associations with whether the participants started an intervention or canceled a started intervention. Next, we found that several in-the-moment driving characteristics (like current road type, past average speed, and future brake behavior) showed significant associations with drivers' responsiveness to the intervention. Further, we identified several driving behaviors that "negated" the effectiveness of interventions and highlight the potential of using such "negative" driving characteristics to better inform intervention delivery. Finally, we compared trips with and without intervention and found that both interventions employed in our study did not have a negative effect on driving behavior. Based on our analyses, we provide solid recommendations on how to deliver interventions to maximize responsiveness and effectiveness and minimize the burden on the drivers.

Conditional Entropy: A Potential Digital Marker for Stress

Recent decades have witnessed a substantial progress in the utilization of brain activity for the identification of stress digital markers. In particular, the success of entropic measures for this purpose is very appealing, considering (1) their suitability for capturing both linear and non-linear characteristics of brain activity recordings and (2) their direct association with the brain signal variability. These findings rely on external stimuli to induce the brain stress response. On the other hand, research suggests that the use of different types of experimentally induced psychological and physical stressors could potentially yield differential impacts on the brain response to stress and therefore should be dissociated from more general patterns. The present study takes a step toward addressing this issue by introducing conditional entropy (CE) as a potential electroencephalography (EEG)-based resting-state digital marker of stress. For this purpose, we use the resting-state multi-channel EEG recordings of 20 individuals whose responses to stress-related questionnaires show significantly higher and lower level of stress. Through the application of representational similarity analysis (RSA) and K-nearest-neighbor (KNN) classification, we verify the potential that the use of CE can offer to the solution concept of finding an effective digital marker for stress.

Smoking Cessation Using Wearable Sensors: Protocol for a Microrandomized Trial

BACKGROUND

Cigarette smoking has numerous health consequences and is the leading cause of morbidity and mortality in the United States. Mindfulness has the ability to enhance resilience to stressors and can strengthen an individual's ability to deal with discomfort, which may be particularly useful when managing withdrawal and craving to smoke.

OBJECTIVE

This study aims to evaluate feasibility results from an intervention that provides real-time, real-world mindfulness strategies to a sample of racially and ethnically diverse smokers making a quit attempt.

METHODS

This study uses a microrandomized trial design to deliver mindfulness-based strategies in real time to individuals attempting to quit smoking. Data will be collected via wearable sensors, a study smartphone, and questionnaires filled out during the in-person study visits.

RESULTS

Recruitment is complete, and data management is ongoing.

CONCLUSIONS

The data collected during this feasibility trial will provide preliminary findings about whether mindfulness strategies delivered in real time are a useful quit smoking aid that warrants additional investigation.

TRIAL REGISTRATION

Clinicaltrials.gov NCT03404596; https://clinicaltrials.gov/ct2/show/NCT03404596.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/22877.

The Validity of MotionSense HRV in Estimating Sedentary Behavior and Physical Activity under Free-Living and Simulated Activity Settings

MotionSense HRV is a wrist-worn accelerometery-based sensor that is paired with a smartphone and is thus capable of measuring the intensity, duration, and frequency of physical activity (PA). However, little information is available on the validity of the MotionSense HRV. Therefore, the purpose of this study was to assess the concurrent validity of the MotionSense HRV in estimating sedentary behavior (SED) and PA. A total of 20 healthy adults (age: 32.5 ± 15.1 years) wore the MotionSense HRV and ActiGraph GT9X accelerometer (GT9X) on their non-dominant wrist for seven consecutive days during free-living conditions. Raw acceleration data from the devices were summarized into average time (min/day) spent in SED and moderate-to-vigorous PA (MVPA). Additionally, using the Cosemed K5 indirect calorimetry system (K5) as a criterion measure, the validity of the MotionSense HRV was examined in simulated free-living conditions. Pearson correlations, mean absolute percent errors (MAPE), Bland–Altman (BA) plots, and equivalence tests were used to examine the validity of the MotionSense HRV against criterion measures. The correlations between the MotionSense HRV and GT9X were high and the MAPE were low for both the SED (r = 0.99, MAPE = 2.4%) and MVPA (r = 0.97, MAPE = 9.1%) estimates under free-living conditions. BA plots illustrated that there was no systematic bias between the MotionSense HRV and criterion measures. The estimates of SED and MVPA from the MotionSense HRV were significantly equivalent to those from the GT9X; the equivalence zones were set at 16.5% for SED and 29% for MVPA. The estimates of SED and PA from the MotionSense HRV were less comparable when compared with those from the K5. The MotionSense HRV yielded comparable estimates for SED and PA when compared with the GT9X accelerometer under free-living conditions. We confirmed the promising application of the MotionSense HRV for monitoring PA patterns for practical and research purposes.

Sensing everyday activity: Parent perceptions and feasibility

Mobile and wearable sensors provide a unique opportunity to capture the daily activities and interactions that shape developmental trajectories, with potential to revolutionize the study of development (de Barbaro, 2019). However, developmental research employing sensors is still in its infancy, and parents' comfort using these devices is uncertain. This exploratory report assesses parent willingness to participate in sensor studies via a nationally representative survey (N = 210) and live recruitment of a low-income, minority population for an ongoing study (N = 359). The survey allowed us to assess how protocol design influences acceptability, including various options for devices and datastream resolution, conditions of data sharing, and feedback. By contrast, our recruitment data provided insight into parents' true willingness to participate in a sensor study, with a protocol including 72 h of continuous audio, motion, and physiological data. Our results indicate that parents are relatively conservative when considering participation in sensing studies. However, nearly 41 % of surveyed parents reported that they would be at least somewhat willing to participate in studies with audio or video recordings, 26 % were willing or extremely willing, and 14 % reported being extremely willing. These results roughly paralleled our recruitment results, where 58 % of parents indicated interest, 29 % of parents scheduled to participate, and 10 % ultimately participated. Additionally, 70 % of caregivers stated their reason for not participating in the study was due to barriers unrelated to sensing while about 25 % noted barriers due to either privacy concerns or the physical sensors themselves. Parents' willingness to collect sensitive datastreams increased if data stayed within the household for individual use only, are shared anonymously with researchers, or if parents receive feedback from devices. Overall, our findings suggest that given the correct circumstances, mobile sensors are a feasible and promising tool for characterizing children's daily interactions and their role in development.

Workshop on Implementation Science and Digital Therapeutics for Behavioral Health

Digital therapeutics can overcome many of the barriers to translation of evidence-based treatment for substance use, mental health, and other behavioral health conditions. Delivered via nearly ubiquitous platforms such as the web, smartphone applications, text messaging, and videoconferencing, digital therapeutics can transcend the time and geographic boundaries of traditional clinical settings so that individuals can access care when and where they need it. There is strong empirical support for digital therapeutic approaches for behavioral health, yet implementation science with regard to scaling use of digital therapeutics for behavioral health is still in its early stages. In this paper, we summarize the proceedings of a day-long workshop, "Implementation Science and Digital Therapeutics," sponsored and hosted by the Center for Technology and Behavioral Health at Dartmouth College. The Center for Technology and Behavioral Health is an interdisciplinary P30 Center of Excellence funded by the National Institute on Drug Abuse, with the mission of promoting state-of-the-technology and state-of-the-science for the development, evaluation, and sustainable implementation of digital therapeutic approaches for substance use and related conditions. Workshop presentations were grounded in current models of implementation science. Directions and opportunities for collaborative implementation science research to promote broad adoption of digital therapeutics for behavioral health are offered.

Automated Detection of Stressful Conversations Using Wearable Physiological and Inertial Sensors

Stressful conversation is a frequently occurring stressor in our daily life. Stressors not only adversely affect our physical and mental health but also our relationships with family, friends, and coworkers. In this paper, we present a model to automatically detect stressful conversations using wearable physiological and inertial sensors. We conducted a lab and a field study with cohabiting couples to collect ecologically valid sensor data with temporally-precise labels of stressors. We introduce the concept of stress cycles, i.e., the physiological arousal and recovery, within a stress event. We identify several novel features from stress cycles and show that they exhibit distinguishing patterns during stressful conversations when compared to physiological response due to other stressors. We observe that hand gestures also show a distinct pattern when stress occurs due to stressful conversations. We train and test our model using field data collected from 38 participants. Our model can determine whether a detected stress event is due to a stressful conversation with an F1-score of 0.83, using features obtained from only one stress cycle, facilitating intervention delivery within 3.9 minutes since the start of a stressful conversation.

PASS: A Multimodal Database of Physical Activity and Stress for Mobile Passive Body/ Brain-Computer Interface Research

With the burgeoning of wearable devices and passive body/brain-computer interfaces (B/BCIs), automated stress monitoring in everyday settings has gained significant attention recently, with applications ranging from serious games to clinical monitoring. With mobile users, however, challenges arise due to other overlapping (and potentially confounding) physiological responses (e.g., due to physical activity) that may mask the effects of stress, as well as movement artifacts that can be introduced in the measured signals. For example, the classical increase in heart rate can no longer be attributed solely to stress and could be caused by the activity itself. This makes the development of mobile passive B/BCIs challenging. In this paper, we introduce PASS, a multimodal database of Physical Activity and StresS collected from 48 participants. Participants performed tasks of varying stress levels at three different activity levels and provided quantitative ratings of their perceived stress and fatigue levels. To manipulate stress, two video games (i.e., a calm exploration game and a survival game) were used. Peripheral physical activity (electrocardiography, electrodermal activity, breathing, skin temperature) as well as cerebral activity (electroencephalography) were measured throughout the experiment. A complete description of the experimental protocol is provided and preliminary analyses are performed to investigate the physiological reactions to stress in the presence of physical activity. The PASS database, including raw data and subjective ratings has been made available to the research community at http://musaelab.ca/pass-database/. It is hoped that this database will help advance mobile passive B/BCIs for use in everyday settings.

A Blockchain-Based Consent Platform for Active Assisted Living: Modeling Study and Conceptual Framework

BACKGROUND

Recent advancements in active assisted living (AAL) technologies allow older adults to age well in place. However, sensing technologies increase the complexity of data collection points, making it difficult for users to consent to data collection. One possible solution for improving transparency in the consent management process is the use of blockchain, an immutable and timestamped ledger.

OBJECTIVE

This study aims to provide a conceptual framework based on technology aimed at mitigating trust issues in the consent management process.

METHODS

The consent management process was modeled using established methodologies to obtain a mapping of trust issues. This mapping was then used to develop a conceptual framework based on previous monitoring and surveillance architectures for connected devices.

RESULTS

In this paper, we present a model that maps trust issues in the informed consent process; a conceptual framework capable of providing all the necessary underlining technologies, components, and functionalities required to develop applications capable of managing the process of informed consent for AAL, powered by blockchain technology to ensure transparency; and a diagram showing an instantiation of the framework with entities comprising the participants in the blockchain network, suggesting possible technologies that can be used.

CONCLUSIONS

Our conceptual framework provides all the components and technologies that are required to enhance the informed consent process. Blockchain technology can help overcome several privacy challenges and mitigate trust issues that are currently present in the consent management process of data collection involving AAL technologies.

Evaluating the Reproducibility of Physiological Stress Detection Models

Recent advances in wearable sensor technologies have led to a variety of approaches for detecting physiological stress. Even with over a decade of research in the domain, there still exist many significant challenges, including a near-total lack of reproducibility across studies. Researchers often use some physiological sensors (custom-made or off-the-shelf), conduct a study to collect data, and build machine-learning models to detect stress. There is little effort to test the applicability of the model with similar physiological data collected from different devices, or the efficacy of the model on data collected from different studies, populations, or demographics. This paper takes the first step towards testing reproducibility and validity of methods and machine-learning models for stress detection. To this end, we analyzed data from 90 participants, from four independent controlled studies, using two different types of sensors, with different study protocols and research goals. We started by evaluating the performance of models built using data from one study and tested on data from other studies. Next, we evaluated new methods to improve the performance of stress-detection models and found that our methods led to a consistent increase in performance across all studies, irrespective of the device type, sensor type, or the type of stressor. Finally, we developed and evaluated a clustering approach to determine the stressed/not-stressed classification when applying models on data from different studies, and found that our approach performed better than selecting a threshold based on training data. This paper's thorough exploration of reproducibility in a controlled environment provides a critical foundation for deeper study of such methods, and is a prerequisite for tackling reproducibility in free-living conditions.

Moving the dial on prenatal stress mechanisms of neurodevelopmental vulnerability to mental health problems: A personalized prevention proof of concept

Prenatal stress exposure increases vulnerability to virtually all forms of psychopathology. Based on this robust evidence base, we propose a "Mental Health, Earlier" paradigm shift for prenatal stress research, which moves from the documentation of stress-related outcomes to their prevention, with a focus on infant neurodevelopmental indicators of vulnerability to subsequent mental health problems. Achieving this requires an expansive team science approach. As an exemplar, we introduce the Promoting Healthy Brain Project (PHBP), a randomized trial testing the impact of the Wellness-4-2 personalized prenatal stress-reduction intervention on stress-related alterations in infant neurodevelopmental trajectories in the first year of life. Wellness-4-2 utilizes bio-integrated stress monitoring for just-in-time adaptive intervention. We highlight unique challenges and opportunities this novel team science approach presents in synergizing expertise across predictive analytics, bioengineering, health information technology, prevention science, maternal-fetal medicine, neonatology, pediatrics, and neurodevelopmental science. We discuss how innovations across many areas of study facilitate this personalized preventive approach, using developmentally sensitive brain and behavioral methods to investigate whether altering children's adverse gestational exposures, i.e., maternal stress in the womb, can improve their mental health outlooks. In so doing, we seek to propel developmental SEED research towards preventive applications with the potential to reduce the pernicious effect of prenatal stress on neurodevelopment, mental health, and wellbeing.

Can We Ditch Feature Engineering? End-to-End Deep Learning for Affect Recognition from Physiological Sensor Data

To further extend the applicability of wearable sensors in various domains such as mobile health systems and the automotive industry, new methods for accurately extracting subtle physiological information from these wearable sensors are required. However, the extraction of valuable information from physiological signals is still challenging—smartphones can count steps and compute heart rate, but they cannot recognize emotions and related affective states. This study analyzes the possibility of using end-to-end multimodal deep learning (DL) methods for affect recognition. Ten end-to-end DL architectures are compared on four different datasets with diverse raw physiological signals used for affect recognition, including emotional and stress states. The DL architectures specialized for time-series classification were enhanced to simultaneously facilitate learning from multiple sensors, each having their own sampling frequency. To enable fair comparison among the different DL architectures, Bayesian optimization was used for hyperparameter tuning. The experimental results showed that the performance of the models depends on the intensity of the physiological response induced by the affective stimuli, i.e., the DL models recognize stress induced by the Trier Social Stress Test more successfully than they recognize emotional changes induced by watching affective content, e.g., funny videos. Additionally, the results showed that the CNN-based architectures might be more suitable than LSTM-based architectures for affect recognition from physiological sensors.