Using Machine Learning to Derive Just-In-Time and Personalized Predictors of Stress: Observational Study Bridging the Gap Between Nomothetic and Ideographic Approaches

Temporal prediction of suicidal ideation in an ecological momentary assessment study with recurrent neural networks

INTRODUCTION

Ecological Momentary Assessment (EMA) holds promise for providing insights into daily life experiences when studying mental health phenomena. However, commonly used mixed-effects linear statistical models do not fully utilize the richness of the ultidimensional time-varying data that EMA yields. Recurrent Neural Networks (RNNs) provide an alternative data analytic method to leverage more information and potentially improve prediction, particularly for non-normally distributed outcomes.

METHODS

As part of a broader research study of suicidal thoughts and behavior in people with borderline personality disorder (BPD), eighty-four participants engaged in EMA data collection over one week, answering questions multiple times each day about suicidal ideation (SI), stressful events, coping strategy use, and affect. RNNs and mixed-effects linear regression models (MEMs) were trained and used to predict SI. Root mean squared error (RMSE), mean absolute percent error (MAPE), and a pseudo-R2 accuracy metric were used to compare SI prediction accuracy between the two modeling methods.

RESULTS

RNNs had superior accuracy metrics (full model: RMSE = 3.41, MAPE = 42 %, pseudo-R2 = 26 %) compared with MEMs (full model: RMSE = 3.84, MAPE = 56 %, pseudo-R2 = 16 %). Importantly, RNNs showed significantly more accurate prediction at higher values of SI. Additionally, RNNs predicted, with significantly higher accuracy, the SI scores of participants with depression diagnoses and of participants with higher depression scores at baseline.

CONCLUSION

In this EMA study with a moderately sized sample, RNNs were better able to learn and predict daily SI compared with mixed-effects models. RNNs should be considered as an option for EMA analysis.

Predicting daily cognition and lifestyle behaviors for older adults using smart home data and ecological momentary assessment

OBJECTIVE

Extraction of digital markers from passive sensors placed in homes is a promising method for understanding real-world behaviors. In this study, machine learning (ML) and multilevel modeling (MLM) are used to examine types of digital markers and whether smart home sensors can predict cognitive functioning, lifestyle behaviors, and contextual factors measured through ecological momentary assessment (EMA).

METHOD

Smart home sensors were installed in the homes of 44 community-dwelling midlife and older adults for 3-4 months. Sensor data were categorized into eight digital markers. Participants responded to iPad-delivered EMA prompts 4×/day for 2 wk. Prompts included an n-back task and survey on recent (past 2 h) lifestyle and contextual factors.

RESULTS

ML marker rankings revealed that sensor counts (indicating increased activity) and time outside the home were among the most influential markers for all survey questions. Additionally, MLM revealed for every 1000 sensor counts, mental sharpness, social, physical, and cognitive EMA responses increased by 0.134-0.155 points on a 5-point scale. For every additional 30-minutes spent outside home, social, physical, and cognitive EMA responses increased by 0.596, 0.472, and 0.157 points. Advanced ML joint classification/regression significantly predicted EMA responses from smart home digital markers with error of 0.370 on a 5-point scale, and n-back performance with a normalized error of 0.040.

CONCLUSION

Results from ML and MLM were complimentary and comparable, suggesting that machine learning may be used to develop generalized models to predict everyday cognition and track lifestyle behaviors and contextual factors that impact health outcomes using smart home sensor data.

Smartphone sensor data estimate alcohol craving in a cohort of patients with alcohol-associated liver disease and alcohol use disorder

BACKGROUND

Sensors within smartphones, such as accelerometer and location, can describe longitudinal markers of behavior as represented through devices in a method called digital phenotyping. This study aimed to assess the feasibility of digital phenotyping for patients with alcohol-associated liver disease and alcohol use disorder, determine correlations between smartphone data and alcohol craving, and establish power assessment for future studies to prognosticate clinical outcomes.

METHODS

A total of 24 individuals with alcohol-associated liver disease and alcohol use disorder were instructed to download the AWARE application to collect continuous sensor data and complete daily ecological momentary assessments on alcohol craving and mood for up to 30 days. Data from sensor streams were processed into features like accelerometer magnitude, number of calls, and location entropy, which were used for statistical analysis. We used repeated measures correlation for longitudinal data to evaluate associations between sensors and ecological momentary assessments and standard Pearson correlation to evaluate within-individual relationships between sensors and craving.

RESULTS

Alcohol craving significantly correlated with mood obtained from ecological momentary assessments. Across all sensors, features associated with craving were also significantly correlated with all moods (eg, loneliness and stress) except boredom. Individual-level analysis revealed significant relationships between craving and features of location entropy and average accelerometer magnitude.

CONCLUSIONS

Smartphone sensors may serve as markers for alcohol craving and mood in alcohol-associated liver disease and alcohol use disorder. Findings suggest that location-based and accelerometer-based features may be associated with alcohol craving. However, data missingness and low participant retention remain challenges. Future studies are needed for further digital phenotyping of relapse risk and progression of liver disease.

An interpretable machine learning approach to multimodal stress detection in a simulated office environment

BACKGROUND AND OBJECTIVE

Work-related stress affects a large part of today's workforce and is known to have detrimental effects on physical and mental health. Continuous and unobtrusive stress detection may help prevent and reduce stress by providing personalised feedback and allowing for the development of just-in-time adaptive health interventions for stress management. Previous studies on stress detection in work environments have often struggled to adequately reflect real-world conditions in controlled laboratory experiments. To close this gap, in this paper, we present a machine learning methodology for stress detection based on multimodal data collected from unobtrusive sources in an experiment simulating a realistic group office environment (N=90).

METHODS

We derive mouse, keyboard and heart rate variability features to detect three levels of perceived stress, valence and arousal with support vector machines, random forests and gradient boosting models using 10-fold cross-validation. We interpret the contributions of features to the model predictions with SHapley Additive exPlanations (SHAP) value plots.

RESULTS

The gradient boosting models based on mouse and keyboard features obtained the highest average F1 scores of 0.625, 0.631 and 0.775 for the multiclass prediction of perceived stress, arousal and valence, respectively. Our results indicate that the combination of mouse and keyboard features may be better suited to detect stress in office environments than heart rate variability, despite physiological signal-based stress detection being more established in theory and research. The analysis of SHAP value plots shows that specific mouse movement and typing behaviours may characterise different levels of stress.

CONCLUSIONS

Our study fills different methodological gaps in the research on the automated detection of stress in office environments, such as approximating real-life conditions in a laboratory and combining physiological and behavioural data sources. Implications for field studies on personalised, interpretable ML-based systems for the real-time detection of stress in real office environments are also discussed.

Artificial intelligence and machine learning in mobile apps for mental health: A scoping review

Mental health conditions can have significant negative impacts on wellbeing and healthcare systems. Despite their high prevalence worldwide, there is still insufficient recognition and accessible treatments. Many mobile apps are available to the general population that aim to support mental health needs; however, there is limited evidence of their effectiveness. Mobile apps for mental health are beginning to incorporate artificial intelligence and there is a need for an overview of the state of the literature on these apps. The purpose of this scoping review is to provide an overview of the current research landscape and knowledge gaps regarding the use of artificial intelligence in mobile health apps for mental health. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) and Population, Intervention, Comparator, Outcome, and Study types (PICOS) frameworks were used to structure the review and the search. PubMed was systematically searched for randomised controlled trials and cohort studies published in English since 2014 that evaluate artificial intelligence- or machine learning-enabled mobile apps for mental health support. Two reviewers collaboratively screened references (MMI and EM), selected studies for inclusion based on the eligibility criteria and extracted the data (MMI and CL), which were synthesised in a descriptive analysis. 1,022 studies were identified in the initial search and 4 were included in the final review. The mobile apps investigated incorporated different artificial intelligence and machine learning techniques for a variety of purposes (risk prediction, classification, and personalisation) and aimed to address a wide range of mental health needs (depression, stress, and suicide risk). The studies' characteristics also varied in terms of methods, sample size, and study duration. Overall, the studies demonstrated the feasibility of using artificial intelligence to support mental health apps, but the early stages of the research and weaknesses in the study designs highlight the need for more research into artificial intelligence- and machine learning-enabled mental health apps and stronger evidence of their effectiveness. This research is essential and urgent, considering the easy availability of these apps to a large population.

Predicting the Next-Day Perceived and Physiological Stress of Pregnant Women by Using Machine Learning and Explainability: Algorithm Development and Validation

Background

Cognitive behavioral therapy–based interventions are effective in reducing prenatal stress, which can have severe adverse health effects on mothers and newborns if unaddressed. Predicting next-day physiological or perceived stress can help to inform and enable pre-emptive interventions for a likely physiologically and perceptibly stressful day. Machine learning models are useful tools that can be developed to predict next-day physiological and perceived stress by using data collected from the previous day. Such models can improve our understanding of the specific factors that predict physiological and perceived stress and allow researchers to develop systems that collect selected features for assessment in clinical trials to minimize the burden of data collection.

Objective

The aim of this study was to build and evaluate a machine-learned model that predicts next-day physiological and perceived stress by using sensor-based, ecological momentary assessment (EMA)–based, and intervention-based features and to explain the prediction results.

Methods

We enrolled pregnant women into a prospective proof-of-concept study and collected electrocardiography, EMA, and cognitive behavioral therapy intervention data over 12 weeks. We used the data to train and evaluate 6 machine learning models to predict next-day physiological and perceived stress. After selecting the best performing model, Shapley Additive Explanations were used to identify the feature importance and explainability of each feature.

Results

A total of 16 pregnant women enrolled in the study. Overall, 4157.18 hours of data were collected, and participants answered 2838 EMAs. After applying feature selection, 8 and 10 features were found to positively predict next-day physiological and perceived stress, respectively. A random forest classifier performed the best in predicting next-day physiological stress (F1 score of 0.84) and next-day perceived stress (F1 score of 0.74) by using all features. Although any subset of sensor-based, EMA-based, or intervention-based features could reliably predict next-day physiological stress, EMA-based features were necessary to predict next-day perceived stress. The analysis of explainability metrics showed that the prolonged duration of physiological stress was highly predictive of next-day physiological stress and that physiological stress and perceived stress were temporally divergent.

Conclusions

In this study, we were able to build interpretable machine learning models to predict next-day physiological and perceived stress, and we identified unique features that were highly predictive of next-day stress that can help to reduce the burden of data collection.

Innovative methods for observing and changing complex health behaviors: four propositions

Precision health initiatives aim to progressively move from traditional, group-level approaches to health diagnostics and treatments toward ones that are individualized, contextualized, and timely. This article aims to provide an overview of key methods and approaches that can help facilitate this transition in the health behavior change domain. This article is a narrative review of the methods used to observe and change complex health behaviors. On the basis of the available literature, we argue that health behavior change researchers should progressively transition from (i) low- to high-resolution behavioral assessments, (ii) group-only to group- and individual-level statistical inference, (iii) narrative theoretical models to dynamic computational models, and (iv) static to adaptive and continuous tuning interventions. Rather than providing an exhaustive and technical presentation of each method and approach, this article articulates why and how researchers interested in health behavior change can apply these innovative methods. Practical examples contributing to these efforts are presented. If successfully adopted and implemented, the four propositions in this article have the potential to greatly improve our public health and behavior change practices in the near future.

Single-Subject Research in Psychiatry: Facts and Fictions

Scientific evidence in the field of psychiatry is mainly derived from group-based ("nomothetic") studies that yield group-aggregated results, while often the need is to answer questions that apply to individuals. Particularly in the presence of great inter-individual differences and temporal complexities, information at the individual-person level may be valuable for personalized treatment decisions, individual predictions and diagnostics. The single-subject study design can be used to make inferences about individual persons. Yet, the single-subject study is not often used in the field of psychiatry. We believe that this is because of a lack of awareness of its value rather than a lack of usefulness or feasibility. In the present paper, we aimed to resolve some common misconceptions and beliefs about single-subject studies by discussing some commonly heard "facts and fictions." We also discuss some situations in which the single-subject study is more or less appropriate, and the potential of combining single-subject and group-based study designs into one study. While not intending to plea for single-subject studies at the expense of group-based studies, we hope to increase awareness of the value of single-subject research by informing the reader about several aspects of this design, resolving misunderstanding, and providing references for further reading.

Efficacy of a Smartphone App Intervention for Reducing Caregiver Stress: Randomized Controlled Trial

BACKGROUND

Caregivers play a pivotal role in maintaining an economically viable health care system, yet they are characterized by low levels of psychological well-being and consistently report unmet needs for psychological support. Mobile app-based (mobile health [mHealth]) interventions present a novel approach to both reducing stress and improving well-being.

OBJECTIVE

This study aims to evaluate the effectiveness of a self-guided mobile app-based psychological intervention for people providing care to family or friends with a physical or mental disability.

METHODS

In a randomized, single-blind, controlled trial, 183 caregivers recruited through the web were randomly allocated to either an intervention (n=73) or active control (n=110) condition. The intervention app contained treatment modules combining daily self-monitoring with third-wave (mindfulness-based) cognitive-behavioral therapies, whereas the active control app contained only self-monitoring features. Both programs were completed over a 5-week period. It was hypothesized that intervention app exposure would be associated with decreases in depression, anxiety, and stress, and increases in well-being, self-esteem, optimism, primary and secondary control, and social support. Outcomes were assessed at baseline, postintervention, and 3-4 months postintervention. App quality was also assessed.

RESULTS

In total, 25% (18/73) of the intervention participants were lost to follow-up at 3 months, and 30.9% (34/110) of the participants from the wait-list control group dropped out before the postintervention survey. The intervention group experienced reductions in stress (b=-2.07; P=.04) and depressive symptoms (b=-1.36; P=.05) from baseline to postintervention. These changes were further enhanced from postintervention to follow-up, with the intervention group continuing to report lower levels of depression (b=-1.82; P=.03) and higher levels of emotional well-being (b=6.13; P<.001), optimism (b=0.78; P=.007), self-esteem (b=-0.84; P=.005), support from family (b=2.15; P=.001), support from significant others (b=2.66; P<.001), and subjective well-being (b=4.82; P<.001). On average, participants completed 2.5 (SD 1.05) out of 5 treatment modules. The overall quality of the app was also rated highly, with a mean score of 3.94 out of a maximum score of 5 (SD 0.58).

CONCLUSIONS

This study demonstrates that mHealth psychological interventions are an effective treatment option for caregivers experiencing high levels of stress. Recommendations for improving mHealth interventions for caregivers include offering flexibility and customization in the treatment design.

TRIAL REGISTRATION

Australian New Zealand Clinical Trial Registry ACTRN12616000996460; https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=371170.

Prediction of stress and drug craving ninety minutes in the future with passively collected GPS data

Just-in-time adaptive interventions (JITAIs), typically smartphone apps, learn to deliver therapeutic content when users need it. The challenge is to "push" content at algorithmically chosen moments without making users trigger it with effortful input. We trained a randomForest algorithm to predict heroin craving, cocaine craving, or stress (reported via smartphone app 3x/day) 90 min into the future, using 16 weeks of field data from 189 outpatients being treated for opioid-use disorder. We used only one form of continuous input (along with person-level demographic data), collected passively: an indicator of environmental exposures along the past 5 h of movement, as assessed by GPS. Our models achieved excellent overall accuracy-as high as 0.93 by the end of 16 weeks of tailoring-but this was driven mostly by correct predictions of absence. For predictions of presence, "believability" (positive predictive value, PPV) usually peaked in the high 0.70s toward the end of the 16 weeks. When the prediction target was more rare, PPV was lower. Our findings complement those of other investigators who use machine learning with more broadly based "digital phenotyping" inputs to predict or detect mental and behavioral events. When target events are comparatively subtle, like stress or drug craving, accurate detection or prediction probably needs effortful input from users, not passive monitoring alone. We discuss ways in which accuracy is difficult to achieve or even assess, and warn that high overall accuracy (including high specificity) can mask the abundance of false alarms that low PPV reveals.

Why we need a small data paradigm

BACKGROUND

There is great interest in and excitement about the concept of personalized or precision medicine and, in particular, advancing this vision via various 'big data' efforts. While these methods are necessary, they are insufficient to achieve the full personalized medicine promise. A rigorous, complementary 'small data' paradigm that can function both autonomously from and in collaboration with big data is also needed. By 'small data' we build on Estrin's formulation and refer to the rigorous use of data by and for a specific N-of-1 unit (i.e., a single person, clinic, hospital, healthcare system, community, city, etc.) to facilitate improved individual-level description, prediction and, ultimately, control for that specific unit.

MAIN BODY

The purpose of this piece is to articulate why a small data paradigm is needed and is valuable in itself, and to provide initial directions for future work that can advance study designs and data analytic techniques for a small data approach to precision health. Scientifically, the central value of a small data approach is that it can uniquely manage complex, dynamic, multi-causal, idiosyncratically manifesting phenomena, such as chronic diseases, in comparison to big data. Beyond this, a small data approach better aligns the goals of science and practice, which can result in more rapid agile learning with less data. There is also, feasibly, a unique pathway towards transportable knowledge from a small data approach, which is complementary to a big data approach. Future work should (1) further refine appropriate methods for a small data approach; (2) advance strategies for better integrating a small data approach into real-world practices; and (3) advance ways of actively integrating the strengths and limitations from both small and big data approaches into a unified scientific knowledge base that is linked via a robust science of causality.

CONCLUSION

Small data is valuable in its own right. That said, small and big data paradigms can and should be combined via a foundational science of causality. With these approaches combined, the vision of precision health can be achieved.