Reproducible Analysis Pipeline for Data Streams: Open-Source Software to Process Data Collected With Mobile Devices

Framework for Ranking Machine Learning Predictions of Limited, Multimodal, and Longitudinal Behavioral Passive Sensing Data: Combining User-Agnostic and Personalized Modeling

BACKGROUND

Passive mobile sensing provides opportunities for measuring and monitoring health status in the wild and outside of clinics. However, longitudinal, multimodal mobile sensor data can be small, noisy, and incomplete. This makes processing, modeling, and prediction of these data challenging. The small size of the data set restricts it from being modeled using complex deep learning networks. The current state of the art (SOTA) tackles small sensor data sets following a singular modeling paradigm based on traditional machine learning (ML) algorithms. These opt for either a user-agnostic modeling approach, making the model susceptible to a larger degree of noise, or a personalized approach, where training on individual data alludes to a more limited data set, giving rise to overfitting, therefore, ultimately, having to seek a trade-off by choosing 1 of the 2 modeling approaches to reach predictions.

OBJECTIVE

The objective of this study was to filter, rank, and output the best predictions for small, multimodal, longitudinal sensor data using a framework that is designed to tackle data sets that are limited in size (particularly targeting health studies that use passive multimodal sensors) and that combines both user agnostic and personalized approaches, along with a combination of ranking strategies to filter predictions.

METHODS

In this paper, we introduced a novel ranking framework for longitudinal multimodal sensors (FLMS) to address challenges encountered in health studies involving passive multimodal sensors. Using the FLMS, we (1) built a tensor-based aggregation and ranking strategy for final interpretation, (2) processed various combinations of sensor fusions, and (3) balanced user-agnostic and personalized modeling approaches with appropriate cross-validation strategies. The performance of the FLMS was validated with the help of a real data set of adolescents diagnosed with major depressive disorder for the prediction of change in depression in the adolescent participants.

RESULTS

Predictions output by the proposed FLMS achieved a 7% increase in accuracy and a 13% increase in recall for the real data set. Experiments with existing SOTA ML algorithms showed an 11% increase in accuracy for the depression data set and how overfitting and sparsity were handled.

CONCLUSIONS

The FLMS aims to fill the gap that currently exists when modeling passive sensor data with a small number of data points. It achieves this through leveraging both user-agnostic and personalized modeling techniques in tandem with an effective ranking strategy to filter predictions.

Machine Learning for Multimodal Mental Health Detection: A Systematic Review of Passive Sensing Approaches

As mental health (MH) disorders become increasingly prevalent, their multifaceted symptoms and comorbidities with other conditions introduce complexity to diagnosis, posing a risk of underdiagnosis. While machine learning (ML) has been explored to mitigate these challenges, we hypothesized that multiple data modalities support more comprehensive detection and that non-intrusive collection approaches better capture natural behaviors. To understand the current trends, we systematically reviewed 184 studies to assess feature extraction, feature fusion, and ML methodologies applied to detect MH disorders from passively sensed multimodal data, including audio and video recordings, social media, smartphones, and wearable devices. Our findings revealed varying correlations of modality-specific features in individualized contexts, potentially influenced by demographics and personalities. We also observed the growing adoption of neural network architectures for model-level fusion and as ML algorithms, which have demonstrated promising efficacy in handling high-dimensional features while modeling within and cross-modality relationships. This work provides future researchers with a clear taxonomy of methodological approaches to multimodal detection of MH disorders to inspire future methodological advancements. The comprehensive analysis also guides and supports future researchers in making informed decisions to select an optimal data source that aligns with specific use cases based on the MH disorder of interest.

Smartphone sensor data estimate alcohol craving in a cohort of patients with alcohol-associated liver disease and alcohol use disorder

BACKGROUND

Sensors within smartphones, such as accelerometer and location, can describe longitudinal markers of behavior as represented through devices in a method called digital phenotyping. This study aimed to assess the feasibility of digital phenotyping for patients with alcohol-associated liver disease and alcohol use disorder, determine correlations between smartphone data and alcohol craving, and establish power assessment for future studies to prognosticate clinical outcomes.

METHODS

A total of 24 individuals with alcohol-associated liver disease and alcohol use disorder were instructed to download the AWARE application to collect continuous sensor data and complete daily ecological momentary assessments on alcohol craving and mood for up to 30 days. Data from sensor streams were processed into features like accelerometer magnitude, number of calls, and location entropy, which were used for statistical analysis. We used repeated measures correlation for longitudinal data to evaluate associations between sensors and ecological momentary assessments and standard Pearson correlation to evaluate within-individual relationships between sensors and craving.

RESULTS

Alcohol craving significantly correlated with mood obtained from ecological momentary assessments. Across all sensors, features associated with craving were also significantly correlated with all moods (eg, loneliness and stress) except boredom. Individual-level analysis revealed significant relationships between craving and features of location entropy and average accelerometer magnitude.

CONCLUSIONS

Smartphone sensors may serve as markers for alcohol craving and mood in alcohol-associated liver disease and alcohol use disorder. Findings suggest that location-based and accelerometer-based features may be associated with alcohol craving. However, data missingness and low participant retention remain challenges. Future studies are needed for further digital phenotyping of relapse risk and progression of liver disease.

Exploratory Machine Learning Modeling of Adaptive and Maladaptive Personality Traits from Passively Sensed Behavior

Continuous passive sensing of daily behavior from mobile devices has the potential to identify behavioral patterns associated with different aspects of human characteristics. This paper presents novel analytic approaches to extract and understand these behavioral patterns and their impact on predicting adaptive and maladaptive personality traits. Our machine learning analysis extends previous research by showing that both adaptive and maladaptive traits are associated with passively sensed behavior providing initial evidence for the utility of this type of data to study personality and its pathology. The analysis also suggests directions for future confirmatory studies into the underlying behavior patterns that link adaptive and maladaptive variants consistent with contemporary models of personality pathology.

Detecting Mental Health Behaviors Using Mobile Interactions: Exploratory Study Focusing on Binge Eating

BACKGROUND

Binge eating is a subjective loss of control while eating, which leads to the consumption of large amounts of food. It can cause significant emotional distress and is often accompanied by purging behaviors (eg, meal skipping, overexercising, or vomiting).

OBJECTIVE

The aim of this study was to explore the potential of mobile sensing to detect indicators of binge-eating episodes, with a view toward informing the design of future context-aware mobile interventions.

METHODS

This study was conducted in 2 stages. The first involved the development of the DeMMI (Detecting Mental health behaviors using Mobile Interactions) app. As part of this, we conducted a consultation session to explore whether the types of sensor data we were proposing to capture were useful and appropriate, as well as to gather feedback on some specific app features relating to self-reporting. The second stage involved conducting a 6-week period of data collection with 10 participants experiencing binge eating (logging both their mood and episodes of binge eating) and 10 comparison participants (logging only mood). An optional interview was conducted after the study, which discussed their experience using the app, and 8 participants (n=3, 38% binge eating and n=5, 63% comparisons) consented.

RESULTS

The findings showed unique differences in the types of sensor data that were triangulated with the individuals' episodes (with nearby Bluetooth devices, screen and app use features, mobility features, and mood scores showing relevance). Participants had a largely positive opinion about the app, its unobtrusive role, and its ease of use. Interacting with the app increased participants' awareness of and reflection on their mood and phone usage patterns. Moreover, they expressed no privacy concerns as these were alleviated by the study information sheet.

CONCLUSIONS

This study contributes a series of recommendations for future studies wishing to scale our approach and for the design of bespoke mobile interventions to support this population.