1
|
King ZD, Yu H, Vaessen T, Myin-Germeys I, Sano A. Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study. JMIR Mhealth Uhealth 2024; 12:e46347. [PMID: 38324358 PMCID: PMC10882474 DOI: 10.2196/46347] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2023] [Revised: 11/01/2023] [Accepted: 11/21/2023] [Indexed: 02/08/2024] Open
Abstract
BACKGROUND As mobile health (mHealth) studies become increasingly productive owing to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. Many health constructs are dependent on subjective responses, and without such responses, researchers are left with little to no ground truth to accompany our ever-growing biobehavioral data. This issue can significantly impact the quality of a study, particularly for populations known to exhibit lower compliance rates. To address this challenge, researchers have proposed innovative approaches that use machine learning (ML) and sensor data to modify the timing and delivery of surveys. However, an overarching concern is the potential introduction of biases or unintended influences on participants' responses when implementing new survey delivery methods. OBJECTIVE This study aims to demonstrate the potential impact of an ML-based ecological momentary assessment (EMA) delivery system (using receptivity as the predictor variable) on the participants' reported emotional state. We examine the factors that affect participants' receptivity to EMAs in a 10-day wearable and EMA-based emotional state-sensing mHealth study. We study the physiological relationships indicative of receptivity and affect while also analyzing the interaction between the 2 constructs. METHODS We collected data from 45 healthy participants wearing 2 devices measuring electrodermal activity, accelerometer, electrocardiography, and skin temperature while answering 10 EMAs daily, containing questions about perceived mood. Owing to the nature of our constructs, we can only obtain ground truth measures for both affect and receptivity during responses. Therefore, we used unsupervised and supervised ML methods to infer affect when a participant did not respond. Our unsupervised method used k-means clustering to determine the relationship between physiology and receptivity and then inferred the emotional state during nonresponses. For the supervised learning method, we primarily used random forest and neural networks to predict the affect of unlabeled data points as well as receptivity. RESULTS Our findings showed that using a receptivity model to trigger EMAs decreased the reported negative affect by >3 points or 0.29 SDs in our self-reported affect measure, scored between 13 and 91. The findings also showed a bimodal distribution of our predicted affect during nonresponses. This indicates that this system initiates EMAs more commonly during states of higher positive emotions. CONCLUSIONS Our results showed a clear relationship between affect and receptivity. This relationship can affect the efficacy of an mHealth study, particularly those that use an ML algorithm to trigger EMAs. Therefore, we propose that future work should focus on a smart trigger that promotes EMA receptivity without influencing affect during sampled time points.
Collapse
Affiliation(s)
- Zachary D King
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, United States
| | - Han Yu
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, United States
| | - Thomas Vaessen
- Center For Contextual Psychiatry, Katholieke Universiteit Leuven, Leuven, Belgium
- Department of Psychology, Health & Technology, University of Twente, Enschede, Netherlands
| | - Inez Myin-Germeys
- Center For Contextual Psychiatry, Katholieke Universiteit Leuven, Leuven, Belgium
| | - Akane Sano
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, United States
| |
Collapse
|
2
|
Robles-Granda P, Lin S, Wu X, Martinez GJ, Mattingly SM, Moskal E, Striegel A, Chawla NV, D'Mello S, Gregg J, Nies K, Mark G, Grover T, Campbell AT, Mirjafari S, Saha K, De Choudhury M, Dey AK. Jointly Predicting Job Performance, Personality, Cognitive Ability, Affect, and Well-Being. IEEE COMPUT INTELL M 2021. [DOI: 10.1109/mci.2021.3061877] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
3
|
Hamaker EL, Mulder JD, van IJzendoorn MH. Description, prediction and causation: Methodological challenges of studying child and adolescent development. Dev Cogn Neurosci 2020; 46:100867. [PMID: 33186867 PMCID: PMC7670214 DOI: 10.1016/j.dcn.2020.100867] [Citation(s) in RCA: 20] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2020] [Revised: 09/24/2020] [Accepted: 09/28/2020] [Indexed: 12/14/2022] Open
Abstract
Scientific research can be categorized into: a) descriptive research, with the main goal to summarize characteristics of a group (or person); b) predictive research, with the main goal to forecast future outcomes that can be used for screening, selection, or monitoring; and c) explanatory research, with the main goal to understand the underlying causal mechanism, which can then be used to develop interventions. Since each goal requires different research methods in terms of design, operationalization, model building and evaluation, it should form an important basis for decisions on how to set up and execute a study. To determine the extent to which developmental research is motivated by each goal and how this aligns with the research designs that are used, we evaluated 100 publications from the Consortium on Individual Development (CID). This analysis shows that the match between research goal and research design is not always optimal. We discuss alternative techniques, which are not yet part of the developmental scientist's standard toolbox, but that may help bridge some of the lurking gaps that developmental scientists encounter between their research design and their research goal. These include unsupervised and supervised machine learning, directed acyclical graphs, Mendelian randomization, and target trials.
Collapse
Affiliation(s)
- Ellen L Hamaker
- Methodology and Statistics, Faculty of Social and Behavioural Sciences, Utrecht University, The Netherlands.
| | - Jeroen D Mulder
- Methodology and Statistics, Faculty of Social and Behavioural Sciences, Utrecht University, The Netherlands
| | - Marinus H van IJzendoorn
- Department of Psychology, Education and Child Studies, Erasmus University Rotterdam, The Netherlands; School of Clinical Medicine, University of Cambridge, UK
| |
Collapse
|
4
|
Akhter-Khan SC, Au R. Why Loneliness Interventions Are Unsuccessful: A Call for Precision Health. ADVANCES IN GERIATRIC MEDICINE AND RESEARCH 2020; 2:e200016. [PMID: 36037052 PMCID: PMC9410567 DOI: 10.20900/agmr20200016] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 04/11/2023]
Abstract
Background Loneliness has drawn increasing attention over the past few decades due to rising recognition of its close connection with serious health issues, like dementia. Yet, researchers are failing to find solutions to alleviate the globally experienced burden of loneliness. Purpose This review aims to shed light on possible reasons for why interventions have been ineffective. We suggest new directions for research on loneliness as it relates to precision health, emerging technologies, digital phenotyping, and machine learning. Results Current loneliness interventions are unsuccessful due to (i) their inconsideration of loneliness as a heterogeneous construct and (ii) not being targeted at individuals' needs and contexts. We propose a model for how loneliness interventions can move towards finding the right solution for the right person at the right time. Taking a precision health approach, we explore how transdisciplinary research can contribute to creating a more holistic picture of loneliness and shift interventions from treatment to prevention. Conclusions We urge the field to rethink metrics to account for diverse intra-individual experiences and trajectories of loneliness. Big data sharing and evolving technologies that emphasize human connection raise hope for realizing our model of precision health applied to loneliness. There is an urgent need for precise, integrated, and theory-driven interventions that focus on individuals' needs and the subjective burden of loneliness in the ageing context.
Collapse
Affiliation(s)
- Samia C. Akhter-Khan
- Department of Psychology, Humboldt University of Berlin, 10117 Berlin, Germany
- Department of Psychology & Neuroscience, Duke University Graduate School, NC 27705, USA
| | - Rhoda Au
- Departments of Anatomy & Neurobiology and Neurology, Boston University Alzheimer’s Disease Center, Framingham Heart Study, Boston University School of Medicine, Boston, MA 02118, USA
- Department of Epidemiology, Boston University School of Public Health, Boston, MA 02118, USA
| |
Collapse
|
5
|
Epstein DH, Tyburski M, Kowalczyk WJ, Burgess-Hull AJ, Phillips KA, Curtis BL, Preston KL. Prediction of stress and drug craving ninety minutes in the future with passively collected GPS data. NPJ Digit Med 2020; 3:26. [PMID: 32195362 PMCID: PMC7055250 DOI: 10.1038/s41746-020-0234-6] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2019] [Accepted: 02/06/2020] [Indexed: 12/16/2022] Open
Abstract
Just-in-time adaptive interventions (JITAIs), typically smartphone apps, learn to deliver therapeutic content when users need it. The challenge is to "push" content at algorithmically chosen moments without making users trigger it with effortful input. We trained a randomForest algorithm to predict heroin craving, cocaine craving, or stress (reported via smartphone app 3x/day) 90 min into the future, using 16 weeks of field data from 189 outpatients being treated for opioid-use disorder. We used only one form of continuous input (along with person-level demographic data), collected passively: an indicator of environmental exposures along the past 5 h of movement, as assessed by GPS. Our models achieved excellent overall accuracy-as high as 0.93 by the end of 16 weeks of tailoring-but this was driven mostly by correct predictions of absence. For predictions of presence, "believability" (positive predictive value, PPV) usually peaked in the high 0.70s toward the end of the 16 weeks. When the prediction target was more rare, PPV was lower. Our findings complement those of other investigators who use machine learning with more broadly based "digital phenotyping" inputs to predict or detect mental and behavioral events. When target events are comparatively subtle, like stress or drug craving, accurate detection or prediction probably needs effortful input from users, not passive monitoring alone. We discuss ways in which accuracy is difficult to achieve or even assess, and warn that high overall accuracy (including high specificity) can mask the abundance of false alarms that low PPV reveals.
Collapse
Affiliation(s)
- David H. Epstein
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Blvd., Suite 200, Baltimore, MD 21224 USA
| | - Matthew Tyburski
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Blvd., Suite 200, Baltimore, MD 21224 USA
| | - William J. Kowalczyk
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Blvd., Suite 200, Baltimore, MD 21224 USA
| | - Albert J. Burgess-Hull
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Blvd., Suite 200, Baltimore, MD 21224 USA
| | - Karran A. Phillips
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Blvd., Suite 200, Baltimore, MD 21224 USA
| | - Brenda L. Curtis
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Blvd., Suite 200, Baltimore, MD 21224 USA
| | - Kenzie L. Preston
- Intramural Research Program, National Institute on Drug Abuse, 251 Bayview Blvd., Suite 200, Baltimore, MD 21224 USA
| |
Collapse
|
6
|
Osotsi A, Oravecz Z, Li Q, Smyth J, Brick TR. Individualized Modeling to Distinguish Between High and Low Arousal States Using Physiological Data. JOURNAL OF HEALTHCARE INFORMATICS RESEARCH 2020; 4:91-109. [DOI: 10.1007/s41666-019-00064-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2018] [Revised: 09/30/2019] [Accepted: 11/27/2019] [Indexed: 10/25/2022]
|
7
|
Rozet A, Kronish IM, Schwartz JE, Davidson KW. Using Machine Learning to Derive Just-In-Time and Personalized Predictors of Stress: Observational Study Bridging the Gap Between Nomothetic and Ideographic Approaches. J Med Internet Res 2019; 21:e12910. [PMID: 31025942 PMCID: PMC6658264 DOI: 10.2196/12910] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2018] [Revised: 01/24/2019] [Accepted: 03/03/2019] [Indexed: 11/13/2022] Open
Abstract
BACKGROUND Investigations into person-specific predictors of stress have typically taken either a population-level nomothetic approach or an individualized ideographic approach. Nomothetic approaches can quickly identify predictors but can be hindered by the heterogeneity of these predictors across individuals and time. Ideographic approaches may result in more predictive models at the individual level but require a longer period of data collection to identify robust predictors. OBJECTIVE Our objectives were to compare predictors of stress identified through nomothetic and ideographic models and to assess whether sequentially combining nomothetic and ideographic models could yield more accurate and actionable predictions of stress than relying on either model. At the same time, we sought to maintain the interpretability necessary to retrieve individual predictors of stress despite using nomothetic models. METHODS Data collected in a 1-year observational study of 79 participants performing low levels of exercise were used. Physical activity was continuously and objectively monitored by actigraphy. Perceived stress was recorded by participants via daily ecological momentary assessments on a mobile app. Environmental variables including daylight time, temperature, and precipitation were retrieved from the public archives. Using these environmental, actigraphy, and mobile assessment data, we built machine learning models to predict individual stress ratings using linear, decision tree, and neural network techniques employing nomothetic and ideographic approaches. The accuracy of the approaches for predicting individual stress ratings was compared based on classification errors. RESULTS Across the group of patients, an individual's recent history of stress ratings was most heavily weighted in predicting a future stress rating in the nomothetic recurrent neural network model, whereas environmental factors such as temperature and daylight, as well as duration and frequency of bouts of exercise, were more heavily weighted in the ideographic models. The nomothetic recurrent neural network model was the highest performing nomothetic model and yielded 72% accuracy for an 80%/20% train/test split. Using the same 80/20 split, the ideographic models yielded 75% accuracy. However, restricting ideographic models to participants with more than 50 valid days in the training set, with the same 80/20 split, yielded 85% accuracy. CONCLUSIONS We conclude that for some applications, nomothetic models may be useful for yielding higher initial performance while still surfacing personalized predictors of stress, before switching to ideographic models upon sufficient data collection.
Collapse
Affiliation(s)
- Alan Rozet
- Center for Behavioral Cardiovascular Health, Columbia University Irving Medical Center, New York, NY, United States
| | - Ian M Kronish
- Center for Behavioral Cardiovascular Health, Columbia University Irving Medical Center, New York, NY, United States
| | - Joseph E Schwartz
- Center for Behavioral Cardiovascular Health, Columbia University Irving Medical Center, New York, NY, United States
| | - Karina W Davidson
- Feinstein Institute for Medical Research, Northwell Health, New York, NY, United States
| |
Collapse
|
8
|
De Stefani J, Le Borgne YA, Caelen O, Hattab D, Bontempi G. Batch and incremental dynamic factor machine learning for multivariate and multi-step-ahead forecasting. INTERNATIONAL JOURNAL OF DATA SCIENCE AND ANALYTICS 2018. [DOI: 10.1007/s41060-018-0150-x] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
9
|
Why Do Workers Take Safety Risks?—A Conceptual Model for the Motivation Underpinning Perverse Agency. SAFETY 2018. [DOI: 10.3390/safety4020024] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022] Open
|
10
|
Tuarob S, Strong R, Chandra A, Tucker CS. Discovering Discontinuity in Big Financial Transaction Data. ACM TRANSACTIONS ON MANAGEMENT INFORMATION SYSTEMS 2018. [DOI: 10.1145/3159445] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
Business transactions are typically recorded in the company ledger. The primary purpose of such financial information is to accompany a monthly or quarterly report for executives to make sound business decisions and strategies for the next business period. These business strategies often result in transitions that cause underlying infrastructures and components to change, including alteration in the nomenclature system of the business components. As a result, a transaction stream of an affected component would be replaced by another stream with a different component name, resulting in discontinuity of a financial stream of the same component. Recently, advancement in large-scale data mining technologies has enabled a set of critical applications to utilize knowledge extracted from a vast amount of existing data that would otherwise have been unused or underutilized. In financial and services computing domains, recent studies have illustrated that historical financial data could be used to predict future revenues and profits, optimizing costs, among other potential applications. These prediction models rely on long-term availability of the historical data that traces back for multiple years. However, the discontinuity of the financial transaction stream associated with a business component has limited the learning capability of the prediction models. In this article, we propose a set of machine learning–based algorithms to automatically discover component name replacements, using information available in general ledger databases. The algorithms are designed to be scalable for handling massive data points, especially in large companies. Furthermore, the proposed algorithms are generalizable to other domains whose data is time series and shares the same nature as the financial data available in business ledgers. A case study of real-world IBM service delivery retrieved from four different geographical regions is used to validate the efficacy of the proposed methodology.
Collapse
Affiliation(s)
| | - Ray Strong
- IBM Almaden Research Center, San Jose, CA
| | | | | |
Collapse
|