Evidence for the Feasibility of Person-Specific Ecological Momentary Assessment Across Diverse Populations and Study Designs

Examining missingness at the momentary level in clinical research using ecological momentary assessment: Implications for suicide research

The use of intensive time sampling methods, such as ecological momentary assessment (EMA), has increased in clinical, and specifically suicide, research during the past decade. While EMA can capture dynamic intraindividual processes, repeated assessments increase participant burden, potentially resulting in low compliance. This study aimed to shed light on study-level and psychological variables, including suicidal ideation (SI), that may predict momentary prompt (i.e., prompt-to-prompt) completion. We combined data from three EMA studies examining mental health difficulties (N = 103; 10,656 prompts; 7144 completed), using multilevel models and machine learning to determine how well we can predict prompt-to-prompt completion and which variables are most important. The two most important variables in prompt-to-prompt completion were hours since the last prompt and time in study. Psychological variables added little predictive validity; similarly, trait-level SI demonstrated a small effect on prompt-to-prompt completion. Our study showed how study-level characteristics can be used to explain prompt-to-prompt compliance rates in EMA research, highlighting the potential for developing adaptive assessment schedules to improve compliance.

Smartphone-Based Survey and Message Compliance in Adults Initially Unready to Quit Smoking: Secondary Analysis of a Randomized Controlled Trial

BACKGROUND

Efficacy of smartphone-based interventions depends on intervention content quality and level of exposure to that content. Smartphone-based survey completion rates tend to decline over time; however, few studies have identified variables that predict this decline over longer-term interventions (eg, 26 weeks).

OBJECTIVE

This study aims to identify predictors of survey completion and message viewing over time within a 26-week smoking cessation trial.

METHODS

This study examined data from a 3-group pilot randomized controlled trial of adults who smoke (N=152) and were not ready to quit smoking within the next 30 days. For 182 days, two intervention groups received smartphone-based morning and evening messages based on current readiness to quit smoking. The control group received 2 daily messages unrelated to smoking. All participants were prompted to complete 26 weekly smartphone-based surveys that assessed smoking behavior, quit attempts, and readiness to quit. Compliance was operationalized as percentages of weekly surveys completed and daily messages viewed. Linear regression and mixed-effects models were used to identify predictors (eg, intervention group, age, and sex) of weekly survey completion and daily message viewing and decline in compliance over time.

RESULTS

The sample (mean age 50, SD 12.5, range 19-75 years; mean years of education 13.3, SD 1.6, range 10-20 years) was 67.8% (n=103) female, 74.3% (n=113) White, 77% (n=117) urban, and 52.6% (n=80) unemployed, and 61.2% (n=93) had mental health diagnoses. On average, participants completed 18.3 (71.8%) out of 25.5 prompted weekly surveys and viewed 207.3 (60.6%) out of 345.1 presented messages (31,503/52,460 total). Age was positively associated with overall weekly survey completion (P=.003) and daily message viewing (P=.02). Mixed-effects models indicated a decline in survey completion from 77% (114/148) in the first week of the intervention to 56% (84/150) in the last week of the intervention (P<.001), which was significantly moderated by age, sex, ethnicity, municipality (ie, rural/urban), and employment status. Similarly, message viewing declined from 72.3% (1533/2120) in the first week of the intervention to 44.6% (868/1946) in the last week of the intervention (P<.001). This decline in message viewing was significantly moderated by age, sex, municipality, employment status, and education.

CONCLUSIONS

This study demonstrated the feasibility of a 26-week smartphone-based smoking cessation intervention. Study results identified subgroups that displayed accelerated rates in the decline of survey completion and message viewing. Future research should identify ways to maintain high levels of interaction with mobile health interventions that span long intervention periods, especially among subgroups that have demonstrated declining rates of intervention engagement over time.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03405129; https://clinicaltrials.gov/ct2/show/NCT03405129.

Recruitment and retention of adolescents for an ecological momentary assessment measurement burst mental health study: The MHIM engagement strategy

INTRODUCTION

Recruitment and long-term retention of adolescent participants in longitudinal research are challenging and may be especially so in studies involving remote measurement and biosampling components. The ability to effectively recruit and retain participants can be supported by the use of specific evidence-based engagement strategies that are built in from the earliest stages.

METHODS

Informed by a review of the evidence on effective engagement strategies and consultations with adolescents (via two Young Person Advisory Groups [YPAGs]; ages 11-13 and 14-17), the current protocol describes the planned participant engagement strategy for the Mental Health in the Moment Study: a multimodal measurement burst study of adolescent mental health across ages 11-19.

RESULTS

The protocol incorporates engagement strategies in four key domains: consultations/co-design with the target population, incentives, relationship-building and burden/barrier reduction. In addition to describing general engagement strategies in longitudinal studies, we also discuss specific concerns regarding engagement in data collection methods such as biosampling and ecological momentary assessment where a paucity of evidence exists.

CONCLUSION

Engagement strategies for adolescent mental health studies should be based on existing evidence and consultations with adolescents. We present our approach in developing the planned engagement strategies and also discuss limitations and future directions in engaging adolescents in longitudinal research.

PATIENT OR PUBLIC CONTRIBUTION

The study design for this project places a strong emphasis on the active engagement of adolescents throughout its development. Specifically, the feedback and suggestions provided by the YPAGs have been instrumental in refining our strategies for maximising the recruitment and retention of participants.

Diversity, equity, and inclusivity in observational ambulatory assessment: Recommendations from two decades of Electronically Activated Recorder (EAR) research

Ambient audio sampling methods such as the Electronically Activated Recorder (EAR) have become increasingly prominent in clinical and social sciences research. These methods record snippets of naturalistically assessed audio from participants' daily lives, enabling novel observational research about the daily social interactions, identities, environments, behaviors, and speech of populations of interest. In practice, these scientific opportunities are equaled by methodological challenges: researchers' own cultural backgrounds and identities can easily and unknowingly permeate the collection, coding, analysis, and interpretation of social data from daily life. Ambient audio sampling poses unique and significant challenges to cultural humility, diversity, equity, and inclusivity (DEI) in scientific research that require systematized attention. Motivated by this observation, an international consortium of 21 researchers who have used ambient audio sampling methodologies created a workgroup with the aim of improving upon existing published guidelines. We pooled formally and informally documented challenges pertaining to DEI in ambient audio sampling from our collective experience on 40+ studies (most of which used the EAR app) in clinical and healthy populations ranging from children to older adults. This article presents our resultant recommendations and argues for the incorporation of community-engaged research methods in observational ambulatory assessment designs looking forward. We provide concrete recommendations across each stage typical of an ambient audio sampling study (recruiting and enrolling participants, developing coding systems, training coders, handling multi-linguistic participants, data analysis and interpretation, and dissemination of results) as well as guiding questions that can be used to adapt these recommendations to project-specific constraints and needs.

Fluidity in capability: Longitudinal assessments of suicide capability using ecological momentary assessments

INTRODUCTION

Suicide capability is posited to facilitate the movement from ideation-to-action. Emerging evidence suggests capability comprises both trait- and state-like facets. This study examined fluctuations in and associations of acquired, dispositional, practical, and perceived capabilities, and suicidal mental imagery, and suicidal ideation.

METHOD

Seventy-five adults (48 females, Mage = 36.53 years) with lived experience of suicidal ideation and/or attempt responded to four prompts per day for 2 weeks that assessed suicide capability and suicidal ideation. Mean-squared successive differences and probability of acute change indices and multilevel models were used for analyses.

RESULTS

All facets of suicide capability fluctuated. Acquired and dispositional capabilities were trait-like, with practical and perceived capabilities being state-like. Suicidal mental imagery was the only facet of suicide capability that distinguished participants with a suicide attempt in the past 12 months from participants with a suicide attempt more than 1 year ago and suicide ideators. Suicidal mental imagery was associated with concurrent suicidal ideation and predictive of next assessment suicidal ideation.

CONCLUSION

Suicidal mental imagery may be uniquely associated with suicide capability. This study suggests there are trait- and state-like facets of capability that can combine to potentially ready an individual to engage in suicidal behaviors.

Translating the Transdiagnostic: Aligning Assessment Practices With Research Advances

Researchers and clinicians working within the Diagnostic and Statistical Manual of Mental Disorders: Fifth Edition, Text Rev (DSM-5-TR) framework face a difficult question: what does it mean to have an evidence-based assessment of a nonevidence-based diagnostic construct? Alternative nosological approaches conceptualize psychopathology as (a) hierarchical, allowing researchers to move between levels of description and (b) dimensional, eliminating artificial dichotomies between disorders and the dichotomy between mental illness and mental well-being. In this article, we provide an overview of ongoing efforts to develop validated measures of transdiagnostic nosologies (i.e., the Hierarchical Taxonomy of Psychopathology; HiTOP) with applications for measurement-based care. However, descriptive models like HiTOP, which summarize patterns of covariation among psychopathology symptoms, do not address dynamic processes underlying the problems associated with psychopathology. Ambulatory assessment, well-suited to examine such dynamic processes, has also developed rapidly in recent decades. Thus, the goal of the current article is twofold. First, we provide a brief overview of developments in constructing valid measures of the HiTOP model as well as developments in ambulatory assessment practices. Second, we outline how these parallel developments can be integrated to advance measurement-based treatment. We end with a discussion of some major challenges for future research to address to integrate advances more fully in transdiagnostic and ambulatory assessment practices.

Feasibility of Using Research Electronic Data Capture (REDCap) to Collect Daily Experiences of Parent-Child Dyads: Ecological Momentary Assessment Study

BACKGROUND

Intensive longitudinal data collection, including ecological momentary assessment (EMA), has the potential to reduce recall biases, collect more ecologically valid data, and increase our understanding of dynamic associations between variables. EMA is typically administered using an application that is downloaded on participants' devices, which presents cost and privacy concerns that may limit its use. Research Electronic Data Capture (REDCap), a web-based survey application freely available to nonprofit organizations, may allow researchers to overcome these barriers; however, at present, little guidance is available to researchers regarding the setup of EMA in REDCap, especially for those who are new to using REDCap or lack advanced programming expertise.

OBJECTIVE

We provide an example of a simplified EMA setup in REDCap. This study aims to demonstrate the feasibility of this approach. We provide information on survey completion and user behavior in a sample of parents and children recruited across Canada.

METHODS

We recruited 66 parents and their children (aged 9-13 years old) from an existing longitudinal cohort study to participate in a study on risk and protective factors for children's mental health. Parents received survey prompts (morning and evening) by email or SMS text message for 14 days, twice daily. Each survey prompt contained 2 sections, one for parents and one for children to complete.

RESULTS

The completion rates were good (mean 82%, SD 8%) and significantly higher on weekdays than weekends and in dyads with girls than dyads with boys. Children were available to respond to their own survey questions most of the time (in 1134/1498, 75.7% of surveys submitted). The number of assessments submitted was significantly higher, and response times were significantly faster among participants who selected SMS text message survey notifications compared to email survey notifications. The average response time was 47.0 minutes after the initial survey notification, and the use of reminder messages increased survey completion.

CONCLUSIONS

Our results support the feasibility of using REDCap for EMA studies with parents and children. REDCap also has features that can accommodate EMA studies by recruiting participants across multiple time zones and providing different survey delivery methods. Offering the option of SMS text message survey notifications and reminders may be an important way to increase completion rates and the timeliness of responses. REDCap is a potentially useful tool for researchers wishing to implement EMA in settings in which cost or privacy are current barriers. Researchers should weigh these benefits with the potential limitations of REDCap and this design, including staff time to set up, monitor, and clean the data outputs of the project.

Ecological Momentary Assessment: A Meta-Analysis on Designs, Samples, and Compliance Across Research Fields

Ecological Momentary Assessments (i.e., EMA, repeated assessments in daily life) are widespread in many fields of psychology and related disciplines. Yet, little knowledge exists on how differences in study designs and samples predict study compliance and dropout-two central parameters of data quality in (micro-)longitudinal research. The current meta-analysis included k = 477 articles (496 samples, total N = 677,536). For each article, we coded the design, sample characteristics, compliance, and dropout rate. The results showed that on average EMA studies scheduled six assessments per day, lasted for 7 days, and obtained a compliance of 79%. Studies with more assessments per day scheduled fewer assessment days, yet, the number of assessments did not predict compliance or dropout rates. Compliance was significantly higher in studies providing financial incentives. Otherwise, design or sample characteristics had little effects. We discuss the implications of the findings for planning, reporting, and reviewing EMA studies.

Clinician Opinions: Perceived Utility of and Barriers to Incorporating Pretreatment Ecological Momentary Assessment Into Clinical Care

Increasingly, clinicians have the option of including technological components into clinical care. However, little research has assessed clinicians' interest in utilizing technology in their clinical work. Here, clinicians reported their opinions related to using a mobile assessment platform (MAP) to collect ecological data from clients before providing clinical care. Practicing and training mental health clinicians (N = 221) reported demographics, characteristics of their clinical work, and confidence in their clinical skill. Participants then read a description of MAP and responded to questions about their perceived benefits of and barriers to its use. Last, participants rated their interest in using MAP in their clinical work. These perceptions were then factor-analyzed and the resulting factor scores were regressed onto clinician characteristics. Interest in using MAP was significantly lower for the group that endorsed a psychodynamic/psychoanalytic orientation and those with greater confidence in their clinical skills. Across scales, we found a pattern that participants who did not identify as male, those with a psychodynamic/psychoanalytic orientation, and those with greater confidence in their clinical skills tended to have lower ratings of the benefits of and higher ratings for the barriers to using MAP. Results revealed that significant differences in opinions about incorporating technology into clinical work exist between different groups of clinicians. This information may be useful in future work that attempts to implement technological tools into clinical settings.

Ecological momentary assessment of affect in depression-prone and control samples: Survey compliance and affective yield

BACKGROUND

Ecological momentary assessment (EMA) is a high-frequency ambulatory data collection approach that has come to be widely used in emotion research. It therefore is timely to examine two features of EMA needed for a successful study: compliance with survey prompts and high affective yield (survey prompts that capture affect experience). We posit that compliance may be subject to temporal variation (time-of-day, days in study) and individual differences (depression history), and that affective yield may also differ by social context.

METHODS

We examined these issues in a sample of 318 young adults (Mage = 24.7 years, SD = 2.7), including those with current depression (n = 28), remitted depression (n = 168) and never-depressed controls (n = 122) who participated in a 7-day EMA protocol of negative and positive affect (NA and PA, respectively).

RESULTS

The overall compliance rate was 91% and remained stable across the survey week. However, subjects were significantly less likely to respond to the first daily prompt compared to those that followed. The likelihood of capturing NA and PA decreased with each EMA protocol day, and affective yield across social contexts differed by participants' depression status.

LIMITATIONS

The sample was largely comprised of White young adults. Relative to the remitted and control groups, the sample size for the currently depressed was unbalanced.

CONCLUSION

Researchers can optimize compliance and affective yield within EMA by considering depression, time-of-day, study duration, and social context. Clinicians using EMA to monitor affect may benefit from considering these parameters.

The effects of assessment intensity on participant burden, compliance, within-person variance, and within-person relationships in ambulatory assessment

Considering the very large number of studies that have applied ambulatory assessment (AA) in the last decade across diverse fields of research, knowledge about the effects that these design choices have on participants’ perceived burden, data quantity (i.e., compliance with the AA protocol), and data quality (e.g., within-person relationships between time-varying variables) is surprisingly restricted. The aim of the current research was to experimentally manipulate aspects of an AA study’s assessment intensity—sampling frequency (Study 1) and questionnaire length (Study 2)—and to investigate their impact on perceived burden, compliance, within-person variability, and within-person relationships between time-varying variables. In Study 1, students (n = 313) received either 3 or 9 questionnaires per day for the first 7 days of the study. In Study 2, students (n = 282) received either a 33- or 82-item questionnaire three times a day for 14 days. Within-person variability and within-person relationships were investigated with respect to momentary pleasant-unpleasant mood and state extraversion. The results of Study 1 showed that a higher sampling frequency increased perceived burden but did not affect the other aspects we investigated. In Study 2, longer questionnaire length did not affect perceived burden or compliance but yielded a smaller degree of within-person variability in momentary mood (but not in state extraversion) and a smaller within-person relationship between state extraversion and mood. Differences between Studies 1 and 2 with respect to the type of manipulation of assessment intensity are discussed.

Feasibility of gathering momentary and daily assessments of fear of crime using a smartphone application (STUNDA): Methodological considerations and findings from a study among Swedish university students

The introduction of experience research using smartphone applications has enabled researchers to explore previously uncharted territories within the social science landscape. The use of experience research methodologies may both avoid some of the biases associated with conventional survey methods and enable researchers to gather information on situational dimensions of different outcomes. While existing smartphone-based experience research is found in several scientific disciplines, one outcome that has rarely been studied using this approach is fear of crime. Although fear of crime here only serves as an example of an outcome that may benefit from being examined in situ, the main focus of this article is directed at assessing the feasibility of gathering momentary and daily assessments on fear of crime using an experience sampling research design. A sample of 191 university students participated in a non-incentivized research study by downloading the smartphone application STUNDA, completing a baseline survey and repeatedly answering signal-contingent surveys, event-contingent surveys, and daily assessments across a self-defined study period. The results indicate that it is feasible to conduct research on fear of crime using a smartphone application. However, variations in participation across groups indicate that the methodology is perhaps best suited to dedicated groups of participants with a special interest in the studied outcome. Methodological considerations and implications for future research are discussed.

The Effects of Sampling Frequency and Questionnaire Length on Perceived Burden, Compliance, and Careless Responding in Experience Sampling Data in a Student Population

Currently, little is known about the association between assessment intensity, burden, data quantity, and data quality in experience sampling method (ESM) studies. Researchers therefore have insufficient information to make informed decisions about the design of their ESM study. Our aim was to investigate the effects of different sampling frequencies and questionnaire lengths on burden, compliance, and careless responding. Students ( n = 163) received either a 30- or 60-item questionnaire three, six, or nine times per day for 14 days. Preregistered multilevel regression analyses and analyses of variance were used to analyze the effect of design condition on momentary outcomes, changes in those outcomes over time, and retrospective outcomes. Our findings offer support for increased burden and compromised data quantity and quality with longer questionnaires, but not with increased sampling frequency. We therefore advise against the use of long ESM questionnaires, while high-sampling frequencies do not seem to be associated with negative consequences.