Developing smartphone apps for behavioural studies: The AlcoRisk app case study

Using Monte Carlo Simulation to Forecast the Scientific Utility of Psychological App Studies: A Tutorial

Mobile applications offer a wide range of opportunities for psychological data collection, such as increased ecological validity and greater acceptance by participants compared to traditional laboratory studies. However, app-based psychological data also pose data-analytic challenges because of the complexities introduced by missingness and interdependence of observations. Consequently, researchers must weigh the advantages and disadvantages of app-based data collection to decide on the scientific utility of their proposed app study. For instance, some studies might only be worthwhile if they provide adequate statistical power. However, the complexity of app data forestalls the use of simple analytic formulas to estimate properties such as power. In this paper, we demonstrate how Monte Carlo simulations can be used to investigate the impact of app usage behavior on the utility of app-based psychological data. We introduce a set of questions to guide simulation implementation and showcase how we answered them for the simulation in the context of the guessing game app Who Knows (Rau et al., 2023). Finally, we give a brief overview of the simulation results and the conclusions we have drawn from them for real-world data generation. Our results can serve as an example of how to use a simulation approach for planning real-world app-based data collection.

Detecting and Understanding Social Influence During Drinking Situations: Protocol for a Bluetooth-Based Sensor Feasibility and Acceptability Study

BACKGROUND

High-risk alcohol consumption among young adults frequently occurs in the presence of peers who are also drinking. A high-risk drinking situation may consist of particular social network members who have a primary association with drinking. Fine-grained approaches such as ecological momentary assessment (EMA) are growing in popularity for studying real-time social influence, but studies using these approaches exclusively rely on participant self-report. Passive indicators of peer presence using Bluetooth-based technology to detect real-time interactions have the potential to assist in the development of just-in-time interventions.

OBJECTIVE

This study seeks to examine the feasibility and acceptability of using a Bluetooth-based sensor and smartphone app to measure social contact in real-world drinking situations.

METHODS

Young adults (N=20) who drink heavily and report social drinking will be recruited from the community to participate in a 3-week EMA study. Using a social network interview, index participants will identify and recruit 3 of their friends to carry a Bluetooth beacon. Participants will complete a series of EMA reports on their own personal Android devices including random reports; morning reports; first-drink reports; and signal-contingent reports, which are triggered following the detection of a beacon carried by a peer participant. EMA will assess alcohol use and characteristics of the social environment, including who is nearby and who is drinking. For items about peer proximity and peer drinking, a customized peer list will be presented to participants. Feedback about the study protocol will be ascertained through weekly contact with both index and peer participants, followed by a qualitative interview at the end of the study. We will examine the feasibility and acceptability of recruitment, enrollment of participants and peers, and retention. Feasibility will be determined using indexes of eligibility, enrollment, and recruitment. Acceptability will be determined through participant enrollment and retention, protocol compliance, and participant-reported measures of acceptability. Feasibility and acceptability for peer participants will be informed by enrollment rates, latency to enrollment, compliance with carrying the beacon, and self-reported reasons for compliance or noncompliance with beacon procedures. Finally, EMA data about peer proximity and peer drinking will support the validity of the peer selection process.

RESULTS

Participant recruitment began in February 2023, and enrollment was completed in December 2023. Results will be reported in 2025.

CONCLUSIONS

The protocol allows us to examine the feasibility and acceptability of a Bluetooth-based sensor for the detection of social contact between index participants and their friends, including social interactions during real-world drinking situations. Data from this study will inform just-in-time adaptive interventions seeking to address drinking in the natural environment by providing personalized feedback about a high-risk social context and alerting an individual that they are in a potentially unsafe situation.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50650.

A Bluetooth-Based Smartphone App for Detecting Peer Proximity: Protocol for Evaluating Functionality and Validity

BACKGROUND

While ecological momentary assessment (EMA) is commonly used to study social contexts and social influence in the real world, EMA almost exclusively relies on participant self-report of present circumstances, including the proximity to influential peers. There is the potential for developing a proximity sensing approach that uses small Bluetooth beacons and smartphone-based detection and data collection to collect information about interactions between individuals passively in real time.

OBJECTIVE

This paper aims to describe the methods for evaluating the functionality and validity of a Bluetooth-based beacon and a smartphone app to identify when ≥2 individuals are physically proximal.

METHODS

We will recruit 20 participants aged 18 to 29 years with Android smartphones to complete a 3-week study during which beacon detection and self-report data will be collected using a smartphone app (MEI Research). Using an interviewer-administered social network interview, participants will identify up to 3 peers of the same age who are influential on health behavior (alcohol use in this study). These peers will be asked to carry a Bluetooth beacon (Kontakt asset tag) for the duration of the study; each beacon has a unique ID that, when detected, will be recorded by the app on the participant's phone. Participants will be prompted to respond to EMA surveys (signal-contingent reports) when a peer beacon encounter meets our criteria and randomly 3 times daily (random reports) and every morning (morning reports) to collect information about the presence of peers. In all reports, the individualized list of peers will be presented to participants, followed by questions about peer and participant behavior, including alcohol use. Data from multiple app data sets, including beacon encounter specifications, notification, and app logs, participant EMA self-reports and postparticipation interviews, and peer surveys, will be used to evaluate project goals. We will examine the functionality of the technology, including the stability of the app (eg, app crashes and issues opening the app), beacon-to-app detection (ie, does the app detect proximal beacons?), and beacon encounter notification when encounter criteria are met. The validity of the technology will be defined as the concordance between passive detection of peers via beacon-to-app communication and the participant's EMA report of peer presence. Disagreement between the beacon and self-report data (ie, false negatives and false positives) will be investigated in multiple ways (ie, to determine if the reason was technology-related or participant compliance-related) using encounter data and information collected from participants and peers.

RESULTS

Participant recruitment began in February 2023, and enrollment was completed in December 2023. Results will be reported in 2025.

CONCLUSIONS

This Bluetooth-based technology has important applications and clinical implications for various health behaviors, including the potential for just-in-time adaptive interventions that target high-risk behavior in real time.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/50241.

Mental health-related telemedicine interventions for pregnant women and new mothers: a systematic literature review

BACKGROUND

Pregnancy and the postpartum period are times when women are at increased risk for depression and mental problems. This may also negatively affect the foetus. Thus, there is a need for interventions with low-threshold access and care. Telemedicine interventions are a promising approach to address these issues. This systematic literature review examined the efficacy of telemedicine interventions for pregnant women and/or new mothers to address mental health-related outcomes. The primary objective was to analyse whether telemedicine interventions can reduce mental health problems in pregnant women and new mothers. The secondary aim was to clarify the impact of type of interventions, their frequency and their targets.

METHODS

Inclusion criteria: randomized controlled trials, with participants being pregnant women and/or new mothers (with infants up to twelve months), involving telemedicine interventions of any kind (e.g. websites, apps, chats, telephone), and addressing any mental health-related outcomes like depression, postnatal depression, anxiety, stress and others. Search terms were pregnant women, new mothers, telemedicine, RCT (randomised controlled trials), mental stress as well as numerous synonyms including medical subject headings. The literature search was conducted within the databases PubMed, Cochrane Library, Web of Science and PsycINFO. Screening, inclusion of records and data extraction were performed by two researchers according to the PRISMA guidelines, using the online tool CADIMA.

RESULTS

Forty four articles were included. A majority (62%) reported significantly improved mental health-related outcomes for participants receiving telemedicine interventions compared to control. In particular (internet-delivered) Cognitive Behavioural Therapy was successful for depression and stress, and peer support improved outcomes for postnatal depression and anxiety. Interventions with preventive approaches and interventions aimed at symptom reduction were largely successful. For the most part there was no significant improvement in the symptoms of anxiety.

CONCLUSION

Telemedicine interventions evaluated within RCTs were mostly successful. However, they need to be designed to specifically target a certain mental health issue because there is no one-size-fits-all approach. Further research should focus on which specific interventions are appropriate for which mental health outcomes in terms of intervention delivery modes, content, target approaches, etc. Further investigation is needed, in particular with regard to anxiety.

Mobile Data Collection of Cognitive-Behavioral Tasks in Substance Use Disorders: Where Are We Now?

INTRODUCTION

Over the last decades, our understanding of the cognitive, motivational, and neural processes involved in addictive behavior has increased enormously. A plethora of laboratory-based and cross-sectional studies has linked cognitive-behavioral measures to between-subject differences in drinking behavior. However, such laboratory-based studies inevitably suffer from small sample sizes and the inability to link temporal fluctuations in task measures to fluctuations in real-life substance use. To overcome these problems, several existing behavioral tasks have been transferred to smartphones to allow studying cognition in the field.

METHOD

In this narrative review, we first summarize studies that used existing behavioral tasks in the laboratory and self-reports of substance use with ecological momentary assessment (EMA) in the field. Next, we review studies on psychometric properties of smartphone-based behavioral tasks. Finally, we review studies that used both smartphone-based tasks and self-reports with EMA in the field.

RESULTS

Overall, studies were scarce and heterogenous both in tasks and in study outcomes. Nevertheless, existing findings are promising and point toward several methodological recommendations: concerning psychometrics, studies show that - although more systematic studies are necessary - task validity and reliability can be improved, for example, by analyzing several measurement sessions at once rather than analyzing sessions separately. Studies that use tasks in the field, moreover, show that power can be improved by choosing sampling schemes that combine time-based with event-based sampling, rather than relying on time-based sampling alone. Increasing sampling frequency can further increase power. However, as this also increases the burden to participants, more research is necessary to determine the ideal sampling frequency for each task.

CONCLUSION

Although more research is necessary to systematically study both the psychometrics of smartphone-based tasks and the frequency at which task measures fluctuate, existing studies are promising and reveal important methodological recommendations useful for researchers interested in implementing behavioral tasks in EMA studies.

Collecting qualitative data using a smartphone app: Learning from research involving people with experience of multiple disadvantage

This article discusses the use of a smartphone application (‘app’) as a method for undertaking diary-based qualitative research with a group of participants with lived experience of multiple disadvantage (experience of two or more of homelessness, offending, mental ill-health and substance misuse). As part of a multi-method approach to the study, an app was designed for participants to capture rich, real-time qualitative data through a variety of media – text, audio, image and video – to make it accessible and interactive for participants. There is an identified gap in the use of apps to collect qualitative data with marginalised groups, and this paper reflects on our experience of designing and implementing an app as part of a wider multi-method research project. We argue that, particularly during the challenges of adapting research methods in light of the COVID-19 pandemic, the smartphone app has considerable potential for engaging participants in a flexible way to involve marginalised groups. Qualitative researchers in particular should consider the potential of this approach, and encourage others to share their reflections and learning.

Smartphone-based alcohol interventions: A systematic review on the role of notifications in changing behaviors toward alcohol

Background: Smartphone-based interventions are increasingly being used to facilitate positive behavior change, including reducing alcohol consumption. However, less is known about the effects of notifications to support this change, including intervention engagement and adherence. The aim of this review was to assess the role of notifications in smartphone-based interventions designed to support, manage, or reduce alcohol consumption. Methods: Five electronic databases were searched to identify studies meeting inclusion criteria: (1) studies using a smartphone-based alcohol intervention, (2) the intervention used notifications, and (3) published between 1st January 2007 and 30th April 2021 in English. PROSPERO was searched to identify any completed, ongoing, or planned systematic reviews and meta-analyses of relevance. The reference lists of all included studies were searched. Results: Overall, 14 papers were identified, reporting on 10 different interventions. The strength of the evidence regarding the role and utility of notifications in changing behavior toward alcohol of the reviewed interventions was inconclusive. Only one study drew distinct conclusions about the relationships between notifications and app engagement, and notifications and behavior change. Conclusions: Although there are many smartphone-based interventions to support alcohol reduction, this review highlights a lack of evidence to support the use of notifications (such as push notifications, alerts, prompts, and nudges) used within smartphone interventions for alcohol management aiming to promote positive behavior change. Included studies were limited due to small sample sizes and insufficient follow-up. Evidence for the benefits of smartphone-based alcohol interventions remains promising, but the efficacy of using notifications, especially personalized notifications, within these interventions remain unproven.

Smartphone-Based Interventions to Reduce Sedentary Behavior and Promote Physical Activity Using Integrated Dynamic Models: Systematic Review

BACKGROUND

Traditional psychological theories are inadequate to fully leverage the potential of smartphones and improve the effectiveness of physical activity (PA) and sedentary behavior (SB) change interventions. Future interventions need to consider dynamic models taken from other disciplines, such as engineering (eg, control systems). The extent to which such dynamic models have been incorporated in the development of interventions for PA and SB remains unclear.

OBJECTIVE

This review aims to quantify the number of studies that have used dynamic models to develop smartphone-based interventions to promote PA and reduce SB, describe their features, and evaluate their effectiveness where possible.

METHODS

Databases including PubMed, PsycINFO, IEEE Xplore, Cochrane, and Scopus were searched from inception to May 15, 2019, using terms related to mobile health, dynamic models, SB, and PA. The included studies involved the following: PA or SB interventions involving human adults; either developed or evaluated integrated psychological theory with dynamic theories; used smartphones for the intervention delivery; the interventions were adaptive or just-in-time adaptive; included randomized controlled trials (RCTs), pilot RCTs, quasi-experimental, and pre-post study designs; and were published from 2000 onward. Outcomes included general characteristics, dynamic models, theory or construct integration, and measured SB and PA behaviors. Data were synthesized narratively. There was limited scope for meta-analysis because of the variability in the study results.

RESULTS

A total of 1087 publications were screened, with 11 publications describing 8 studies included in the review. All studies targeted PA; 4 also included SB. Social cognitive theory was the major psychological theory upon which the studies were based. Behavioral intervention technology, control systems, computational agent model, exploit-explore strategy, behavioral analytic algorithm, and dynamic decision network were the dynamic models used in the included studies. The effectiveness of quasi-experimental studies involved reduced SB (1 study; P=.08), increased light PA (1 study; P=.002), walking steps (2 studies; P=.06 and P<.001), walking time (1 study; P=.02), moderate-to-vigorous PA (2 studies; P=.08 and P=.81), and nonwalking exercise time (1 study; P=.31). RCT studies showed increased walking steps (1 study; P=.003) and walking time (1 study; P=.06). To measure activity, 5 studies used built-in smartphone sensors (ie, accelerometers), 3 of which used the phone's GPS, and 3 studies used wearable activity trackers.

CONCLUSIONS

To our knowledge, this is the first systematic review to report on smartphone-based studies to reduce SB and promote PA with a focus on integrated dynamic models. These findings highlight the scarcity of dynamic model-based smartphone studies to reduce SB or promote PA. The limited number of studies that incorporate these models shows promising findings. Future research is required to assess the effectiveness of dynamic models in promoting PA and reducing SB.

TRIAL REGISTRATION

International Prospective Register of Systematic Reviews (PROSPERO) CRD42020139350; https://www.crd.york.ac.uk/PROSPERO/display_record.php?RecordID=139350.

Feasibility of gathering momentary and daily assessments of fear of crime using a smartphone application (STUNDA): Methodological considerations and findings from a study among Swedish university students

The introduction of experience research using smartphone applications has enabled researchers to explore previously uncharted territories within the social science landscape. The use of experience research methodologies may both avoid some of the biases associated with conventional survey methods and enable researchers to gather information on situational dimensions of different outcomes. While existing smartphone-based experience research is found in several scientific disciplines, one outcome that has rarely been studied using this approach is fear of crime. Although fear of crime here only serves as an example of an outcome that may benefit from being examined in situ, the main focus of this article is directed at assessing the feasibility of gathering momentary and daily assessments on fear of crime using an experience sampling research design. A sample of 191 university students participated in a non-incentivized research study by downloading the smartphone application STUNDA, completing a baseline survey and repeatedly answering signal-contingent surveys, event-contingent surveys, and daily assessments across a self-defined study period. The results indicate that it is feasible to conduct research on fear of crime using a smartphone application. However, variations in participation across groups indicate that the methodology is perhaps best suited to dedicated groups of participants with a special interest in the studied outcome. Methodological considerations and implications for future research are discussed.

Notifications to Improve Engagement With an Alcohol Reduction App: Protocol for a Micro-Randomized Trial

BACKGROUND

Drink Less is a behavior change app that aims to help users in the general adult population reduce hazardous and harmful alcohol consumption. The app includes a daily push notification, delivered at 11 am, asking users to "Please complete your mood and drinking diaries." Previous analysis of Drink Less engagement data suggests the current notification strongly influences how users engage with the app in the subsequent hour. To exploit a potential increase of vulnerability of excess drinking and opportunity to engage with the app in the evenings, we changed the delivery time from 11 am to 8 pm. We now aim to further optimise the content and sequence of notifications, testing 30 new evidence-informed notifications targeting the user's perceived usefulness of the app.

OBJECTIVE

The primary objective is to assess whether sending a notification at 8 pm increases behavioral engagement (opening the app) in the subsequent hour. Secondary objectives include comparing the effect of the new bank of messages with the standard message and effect moderation over time. We also aim to more generally understand the role notifications have on the overall duration, depth, and frequency of engagement with Drink Less over the first 30 days after download.

METHODS

This is a protocol for a micro-randomized trial with two additional parallel arms. Inclusion criteria are Drink Less users who (1) consent to participate in the trial; (2) self-report a baseline Alcohol Use Disorders Identification Test score of 8 or above; (3) reside in the United Kingdom; (4) age ≥18 years and; (5) report interest in drinking less alcohol. In the micro-randomized trial, participants will be randomized daily at 8 pm to receive no notification, a notification with text from the new message bank, or the standard message. The primary outcome is the time-varying, binary outcome of "Did the user open the app in the hour from 8 pm to 9 pm?". The primary analysis will estimate the marginal relative risk for the notifications using an estimator developed for micro-randomized trials with binary outcomes. Participants randomized to the parallel arms will receive no notifications (Secondary Arm A), or the standard notification delivered daily at 11 am (Secondary Arm B) over 30 days, allowing the comparison of overall engagement between different notification delivery strategies.

RESULTS

Approval was granted by the University College of London's Departmental Research Ethics Committee (CEHP/2016/556) on October 11, 2019, and The London School of Hygiene and Tropical Medicine Interventions Research Ethics Committee (17929) on November 27, 2019. Recruitment began on January 2, 2020, and is ongoing.

CONCLUSIONS

Understanding how push notifications may impact engagement with a behavior change app can lead to further improvements in engagement, and ultimately help users reduce their alcohol consumption. This understanding may also be generalizable to other apps that target a variety of behavior changes.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/18690.

Strong health messages increase audience brain coupling

Mass media messaging is central for health communication. The success of these efforts, however, depends on whether health messages resonate with their target audiences. Here, we used electroencephalography (EEG) to capture brain responses of young adults - an important target group for alcohol prevention - while they viewed real-life video messages of varying perceived message effectiveness about risky alcohol use. We found that strong messages, which were rated to be more effective, prompted enhanced inter-subject correlation (ISC). In further analyses, we linked ISC to subsequent drinking behavior change and used time-resolved EEG-ISC to model functional neuroimaging data (fMRI) of an independent audience. The EEG measure was not only related to sensory-perceptual brain regions, but also to regions previously related to successful messaging, i.e., cortical midline regions and the insula. The findings suggest EEG-ISC as a marker for audience engagement and effectiveness of naturalistic health messages, which could quantify the impact of mass communication within the brains of small target audiences.

A Smartphone App to Assess Alcohol Consumption Behavior: Development, Compliance, and Reactivity

BACKGROUND

There are disadvantages-largely related to cost, participant burden, and missing data-associated with traditional electronic methods of assessing drinking behavior in real time. This potentially diminishes some of the advantages-namely, enhanced sample size and diversity-typically attributed to these methods. Download of smartphone apps to participants' own phones might preserve these advantages. However, to date, few researchers have detailed the process involved in developing custom-built apps for use in the experimental arena or explored methodological concerns regarding compliance and reactivity.

OBJECTIVE

The aim of this study was to describe the process used to guide the development of a custom-built smartphone app designed to capture alcohol intake behavior in the healthy population. Methodological issues related to compliance with and reactivity to app study protocols were examined. Specifically, we sought to investigate whether hazard and nonhazard drinkers would be equally compliant. We also explored whether reactivity in the form of a decrease in drinking or reduced responding ("yes") to drinking behavior would emerge as a function of hazard or nonhazard group status.

METHODS

An iterative development process that included elements typical of agile software design guided the creation of the CNLab-A app. Healthy individuals used the app to record alcohol consumption behavior each day for 21 days. Submissions were either event- or notification-contingent. We considered the size and diversity of the sample, and assessed the data for evidence of app protocol compliance and reactivity as a function of hazard and nonhazard drinker status.

RESULTS

CNLab-A yielded a large and diverse sample (N=671, mean age 23.12). On average, participants submitted data on 20.27 (SD 1.88) out of 21 days (96.5%, 20.27/21). Both hazard and nonhazard drinkers were highly compliant with app protocols. There were no differences between groups in terms of number of days of app use (P=.49) or average number of app responses (P=.54). Linear growth analyses revealed hazardous drinkers decreased their alcohol intake by 0.80 standard drinks over the 21-day experimental period. There was no change to the drinking of nonhazard individuals. Both hazard and nonhazard drinkers showed a slight decrease in responding ("yes") to drinking behavior over the same period.

CONCLUSIONS

Smartphone apps participants download to their own phones are effective and methodologically sound means of obtaining alcohol consumption information for research purposes. Although further investigation is required, such apps might, in future, allow for a more thorough examination of the antecedents and consequences of drinking behavior.