Toward Usable Evidence: Optimizing Knowledge Accumulation in HCI Research on Health Behavior Change

Conducting Research at the Intersection of HCI and Health: Building and Supporting Teams with Diverse Expertise to Increase Public Health Impact

Research at the intersection of human-computer interaction (HCI) and health is increasingly done by collaborative cross-disciplinary teams. The need for cross-disciplinary teams arises from the interdisciplinary nature of the work itself-with the need for expertise in a health discipline, experimental design, statistics, and computer science, in addition to HCI. This work can also increase innovation, transfer of knowledge across fields, and have a higher impact on communities. To succeed at a collaborative project, researchers must effectively form and maintain a team that has the right expertise, integrate research perspectives and work practices, align individual and team goals, and secure funding to support the research. However, successfully operating as a team has been challenging for HCI researchers, and can be limited due to a lack of training, shared vocabularies, lack of institutional incentives, support from funding agencies, and more; which significantly inhibits their impact. This workshop aims to draw on the wealth of individual experiences in health project team collaboration across the CHI community and beyond. By bringing together different stakeholders involved in HCI health research, together, we will identify needs experienced during interdisciplinary HCI and health collaborations. We will identify existing practices and success stories for supporting team collaboration and increasing HCI capacity in health research. We aim for participants to leave our workshop with a toolbox of methods to tackle future team challenges, a community of peers who can strive for more effective teamwork, and feeling positioned to make the health impact they wish to see through their work.

The Digital Therapeutics Real-World Evidence Framework: An Approach for Guiding Evidence-Based Digital Therapeutics Design, Development, Testing, and Monitoring

Digital therapeutics (DTx) are a promising way to provide safe, effective, accessible, sustainable, scalable, and equitable approaches to advance individual and population health. However, developing and deploying DTx is inherently complex in that DTx includes multiple interacting components, such as tools to support activities like medication adherence, health behavior goal-setting or self-monitoring, and algorithms that adapt the provision of these according to individual needs that may change over time. While myriad frameworks exist for different phases of DTx development, no single framework exists to guide evidence production for DTx across its full life cycle, from initial DTx development to long-term use. To fill this gap, we propose the DTx real-world evidence (RWE) framework as a pragmatic, iterative, milestone-driven approach for developing DTx. The DTx RWE framework is derived from the 4-phase development model used for behavioral interventions, but it includes key adaptations that are specific to the unique characteristics of DTx. To ensure the highest level of fidelity to the needs of users, the framework also incorporates real-world data (RWD) across the entire life cycle of DTx development and use. The DTx RWE framework is intended for any group interested in developing and deploying DTx in real-world contexts, including those in industry, health care, public health, and academia. Moreover, entities that fund research that supports the development of DTx and agencies that regulate DTx might find the DTx RWE framework useful as they endeavor to improve how DTxcan advance individual and population health.

Self-relevant appeals to engage in self-monitoring of alcohol use: A microrandomized trial

OBJECTIVE

While self-monitoring can help mitigate alcohol misuse in young adults, engagement with digital self-monitoring is suboptimal. The present study investigates the utility of two types of digital prompts (reminders) to encourage young adults to self-monitor their alcohol use. These prompts leverage information that is self-relevant (i.e., represents and is valuable) to the person.

METHOD

Five hundred ninety-one college students (Mage = 18; 61% = female, 76% = White) were enrolled in an 8-week intervention study involving biweekly digital self-monitoring of their alcohol use. At baseline, participants selected an item they would like to purchase for themselves and their preferred charitable organization. Then, biweekly, participants were microrandomized to a prompt highlighting the opportunity to either (a) win their preferred item (self-interest prompt); or (b) donate to their preferred charity (prosocial prompt). Following self-monitoring completion, participants allocated reward points toward lottery drawings for their preferred item or charity.

RESULTS

The self-interest (vs. prosocial) prompt was significantly more effective in promoting proximal self-monitoring at the beginning of the study, Est = exp(.14) = 1.15; 95% confidence interval (CI) [1.01, 1.29], whereas the prosocial (vs. self-interest) prompt was significantly more effective at the end, Est = exp(-.17) = 0.84; 95% CI [0.70, 0.98]. Further, the prosocial (vs. self-interest) prompt was significantly more effective among participants who previously allocated all their reward points to drawings for their preferred item, Est = exp(-.15) = 0.86; 95% CI [.75, .97].

CONCLUSIONS

These results suggest that the advantage of prompts that appeal to a person's self-interest (vs. prosocial) motives varies over time and based on what reward options participants prioritized in previous decisions. Theoretical and practical implications for intervention design are discussed. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The mechanics of implementation strategies and measures: advancing the study of implementation mechanisms

BACKGROUND

There is a fundamental gap in understanding the causal mechanisms by which strategies for implementing evidence-based practices address local barriers to effective, appropriate service delivery. Until this gap is addressed, scientific knowledge and practical guidance about which implementation strategies to use in which contexts will remain elusive. This research project aims to identify plausible strategy-mechanism linkages, develop causal models for mechanism evaluation, produce measures needed to evaluate such linkages, and make these models, methods, and measures available in a user-friendly website. The specific aims are as follows: (1) build a database of strategy-mechanism linkages and associated causal pathway diagrams, (2) develop psychometrically strong, pragmatic measures of mechanisms, and (3) develop and disseminate a website of implementation mechanisms knowledge for use by diverse stakeholders.

METHODS

For the first aim, a combination of qualitative inquiry, expert panel methods, and causal pathway diagramming will be used to identify and confirm plausible strategy-mechanism linkages and articulate moderators, preconditions, and proximal and distal outcomes associated with those linkages. For the second aim, rapid-cycle measure development and testing methods will be employed to create reliable, valid, pragmatic measures of six mechanisms of common strategies for which no high-quality measures exist. For the third aim, we will develop a user-friendly website and searchable database that incorporates user-centered design, disseminating the final product using social marketing principles.

DISCUSSION

Once strategy-mechanism linkages are identified using this multi-method approach, implementation scientists can use the searchable database to develop tailored implementation strategies and generate more robust evidence about which strategies work best in which contexts. Moreover, practitioners will be better able to select implementation strategies to address their specific implementation problems. New horizons in implementation strategy development, optimization, evaluation, and deployment are expected to be more attainable as a result of this research, which will lead to enhanced implementation of evidence-based interventions for cancer control, and ultimately improvements in patient outcomes.

mHealth Technology Design and Evaluation for Early Childhood Health Promotion: Systematic Literature Review

BACKGROUND

Recent increases in smartphone ownership among underserved populations have inspired researchers in medicine, computing, and health informatics to design and evaluate mobile health (mHealth) interventions, specifically for those supporting child development and growth. Although these interventions demonstrate possible effectiveness at larger scales, few of these interventions are evaluated to address racial disparities and health equity, which are known factors that affect relevance, uptake, and adherence in target populations.

OBJECTIVE

In this study, we aimed to identify and document the current design and evaluation practices of mHealth technologies that promote early childhood health, with a specific focus on opportunities for those processes to address health disparities and health equity.

METHODS

We completed a systematic literature review of studies that design and evaluate mHealth interventions for early childhood health promotion. We then analyzed these studies to identify opportunities to address racial disparities in early- and late-stage processes and to understand the potential efficacy of these interventions.

RESULTS

Across the literature from medical, computing, and health informatics fields, we identified 15 articles that presented a design or evaluation of a parent-facing health intervention. We found that using mobile-based systems to deliver health interventions was generally well accepted by parents of children aged <5 years. We also found that, when measured, parenting knowledge of early childhood health topics and confidence to engage in health-promoting behaviors improved. Design and evaluation methods held internal consistency within disciplines (eg, experimental study designs were the most prevalent in medical literature, while computing researchers used user-centered design methods in computing fields). However, there is little consistency in design or evaluation methods across fields.

CONCLUSIONS

To support more interventions with a comprehensive design and evaluation process, we recommend attention to design at the intervention (eg, reporting content sources) and system level; interdisciplinary collaboration in early childhood health intervention development can lead to large-scale deployment and success among populations.

TRIAL REGISTRATION

PROSPERO CRD42022359797; https://tinyurl.com/586nx9a2.

Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology

BACKGROUND

Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances the development of HITs, they rarely provide usable evidence that can contribute to our broader understanding of HITs.

OBJECTIVE

In this research, we aim to develop and demonstrate a variation of vignette testing that supports developers and designers in evaluating early-stage HIT designs while generating usable evidence for the broader research community.

METHODS

We proposed a method called health concept surveying for untangling the causal relationships that people develop around conceptual HITs. In health concept surveying, investigators gather reactions to design concepts through a scenario-based survey instrument. As the investigator manipulates characteristics related to their HIT, the survey instrument also measures proximal cognitive factors according to a health behavior change model to project how HIT design decisions may affect the adoption and acceptability of an HIT. Responses to the survey instrument were analyzed using path analysis to untangle the causal effects of these factors on the outcome variables.

RESULTS

We demonstrated health concept surveying in 3 case studies of sensor-based health-screening apps. Our first study (N=54) showed that a wait time incentive could influence more people to go see a dermatologist after a positive test for skin cancer. Our second study (N=54), evaluating a similar application design, showed that although visual explanations of algorithmic decisions could increase participant trust in negative test results, the trust would not have been enough to affect people's decision-making. Our third study (N=263) showed that people might prioritize test specificity or sensitivity depending on the nature of the medical condition.

CONCLUSIONS

Beyond the findings from our 3 case studies, our research uses the framing of the Health Belief Model to elicit and understand the intrinsic and extrinsic factors that may affect the adoption and acceptability of an HIT without having to build a working prototype. We have made our survey instrument publicly available so that others can leverage it for their own investigations.

Integrating User-Centered Design and Behavioral Science to Design a Mobile Intervention for Obesity and Binge Eating: Mixed Methods Analysis

Background

Accounting for how end users engage with technologies is imperative for designing an efficacious mobile behavioral intervention.

Objective

This mixed methods analysis examined the translational potential of user-centered design and basic behavioral science to inform the design of a new mobile intervention for obesity and binge eating.

Methods

A total of 22 adults (7/22, 32% non-Hispanic White; 8/22, 36% male) with self-reported obesity and recurrent binge eating (≥12 episodes in 3 months) who were interested in losing weight and reducing binge eating completed a prototyping design activity over 1 week. Leveraging evidence from behavioral economics on choice architecture, participants chose treatment strategies from 20 options (aligned with treatment targets composing a theoretical model of the relation between binge eating and weight) to demonstrate which strategies and treatment targets are relevant to end users. The process by which participants selected and implemented strategies and their change in outcomes were analyzed.

Results

Although prompted to select one strategy, participants selected between 1 and 3 strategies, citing perceived achievability, helpfulness, or relevance as selection reasons. Over the week, all practiced a strategy at least once; 82% (18/22) struggled with implementation, and 23% (5/22) added a new strategy. Several themes emerged on successes and challenges with implementation, yielding design implications for supporting users in behavior change. In postexperiment reflections, 82% (18/22) indicated the strategy was helpful, and 86% (19/22) planned to continue use. One-week average within-subject changes in weight (–2.2 [SD –5.0] pounds) and binge eating (–1.6 [SD –1.8] episodes) indicated small clinical improvement.

Conclusions

Applying user-centered design and basic behavioral science yielded design insights to incorporate personalization through user choice with guidance, which may enhance engagement with and potential efficacy of digital health interventions.

Optimizing Implementation in Cancer Control (OPTICC): protocol for an implementation science center

BACKGROUND

Evidence-based interventions (EBIs) could reduce cervical cancer deaths by 90%, colorectal cancer deaths by 70%, and lung cancer deaths by 95% if widely and effectively implemented in the USA. Yet, EBI implementation, when it occurs, is often suboptimal. This manuscript outlines the protocol for Optimizing Implementation in Cancer Control (OPTICC), a new implementation science center funded as part of the National Cancer Institute Implementation Science Consortium. OPTICC is designed to address three aims. Aim 1 is to develop a research program that supports developing, testing, and refining of innovative, efficient methods for optimizing EBI implementation in cancer control. Aim 2 is to support a diverse implementation laboratory of clinical and community partners to conduct rapid, implementation studies anywhere along the cancer care continuum for a wide range of cancers. Aim 3 is to build implementation science capacity in cancer control by training new investigators, engaging established investigators in cancer-focused implementation science, and contributing to the Implementation Science Consortium in Cancer.

METHODS

Three cores serve as OPTICC's foundation. The Administrative Core plans coordinates and evaluates the Center's activities and leads its capacity-building efforts. The Implementation Laboratory Core (I-Lab) coordinates a network of diverse clinical and community sites, wherein studies are conducted to optimize EBI implementation, implement cancer control EBIs, and shape the Center's agenda. The Research Program Core conducts innovative implementation studies, measurement and methods studies, and pilot studies that advance the Center's theme. A three-stage approach to optimizing EBI implementation is taken-(I) identify and prioritize determinants, (II) match strategies, and (III) optimize strategies-that is informed by a transdisciplinary team of experts leveraging multiphase optimization strategies and criteria, user-centered design, and agile science.

DISCUSSION

OPTICC will develop, test, and refine efficient and economical methods for optimizing EBI implementation by building implementation science capacity in cancer researchers through applications with our I-Lab partners. Once refined, OPTICC will disseminate its methods as toolkits accompanied by massive open online courses, and an interactive website, the latter of which seeks to simultaneously accumulate knowledge across OPTICC studies.

Precision Health: The Role of the Social and Behavioral Sciences in Advancing the Vision

BACKGROUND

In 2015, Collins and Varmus articulated a vision for precision medicine emphasizing molecular characterization of illness to identify actionable biomarkers to support individualized treatment. Researchers have argued for a broader conceptualization, precision health. Precision health is an ambitious conceptualization of health, which includes dynamic linkages between research and practice as well as medicine, population health, and public health. The goal is a unified approach to match a full range of promotion, prevention, diagnostic, and treatment interventions to fundamental and actionable determinants of health; to not just address symptoms, but to directly target genetic, biological, environmental, and social and behavioral determinants of health.

PURPOSE

The purpose of this paper is to elucidate the role of social and behavioral sciences within precision health.

MAIN BODY

Recent technologies, research frameworks, and methods are enabling new approaches to measure, intervene, and conduct social and behavioral science research. These approaches support three opportunities in precision health that the social and behavioral sciences could colead including: (a) developing interventions that continuously "tune" to each person's evolving needs; (b) enhancing and accelerating links between research and practice; and (c) studying mechanisms of change in real-world contexts. There are three challenges for precision health: (a) methods of knowledge organization and curation; (b) ethical conduct of research; and (c) equitable implementation of precision health.

CONCLUSIONS

Precision health requires active coleadership from social and behavioral scientists. Prior work and evidence firmly demonstrate why the social and behavioral sciences should colead with regard to three opportunity and three challenge areas.

Designing the Future of Children's Mental Health Services

Advancements in evidence-based psychosocial interventions, digital technologies, and implementation strategies (i.e., health services research products) for youth mental health services have yet to yield significant improvement in public health outcomes. Achieving such impact will require that these research products are easy to use, useful, and contextually appropriate. This paper describes how human-centered design (HCD), an approach that aligns product development with the needs of the people and settings that use those products, can be leveraged to improve youth mental health services. We articulate how HCD can advance accessibility, effectiveness, and equity, with specific consideration of unique aspects of youth mental health services.

Integrating Taxonomies Into Theory-Based Digital Health Interventions for Behavior Change: A Holistic Framework

Digital health interventions (DHIs) have been emerging in the last decade. Due to their interdisciplinary nature, DHIs are guided and influenced by theories (eg, behavioral theories, behavior change technologies, and persuasive technology) from different research communities. However, DHIs are always coded using various taxonomies and reported in insufficient perspectives. This inconsistency and incomprehensiveness will cause difficulty in conducting systematic reviews and sharing contributions among communities. Therefore, based on existing related work, we propose a holistic framework that embeds behavioral theories, behavior change technique taxonomy, and persuasive system design principles. Including four development steps, two toolboxes, and one workflow, our framework aims to guide DHI developers to design, evaluate, and report their work in a formative and comprehensive way.

Rethinking Evaluations Of mHealth Systems For Behavior Change

Much of the recent research in mobile health (mHealth) has focused on the development of apps and wearables for promoting healthy behavior changes, such as losing weight, increasing physical activity, or adhering to a medication regimen. These interactive systems help users make changes in their behavior by, for instance, tracking healthrelated activities and states, providing feedback, helping users set and track goals, and facilitating supportive social interactions. We refer to the features that implement such functionality as the system's "intervention components," as they are designed to actuate psychosocial mechanisms (e.g., modeling, selfefficacy, positive reinforcement, etc.) thought to mediate the behavior change process. As with any other type of behavioral intervention, mHealth systems are only effective for some users and some of the time, but insofar as they do work, they do so mainly through the mechanisms of change that are activated via users' interactions with the system's intervention components.

Tutorial for Using Control Systems Engineering to Optimize Adaptive Mobile Health Interventions

BACKGROUND

Adaptive behavioral interventions are individualized interventions that vary support based on a person's evolving needs. Digital technologies enable these adaptive interventions to function at scale. Adaptive interventions show great promise for producing better results compared with static interventions related to health outcomes. Our central thesis is that adaptive interventions are more likely to succeed at helping individuals meet and maintain behavioral targets if its elements can be iteratively improved via data-driven testing (ie, optimization). Control systems engineering is a discipline focused on decision making in systems that change over time and has a wealth of methods that could be useful for optimizing adaptive interventions.

OBJECTIVE

The purpose of this paper was to provide an introductory tutorial on when and what to do when using control systems engineering for designing and optimizing adaptive mobile health (mHealth) behavioral interventions.

OVERVIEW

We start with a review of the need for optimization, building on the multiphase optimization strategy (MOST). We then provide an overview of control systems engineering, followed by attributes of problems that are well matched to control engineering. Key steps in the development and optimization of an adaptive intervention from a control engineering perspective are then summarized, with a focus on why, what, and when to do subtasks in each step.

IMPLICATIONS

Control engineering offers exciting opportunities for optimizing individualization and adaptation elements of adaptive interventions. Arguably, the time is now for control systems engineers and behavioral and health scientists to partner to advance interventions that can be individualized, adaptive, and scalable. This tutorial should aid in creating the bridge between these communities.

Designing for Motivation, Engagement and Wellbeing in Digital Experience

Research in psychology has shown that both motivation and wellbeing are contingent on the satisfaction of certain psychological needs. Yet, despite a long-standing pursuit in human-computer interaction (HCI) for design strategies that foster sustained engagement, behavior change and wellbeing, the basic psychological needs shown to mediate these outcomes are rarely taken into account. This is possibly due to the lack of a clear model to explain these needs in the context of HCI. Herein we introduce such a model: Motivation, Engagement and Thriving in User Experience (METUX). The model provides a framework grounded in psychological research that can allow HCI researchers and practitioners to form actionable insights with respect to how technology designs support or undermine basic psychological needs, thereby increasing motivation and engagement, and ultimately, improving user wellbeing. We propose that in order to address wellbeing, psychological needs must be considered within five different spheres of analysis including: at the point of technology adoption, during interaction with the interface, as a result of engagement with technology-specific tasks, as part of the technology-supported behavior, and as part of an individual's life overall. These five spheres of experience sit within a sixth, society, which encompasses both direct and collateral effects of technology use as well as non-user experiences. We build this model based on existing evidence for basic psychological need satisfaction, including evidence within the context of the workplace, computer games, and health. We extend and hone these ideas to provide practical advice for designers along with real world examples of how to apply the model to design practice.