Use of Human-Centered Design to Improve Implementation of Evidence-Based Psychotherapies in Low-Resource Communities: Protocol for Studies Applying a Framework to Assess Usability

Cocreative Development of Robotic Interaction Systems for Health Care: Scoping Review

Background

Robotic technologies present challenges to health care professionals and are therefore rarely used. Barriers such as lack of controllability and adaptability and complex control functions affect the human-robot relationship. In addition to educational opportunities, the possibility of individual adaptation can improve the usability and practical implementation of robotics. Previous work has focused on developments from a technology-centered perspective and has included user interests too late in the process.

Objective

This study addresses the following research question: What cocreative research approaches are used in the field of nursing robotics to improve the usability, intended use, and goal-directed application of robotic developments for nurses and to support the nursing process?

Methods

This scoping review provides an overview of the topic and the research activities taking place within it. Five databases and the reference lists of the identified publications were searched for studies without further restrictions. Studies were included if they developed and evaluated interaction and control platforms for robotic systems in health care in a cocreative way with end users.

Results

The search resulted in 419 hits, of which 3 publications were included. All publications were feasibility or user studies that were mainly carried out in the European Union. The 3 interaction and control platforms presented were all prototypes and not commercially available. In addition to those in need of care, all studies also included family carers and health care professionals.

Conclusions

Robotic interaction and control platforms in health care are rarely, if ever, developed and evaluated with feasibility or user studies that include prototypes and end users. While the involvement of end users is crucial, this review emphasizes that all stakeholders, including health care professionals, should participate in the development process to ensure a holistic understanding of application needs and a focus on user experiences and practical health care needs. It is emphasized that the active involvement of end users in the development process is critical to effectively meeting the needs of the target group.

Feasibility study of a telehealth school-based behavioral parent training group program for attention-deficit/hyperactivity disorder

OBJECTIVE

To evaluate the feasibility and preliminary efficacy of Telehealth Behavioral Parent Training (T-BPT), a school telehealth group intervention for attention-deficit/hyperactivity disorder (ADHD) with a companion training program for school clinicians.

METHODS

T-BPT was developed in an iterative three-phase design in partnership with community stakeholders during the COVID-19 pandemic. School clinicians (N = 4) delivered T-BPT over 8 weeks to parents (N = 21, groups of 5-6 per school) of children (Grades 2-5) with ADHD while simultaneously receiving training and consultation from PhD-level study trainers. A single-arm open trial was used to assess feasibility, engagement, and preliminary efficacy.

RESULTS

Parents and school clinicians endorsed high feasibility, acceptability, and usability of T-BPT. Parent attendance was high (M = 94.6%) and a majority of parents (66.7%) attended all eight sessions. Preliminary outcomes indicate moderate to large reductions in parent-reported ADHD symptoms (ω2 = .36), functional and clinical global impairment (ω2s= .21 and .19, respectively), and distance learning challenges (ω2 = .22).

CONCLUSIONS

Results were in line with in-person delivery, indicating promising feasibility of school telehealth BPT groups. This study also provided further support for the feasibility of the remote training model for school clinicians. Implications of the commonly endorsed barriers and benefits beyond COVID-19 and relevance to under resourced communities are also discussed.

Leveraging Feedback From Families of Children With Autism to Create Digital Support for Service Navigation: Descriptive Study

BACKGROUND

It is difficult for families to navigate and access services for their children with autism. Barriers to service access are compounded among families from low-resourced backgrounds.

OBJECTIVE

The purpose of our study was to explore the development of an app to facilitate access to services among families of children with autism from low-resourced backgrounds. Our specific aims were to explore feedback from an advisory board about the app and to explore feedback from navigators about the app.

METHODS

Via a multistage codevelopment process, we elicited feedback from 5 key parties: the research team, a community organization, the app development team, the advisory board, and family navigators. Collectively, 36 individuals provided feedback about the development of the app via individual interviews, focus groups, observations, and surveys. The key features of the app included a dashboard showing the service needs of the family and related resources, a messaging feature between the family, the navigator, and the supervisor, and a fidelity checklist and evaluation feature.

RESULTS

The advisory board provided feedback about the app to increase its user-friendliness, include the ability to develop an action plan, improve the identification of needed services, and add information about service providers. Navigators suggested that the app should connect navigators to one another, have a clearer purpose for the notes section, and reflect an easier log-in process. Navigators also wanted training to role-play using the app. After participating in a role play using the app, navigators reported significantly more satisfaction with the app and greater usefulness (P<.001).

CONCLUSIONS

Our work sheds light on the importance of eliciting feedback from end users, especially users who are often overlooked by the research community and app developers. Further, it is important to elicit feedback in multiple ways to improve the app.

Employing user-centered design to develop a remote technology kit for studying young children's social communication skills

OBJECTIVE

The COVID-19 pandemic required behavioral researchers to rapidly pivot to the implementation of remote study protocols to facilitate data collection. Remote implementation required robust and flexible research protocols including reliable audio/visual technology that met all the quality, security, and privacy hallmarks of lab-based equipment, while also being portable and usable by nontechnical staff and participants. The project's primary purpose was to develop a technology kit that could be deployed for data collection in homes with young children. The secondary objective was to determine the feasibility of the kit for use longitudinally across four disparate sites.

METHOD

User-centered design principles were employed in the development and implementation of a technology kit deployed across urban, suburban, and rural participant locations in four states. Preliminary feasibility and usability data were gathered to determine the reliability of the kit across three timepoints.

RESULTS

In study 1, a technology kit was constructed addressing all project needs including the provision of the internet to connect remotely with participants. Staff training protocols and participant-facing materials were developed to accompany deployment procedures. In study 2, data gathered in technology logs demonstrated successful capturing of video footage in 96% of opportunities with most technology challenges mitigated. Subsequent behavioral coding indicated 100% of captured assessment footage has been successfully coded to date. Moreover, participants needed less support for technology setup at their later timepoints, and staff rated the kit as highly usable.

CONCLUSION

This study offers a model for future development of technology use in remote community- and home-based pediatric research.

Centering Communities in Global Health: Using Human-Centered Design to Facilitate Collaboration and Intervention Development

BACKGROUND

Utilizing iterative and collaborative tools, Human-centered Design (HCD) facilitates the creation of tailored solutions for multifaceted issues by fostering empathy and a deep understanding of human behaviors. This paper presents insights gleaned from employing HCD tools to center communities in global health intervention development.

PURPOSE

The study team collaborated with community members in Dailekh, Nepal to co-design interventions to address harms associated with menstrual seclusion, known as chhaupadi.

RESEARCH DESIGN AND STUDY SAMPLE

A Community Design Team, comprising 10 women representing various castes and ages convened for a four-day intervention co-design workshop in the community. A Community Validation Team, comprising 12 individuals from diverse occupational and caste backgrounds provided feedback on the interventions. Additionally, six village leaders participated in Key Informant Interviews to garner additional insights.

DATA COLLECTION

In the study's initial "discovery" phase, the Community Design Team employed HCD tools to generate a nuanced understanding of the context, stakeholders, and community experiences. Subsequently, in the second "design" phase, the Community Design Team crafted interventions to address harms associated with chhaupadi.

RESULTS

Invaluable lessons gained from this study underscore the necessity of crafting contextually suitable tools, checklists, and prompts for participants, allocating sufficient staff, time, and resources, and adapting to participants' literacy levels and engagement preferences, whether through group or individual activities.

CONCLUSIONS

Reflecting on these insights, our experience suggests HCD offers promising tools to authentically and equitably involve participants with diverse backgrounds in articulating their own ideas for community-based solutions in Nepal. Health practitioners, researchers, and intervention development experts are encouraged to consider adopting HCD methodologies to prioritize community voices in devising solutions for complex health challenges.

Applying the Cognitive Walkthrough for Implementation Strategies Methodology to Inform the Redesign of a Selection-Quality Implementation Toolkit for Use in Schools

Background

Implementation strategies are key to enhancing translation of new innovations but there is a need to systematically design and tailor strategies to match the targeted implementation context and address determinants. There are increasing methods to inform the redesign and tailoring of implementation strategies to maximize their usability, feasibility, and appropriateness in new settings such as the Cognitive Walkthrough for Implementation Strategies (CWIS) approach. The aim of the current project is to apply the CWIS approach to inform the redesign of a multifaceted selection-quality implementation toolkit entitled ACT SMARTS for use in middle and high schools.

Methods

We systematically applied CWIS as the second part of a community-partnered iterative redesign of ACT SMARTS for schools to evaluate the usability and inform further toolkit redesign areas. We conducted three CWIS user testing sessions with key end users of school district administrators, school principals, and educators.

Results

Our CWIS application revealed that end users found ACT SMARTS acceptable and relevant but anticipate usability issues engaging in the ACT SMARTS process. Results informed the identification of eleven usability issues and corresponding redesign solutions to enhance the usability of ACT SMARTS for use in middle and high schools.

Conclusions

Results indicated the utility of CWIS in assessing implementation strategy usability in service of informing strategy tailoring and redesign to improve alignment with user and setting needs. Recommendations regarding the use of this participatory approach are discussed.

Applying a User-Centered Design Framework to Develop a Remote Research Assessment Protocol for a Randomized Clinical Trial for Toddlers with Early Autism Characteristics

COVID-19 required many research teams to shift from in-person to remote assessments, which posed both procedural and theoretical challenges. While research has explored the utility of remote assessments for autism diagnosis from the perspective of families and clinicians, less is known about their application in clinical trials. This paper describes the development of a remote research assessment protocol for a randomized clinical trial focusing on the implementation of reciprocal imitation teaching (RIT) with toddlers in Part C early intervention. This project spans two phases. For Phase 1, our team developed and documented a series of steps utilizing user-centered design (UCD) strategies (e.g., recruiting potential users, creating a prototype, engaging in iterative development) for the purpose of redesigning an assessment protocol for a remote environment. For Phase 2, we examined preliminary outcomes of the redesign process. Primary end users (assessors) rated post-redesign usability and acceptability, while acceptability was examined using attrition data from secondary end users (family participants). Preliminary fidelity of implementation was also examined. The iterative redesign process allowed the research team to refine aspects of the assessment that ultimately led to promising preliminary ratings of usability, acceptability, and feasibility, as well as high fidelity. Preliminary data suggest that the redesigned assessment appears to be an acceptable, feasible, and usable tool for autism clinical trial research and that assessors can use it with fidelity. Further research is needed to examine the reliability and validity of the assessment, as well as implementation characteristics on a larger scale.

Harm Reduction in the Field: First Responders' Perceptions of Opioid Overdose Interventions

Introduction

Recent policy changes in Washington State presented a unique opportunity to pair evidence-based interventions with first responder services to combat increasing opioid overdoses. However, little is known about how these interventions should be implemented. In partnership with the Research with Expert Advisors on Drug Use team, a group of academically trained and community-trained researchers with lived and living experience of substance use, we examined facilitators and barriers to adopting leave-behind naloxone, field-based buprenorphine initiation, and HIV and hepatitis C virus (HCV) testing for first responder programs.

Methods

Our team completed semi-structured, qualitative interviews with 32 first responders, mobile integrated health staff, and emergency medical services (EMS) leaders in King County, Washington, from February-May 2022. Semi-structured interviews were recorded, transcribed, and coded using an integrated deductive and inductive thematic analysis approach grounded in community-engaged research principles. We collected data until saturation was achieved. Data collection and analysis were informed by the Consolidated Framework for Implementation Research. Two investigators coded independently until 100% consensus was reached.

Results

Our thematic analysis revealed several perceived facilitators (ie, tension for change, relative advantage, and compatibility) and barriers (ie, limited adaptability, lack of evidence strength and quality, and prohibitive cost) to the adoption of these evidence-based clinical interventions for first responder systems. There was widespread support for the distribution of leave-behind naloxone, although funding was identified as a barrier. Many believed field-based initiation of buprenorphine treatment could provide a more effective response to overdose management, but there were significant concerns that this intervention could run counter to the rapid care model. Lastly, participants worried that HIV and HCV testing was inappropriate for first responders to conduct but recommended that this service be provided by mobile integrated health staff.

Conclusion

These results have informed local EMS strategic planning, which will inform roll out of process improvements in King County, Washington. Future work should evaluate the impact of these interventions on the health of overdose survivors.

Integrating a mental health intervention into PrEP services for South African young women: a human-centred implementation research approach to intervention development

INTRODUCTION

Adolescent girls and young women (AGYW) who may benefit from HIV pre-exposure prophylaxis (PrEP) face high levels of common mental disorders (e.g. depression, anxiety). Common mental disorders can reduce PrEP adherence and increase HIV risk, yet mental health interventions have not been well-integrated into PrEP delivery.

METHODS

We conducted a four-phase human-centred design process, from December 2020 to April 2022, to understand mental health challenges among AGYW in Johannesburg, South Africa and barriers to integrated mental health and PrEP services. In the "Discover" phase, we conducted in-depth interviews with AGYW and key informants (KIs) in Johannesburg. We conducted a rapid qualitative analysis, informed by the Consolidated Framework for Implementation Research (CFIR), to identify facilitators and barriers of integrated mental health and PrEP services and mapped barriers to potential implementation strategies. In the "Design" and "Build" phases, we conducted stakeholder workshops to iteratively adapt an evidence-based mental health intervention, the Friendship Bench, and refine implementation strategies for South African PrEP delivery settings. In the "Test" phase, we piloted our adapted Friendship Bench package.

RESULTS

Interviews with 70 Discover phase participants (48 AGYW, 22 KIs) revealed the importance of integrated mental health and PrEP services for South African AGYW. Interviewees described barriers and implementation strategies for mental health and PrEP services around the CFIR domains: intervention characteristics (e.g. challenges with AGYW "opening up"); outer Johannesburg setting (e.g. community stigma); inner clinic setting (e.g. judgemental healthcare providers); characteristics of counsellors (e.g. training gaps); and the implementation process (e.g. need for demand creation). The Design and Build workshops included 13 AGYW and 15 KIs. Implementation barriers related to the quality and accessibility of public-sector clinic services, lay counsellor training, and community education and demand creation activities were prioritized. This led to 12 key Friendship Bench adaptations and the specification of 10 implementation strategies that were acceptable and feasible in initial pilot testing with three AGYW.

CONCLUSIONS

Using a human-centred approach, we identified determinants and potential solutions for integrating mental health interventions within PrEP services for South African AGYW. This design process centred stakeholders' perspectives, enabling rapid development of an adapted Friendship Bench intervention implementation package.

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): a modified Delphi study

BACKGROUND

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

METHOD

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability and inclusion (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

RESULTS

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

DISCUSSION

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Codesigning Online Continuing Medical Education on Social Health Integration and Social Risk-Informed Care for Primary Care Providers

INTRODUCTION

Adapting clinical care decisions for patient-reported social risks is essential to social health integration and patient-centered care. Most research in this area focuses on awareness and assistance (social-needs-targeted care), such as screening and referral to food, financial, and other resources. Limited evidence for adjustment strategies (social risk-informed care) or adapting care for social risks made it difficult for Kaiser Permanente to implement new initiatives. This article describes a codesign process to build a novel, adjustment-focused continuing medical education course.

METHODS

The authors codeveloped the online continuing medical education course with patients and clinicians using user-centered design. Transcripts from codesign activities were coded and analyzed by thematic analysis to identify major themes, including perceptions of social risk-informed care and barriers to care adjustment.

RESULTS

Practical hurdles for implementing social risk-informed care emerged, including clinicians' concerns about the ethics of adjustment as substandard care, particularly without robust assistance activities. However, patients expressed a desire for their care to be adapted to their social circumstances, to allow for more realistic care plans.

DISCUSSION

Implementation barriers identified from the codesign were addressed through an interactive, case-study approach. Existing evidence on contextualized care and shared decision making informed a general framework for primary care providers to engage in awareness and adjustment activities, paired with 3 interactive case studies based on real-world, clinician-supplied scenarios.

CONCLUSION

The authors recommend that multiple stakeholder perspectives be incorporated during the development of social health integration initiatives, particularly adjustment. Education complemented by active, nuanced, flexible implementation strategies may be necessary for the successful uptake of care-delivery-based social health integration activities.

Conducting Research at the Intersection of HCI and Health: Building and Supporting Teams with Diverse Expertise to Increase Public Health Impact

Research at the intersection of human-computer interaction (HCI) and health is increasingly done by collaborative cross-disciplinary teams. The need for cross-disciplinary teams arises from the interdisciplinary nature of the work itself-with the need for expertise in a health discipline, experimental design, statistics, and computer science, in addition to HCI. This work can also increase innovation, transfer of knowledge across fields, and have a higher impact on communities. To succeed at a collaborative project, researchers must effectively form and maintain a team that has the right expertise, integrate research perspectives and work practices, align individual and team goals, and secure funding to support the research. However, successfully operating as a team has been challenging for HCI researchers, and can be limited due to a lack of training, shared vocabularies, lack of institutional incentives, support from funding agencies, and more; which significantly inhibits their impact. This workshop aims to draw on the wealth of individual experiences in health project team collaboration across the CHI community and beyond. By bringing together different stakeholders involved in HCI health research, together, we will identify needs experienced during interdisciplinary HCI and health collaborations. We will identify existing practices and success stories for supporting team collaboration and increasing HCI capacity in health research. We aim for participants to leave our workshop with a toolbox of methods to tackle future team challenges, a community of peers who can strive for more effective teamwork, and feeling positioned to make the health impact they wish to see through their work.

Designing a Novel Digitally Delivered Antiracism Intervention for Mental Health Clinicians: Exploratory Analysis of Acceptability

BACKGROUND

There is a great need for evidence-based antiracism interventions targeting mental health clinicians to help mitigate mental health disparities in racially and ethnically minoritized groups.

OBJECTIVE

This study provides an exploratory analysis of mental health clinicians' perspectives on the acceptability of a web-based antiracism intervention.

METHODS

Mental health clinicians were recruited from a single academic medical center through outreach emails. Data were collected through individual 30-minute semistructured remote video interviews with participants, then recorded, transcribed, and analyzed using content analysis.

RESULTS

A total of 12 mental health clinicians completed the study; 10 out of 12 (83%) were female candidates. Over half (7/12, 58%) of the respondents desired more robust antiracism training in mental health care. Regarding the web-based antiracism intervention, (8/12, 67%) enjoyed the digitally delivered demo module, (7/12, 58%) of respondents suggested web-based content would be further enhanced with the addition of in-person or online group components.

CONCLUSIONS

Our results suggest a strong need for additional antiracist training for mental health clinicians. Overall, participants responded favorably to novel web-based delivery methods for an antiracism intervention. These findings provide important support for future development and pilot testing of a large-scale digitally enhanced antiracist curriculum targeting mental health clinicians.

A review of implementation and evaluation frameworks for public health interventions to inform co-creation: a Health CASCADE study

BACKGROUND

By including the needs and perspectives of relevant stakeholders, co-creation is seen as a promising approach for tackling complex public health problems. However, recommendations and guidance on how to plan and implement co-creation are lacking. By identifying and analysing existing implementation and evaluation frameworks for public health, this study aims to offer key recommendations for professional stakeholders and researchers wanting to adopt a co-creation approach to public health interventions.

METHODS

Firstly, PubMed and CINAHL databases were screened for articles introducing original implementation and evaluation frameworks for public health interventions. Backwards snowballing techniques were applied to the included papers. Secondly, identified frameworks were classified and relevant data extracted, including steps and constructs present in the frameworks. Lastly, recommendations were derived by conducting thematic analysis on the included frameworks.

RESULTS

Thirty frameworks were identified and data related to their nature and scope extracted. The frameworks' prominent steps and constructs were also retrieved. Recommendations related to implementation and evaluation in the context of co-creation were included.

CONCLUSION

When engaging in co-creation, we recommend including implementation considerations from an early stage and suggest adopting a systems thinking as a way to explore multiple levels of influence, contextual settings and systems from an early planning stage. We highlight the importance of partnering with stakeholders and suggest applying an evaluation design that is iterative and cyclical, which pays particular attention to the experience of the engaged co-creators.

Formative Evaluation of a Comprehensive Self-Management Intervention for Irritable Bowel Syndrome, Comorbid Anxiety, and Depression: Mixed Methods Study

BACKGROUND

Irritable bowel syndrome (IBS) is a disorder of the gut-brain interaction that is associated with abdominal pain, altered bowel patterns, and reduced quality of life. Up to 50% of patients with IBS also report anxiety or depressive symptoms. Although effective self-management interventions exist for individuals with IBS, few have been effectively implemented, and most do not consider the unique needs of patients with comorbid IBS and anxiety or depression.

OBJECTIVE

This study aimed to determine the anticipated acceptability, appropriateness, feasibility, and usability of a comprehensive self-management intervention using an implementation science and human-centered design approach among individuals with comorbid IBS and anxiety or depression and health care providers.

METHODS

A convergent mixed methods design was used to elicit feedback on the comprehensive self-management intervention outline and content to identify refinement needs before testing. Patients with IBS and moderate to severe anxiety or depression and health care providers were purposefully sampled from primary care and gastroenterology settings. Participants completed semistructured interviews and surveys on anticipated acceptability, appropriateness, feasibility, and usability.

RESULTS

Patient participants (n=12) were on average 36.8 (SD 12.2) years of age, and 42% (5/12) were currently receiving psychological therapy. Health care providers (n=14) were from primary care (n=7) and gastroenterology (n=7) settings. The mean usability scores (out of 100) were 52.5 (SD 14.5) for patients and 45.6 (SD 11.6) for providers. For patients and providers, qualitative data expanded the quantitative findings for acceptability and appropriateness. Acceptability findings were the comprehensive nature of the intervention and discussion of the gut-brain interaction. For appropriateness, participants reported that the intervention provided structure, accountability, and support. Feasibility was confirmed for patients, but there was a divergence of findings between quantitative and qualitative measures for providers. Patients focused on intervention feasibility, while providers focused on implementation feasibility in the clinic. Identified usability issues to address before implementation included the intervention delivery format, length, and lack of integration into health care settings that, if not addressed, may limit the reach of the intervention.

CONCLUSIONS

Patients and health care providers found the intervention acceptable and appropriate. Several feasibility and usability issues were identified, including intervention delivery methods, length of intervention, and the best methods to implement in the clinic setting. The next steps are to refine the intervention to address the identified issues and test in a pilot study whether addressing usability issues leads to the anticipated improvements in implementation and uptake.

Refining Cultural Adaptations of a Behavioral Intervention for Latino Caregivers of People Living With Dementia: Qualitative Interview Study in Washington State

BACKGROUND

 In the United States, Latino caregivers of individuals with dementia face unique challenges and an elevated risk of adverse health outcomes. Despite the increasing prevalence of Alzheimer disease and related dementias among Latino adults, few evidence-based interventions are tailored to their cultural context. To address this gap, we examined the cultural adaptations required for the STAR caregivers (STAR-C) virtual intervention, an evidence-based intervention that educates family caregivers to manage behavioral and psychological symptoms of dementia. While STAR-C has shown effectiveness, neither the original in-person nor the virtual intervention considered the distinct experiences of Latino caregivers, who often bring culturally significant values into caregiving interactions.

OBJECTIVE

This study's objective was to test and refine the preliminary cultural adaptations of the STAR-C web-based training modules for Latino caregivers of people living with dementia.

METHODS

Through qualitative interviews with 15 Latino caregivers in Washington State, we identified key adaptations to enhance the cultural relevance of the web-based training modules.

RESULTS

The interviews highlighted 4 main themes for adaptation: the delivery of the STAR-C web-based training modules, comprehensive dementia education, simplified problem-solving strategies, and prioritizing caregiver well-being.

CONCLUSIONS

This study's findings informed the development of culturally adapted STAR-C web-based training modules that aim to provide tailored support to Latino caregivers. While further research is needed to assess the efficacy of these adaptations, our work contributes to bridging the gap in dementia caregiving for Latino families, potentially reducing health disparities and enhancing health care services for this population.

Digital and precision clinical trials: innovations for testing mental health medications, devices, and psychosocial treatments

Mental health treatment advances - including neuropsychiatric medications and devices, psychotherapies, and cognitive treatments - lag behind other fields of clinical medicine such as cardiovascular care. One reason for this gap is the traditional techniques used in mental health clinical trials, which slow the pace of progress, produce inequities in care, and undermine precision medicine goals. Newer techniques and methodologies, which we term digital and precision trials, offer solutions. These techniques consist of (1) decentralized (i.e., fully-remote) trials which improve the speed and quality of clinical trials and increase equity of access to research, (2) precision measurement which improves success rate and is essential for precision medicine, and (3) digital interventions, which offer increased reach of, and equity of access to, evidence-based treatments. These techniques and their rationales are described in detail, along with challenges and solutions for their utilization. We conclude with a vignette of a depression clinical trial using these techniques.

Helping Educational Leaders Mobilize Evidence (HELM): The iterative redesign of the Leadership and Organizational Change for Implementation (LOCI) intervention for use in schools

Background

Few "intervention agnostic" strategies have been developed that can be applied to the broad array of evidence-based practices (EBPs) in schools. This paper describes two studies that reflect the initial iterative redesign phases of an effective leadership-focused implementation strategy-Leadership and Organizational Change for Implementation (LOCI)-to ensure its acceptability, feasibility, contextual appropriateness, and usability when used in elementary schools. Our redesigned strategy-Helping Educational Leaders Mobilize Evidence (HELM)-is designed to improve principals' use of strategic implementation leadership to support the adoption and high-fidelity delivery of a universal EBP to improve student outcomes.

Method

In Study 1, focus groups were conducted (n = 6) with 54 district administrators, principals, and teachers. Stakeholders provided input on the appropriateness of original LOCI components to maximize relevance and utility in schools. Transcripts were coded using conventional content analysis. Key themes referencing low appropriateness were summarized to inform LOCI adaptations. We then held a National Expert Summit (Study 2) with 15 research and practice experts. Participants provided feedback via a nominal group process (NGP; n = 6 groups) and hackathon (n = 4 groups). The research team rated each NGP suggestion for how actionable, impactful/effective, and feasible it was. We also coded hackathon notes for novel ideas or alignment with LOCI components.

Results

Study 1 suggestions included modifications to LOCI content and delivery. Study 2's NGP results revealed most recommendations to be actionable, impactful/effective, and feasible. Hackathon results surfaced two novel ideas (distributed leadership teams and leaders' knowledge to support educators EBP use) and several areas of alignment with LOCI components.

Conclusion

Use of these iterative methods informed the redesign of LOCI and the development of HELM. Because it was collaboratively constructed, HELM has the potential to be an effective implementation strategy to support the use of universal EBP in schools.

AI Conversational Agent to Improve Varenicline Adherence: Protocol for a Mixed Methods Feasibility Study

BACKGROUND

Varenicline is a pharmacological intervention for tobacco dependence that is safe and effective in facilitating smoking cessation. Enhanced adherence to varenicline augments the probability of prolonged smoking abstinence. However, research has shown that one-third of people who use varenicline are nonadherent by the second week. There is evidence showing that behavioral support helps with medication adherence. We have designed an artificial intelligence (AI) conversational agent or health bot, called "ChatV," based on evidence of what works as well as what varenicline is, that can provide these supports. ChatV is an evidence-based, patient- and health care provider-informed health bot to improve adherence to varenicline. ChatV has been programmed to provide medication reminders, answer questions about varenicline and smoking cessation, and track medication intake and the number of cigarettes.

OBJECTIVE

This study aims to explore the feasibility of the ChatV health bot, to examine if it is used as intended, and to determine the appropriateness of proceeding with a randomized controlled trial.

METHODS

We will conduct a mixed methods feasibility study where we will pilot-test ChatV with 40 participants. Participants will be provided with a standard 12-week varenicline regimen and access to ChatV. Passive data collection will include adoption measures (how often participants use the chatbot, what features they used, when did they use it, etc). In addition, participants will complete questionnaires (at 1, 4, 8, and 12 weeks) assessing self-reported smoking status and varenicline adherence, as well as questions regarding the acceptability, appropriateness, and usability of the chatbot, and participate in an interview assessing acceptability, appropriateness, fidelity, and adoption. We will use "stop, amend, and go" progression criteria for pilot studies to decide if a randomized controlled trial is a reasonable next step and what modifications are required. A health equity lens will be adopted during participant recruitment and data analysis to understand and address the differences in uptake and use of this digital health solution among diverse sociodemographic groups. The taxonomy of implementation outcomes will be used to assess feasibility, that is, acceptability, appropriateness, fidelity, adoption, and usability. In addition, medication adherence and smoking cessation will be measured to assess the preliminary treatment effect. Interview data will be analyzed using the framework analysis method.

RESULTS

Participant enrollment for the study will begin in January 2024.

CONCLUSIONS

By using predetermined progression criteria, the results of this preliminary study will inform the determination of whether to advance toward a larger randomized controlled trial to test the effectiveness of the health bot. Additionally, this study will explore the acceptability, appropriateness, fidelity, adoption, and usability of the health bot. These insights will be instrumental in refining the intervention and the health bot.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05997901; https://classic.clinicaltrials.gov/ct2/show/NCT05997901.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/53556.

A chatbot-delivered intervention for optimizing social media use and reducing perceived isolation among rural-living LGBTQ+ youth: Development, acceptability, usability, satisfaction, and utility

Background

Lesbian, gay, bisexual, transgender, and queer/questioning (LGBTQ+) youth are at higher risk of isolation and depression than their heterosexual peers. Having access to tailored mental health resources is a documented concern for rural living LGBTQ+ youth. Social media provides access to connections to a broader and like-minded community of peers, but it also is a vehicle for negative interactions. We developed REALbot, an automated, social media-based educational intervention to improve social media efficacy, reduce perceived isolation, and bolster connections for rural living LGBTQ+ youth. This report presents data on the acceptability, feasibility, and utility of REALbot among its target audience of rural living LGBTQ+ youth.

Methods

We conducted a week-long exploratory study with a single non-comparison group of 20 rural-living LGBTQ+ youth aged 14-19 recruited from social media to test our Facebook- and Instagram-delivered chatbot. We assessed pre- and post-test scores of social media self-efficacy, social isolation (4-item Patient-Reported Outcomes Measurement System - PROMIS), and depressive symptoms (Patient Health Questionnaire, Adolescent Version - PHQ-A). At post-test, we assessed acceptability (User Experience Questionnaire - UEQ-S), usability (Chatbot Usability Questionnaire -CUQ and Post-Study Satisfaction and Usability Questionnaire -PSSUQ), and satisfaction with the chatbot (Client Satisfaction Questionnaire - CSQ), along with two open-ended questions on 'likes' and 'dislikes' about the intervention. We compared pre- and post-test scores with standard univariate statistics. Means and standard deviations were calculated for usability, acceptability, and satisfaction. To analyze the responses to post-test open-end questions, we used a content analysis approach.

Results

Acceptability of REALbot was high with UEQ-S 5.3 out of 7 (SD = 1.1) and received high usability scores with CUQ and PSSUQ (mean score (M) = 78.0, SD = 14.5 and M = 86.9, SD = 25.2, respectively), as well as high user satisfaction with CSQ (M = 24.9, SD = 5.4). Themes related to user 'likes' and 'dislikes' were organized in two main categories: usability and content provided. Participants were engaged with the chatbot, sending an average of 49.3 messages (SD = 43.6, median = 30). Pre-/post- changes in scores of perceived isolation, depressive symptoms and social media self-efficacy were not significant (p's > 0.08).

Conclusion

REALbot deployment was found to be feasible and acceptable, with good usability and user satisfaction scores. Participants reported changes from pre- to post-test in most outcomes of interest and effect sizes were small to medium. Additional development and a formal evaluation of feasibility and engagement with behavioral targets is warranted.

Development and codesign of flourish: A digital suicide prevention intervention for LGBTQ+ youth who have experienced online victimization

Background

LGBTQ+ youth experience disproportionately high rates of online victimization (OV), referring to harmful remarks, images, or behaviors in online settings, which is associated with suicidal risk. Current services have gaps in supporting LGBTQ+ youth facing OV events. To address these gaps, this study aims to develop Flourish, a digital suicide prevention intervention for LGBTQ+ youth who have experienced OV.

Methods

Qualitative interviews were conducted with 20 LGBTQ+ youth with past-year history of OV and lifetime history of suicidality, 11 of their parents, and 10 LGBTQ+-serving professionals. Subsequently, an iterative codesign process was conducted with 22 youth through individual and group design sessions, followed by usability testing. Data were recorded and transcribed. Qualitative interviews were analyzed using a qualitative description approach, and data from design sessions and usability testing were analyzed using rapid qualitative techniques.

Results

Interviews with youth, parents, and professionals suggested preferences for Flourish to be a partially automated, text message intervention leveraging web-based content that is a safe space for LGBTQ+ youth to seek support for OV through education, coping skills, and help-seeking resources. School and mental health services professionals considered the potential for implementing Flourish within youth services settings. Usability testing, assessed through the System Usability Scale, yielded an average rating of 91, indicating excellent perceived usability.

Conclusions

Flourish has potential to be an acceptable intervention to support LGBTQ+ youth following OV. Future steps will include testing the feasibility and efficacy of Flourish and further examining Flourish's potential for implementation within services for LGBTQ+ youth.

Development, Usability Testing, and Implementation Assessment of Cancer Related Infertility Score Predictor, an Online Cancer Related Infertility Risk Counseling Tool

Purpose: Oncofertility counseling of female cancer patients lacks efficient access to tailored and valid infertility risk estimates to support shared decision-making on fertility preservation treatments. The objective was to develop, conduct user-centered design, and plan clinic-based implementation of the Cancer Related Infertility Score Predictor (CRISP), a web-based tool to support infertility risk counseling. Methods: Using a mixed methods design, literature review was undertaken to abstract data on infertility, primary ovarian insufficiency, and amenorrhea risks of common cancer treatments. The CRISP website was programmed to take user input about patient ages and cancer treatments and generate a risk summary. Using user experience methodology and semistructured interviews, usability testing and implementation assessment were conducted with 12 providers recruited from 5 medical centers in Southern California. Results: The web-based CRISP tool encompasses infertility risk data for 60 treatment regimens among 10 cancer types. Usability testing demonstrated that the tool is intuitive and informed minor modifications, including adding crowd-sourced submission of additional cancer treatments. Participants rated the tool as credible, advantageous over current provider methods to ascertain infertility risks, and useful for tailoring treatment planning and counseling patients. A key barrier was lack of information on some cancer treatments. Fit within clinical workflow was feasible, particularly with electronic health record integration. Conclusions: The novel, web-based CRISP tool is a feasible, acceptable, and appropriate tool to address provider knowledge gap about cancer related infertility risks and use for patient counseling. CRISP has significant potential to support tailored oncofertility counseling in the heterogeneous young cancer patient population.

Evaluation of a Novel Question Prompt List in Pediatric Surgical Oncology

INTRODUCTION

Parents of children with cancer describe interactions with clinicians as emotionally distressing. Patient engagement in treatment discussions decreases decisional conflict and improves decision quality which may limit such distress. We have shown that parents prefer to engage surgeons by asking questions, but parents may not know what to ask. Question Prompt Lists (QPLs), structured lists of questions designed to help patients ask important questions, have not been studied in pediatric surgery. We developed a QPL designed to empower parents to ask meaningful questions during pediatric surgical oncology discussions. We conducted a mixed methods analysis to assess the acceptability, appropriateness, and feasibility of using the QPL.

METHODS

Key stakeholders at an academic children's hospital participated in focus groups to discuss the QPL. Focus groups were recorded and transcribed. Participants were surveyed regarding QPL acceptability, appropriateness, and feasibility. Thematic content analysis of transcripts was performed.

RESULTS

Four parents, five nurses, five nurse practitioners, five oncologists, and four surgeons participated. Seven key themes were identified: (1) QPL as a tool of empowerment; (2) stick to the surgical details; (3) QPLs can impact discussion quality; (4) time consuming, but not overly disruptive; (5) parental emotion may impact QPL use; (6) provide QPLs prior to surgical consultation in both print and digital formats; and (7) expansion of QPLs to other disciplines. Over 70% of participants agreed that the QPL was acceptable, appropriate, and feasible.

CONCLUSIONS

Our novel QPL is acceptable, appropriate, and feasible to use with parents of pediatric surgical oncology patients.

Improving the Usability of Written Exposure Therapy for Therapists in the Department of Veterans Affairs Telemental Health: Formative Study Using Qualitative and User-Centered Design Methods

BACKGROUND

User modifications are common in evidence-based psychosocial interventions (EBPIs) for mental health disorders. Often, EBPIs fit poorly into clinical workflows, require extensive resources, or pose considerable burden to patients and therapists. Implementation science is increasingly researching ways to improve the usability of EBPIs before implementation. A user-centered design can be used to support implementation methods to prioritize user needs and solutions to improve EBPI usability.

OBJECTIVE

Trauma-focused EBPIs are a first-line treatment for patients with posttraumatic stress disorder (PTSD) in the Department of Veterans Affairs. Written exposure therapy (WET) is a brief, trauma-focused EBPI wherein patients handwrite about trauma associated with their PTSD. Initially developed for in-person delivery, WET is increasingly being delivered remotely, and outcomes appear to be equivalent to in-person delivery. However, there are logistical issues in delivering WET via video. In this evaluation, we explored usability issues related to WET telehealth delivery via videoconferencing software and designed a solution for therapist-facing challenges to systematize WET telehealth delivery.

METHODS

The Discover, Design and Build, and Test framework guided this formative evaluation and served to inform a larger Virtual Care Quality Enhancement Research Initiative. We used qualitative descriptive methods in the Discover phase to understand the experiences and needs of 2 groups of users providing care within the Department of Veterans Affairs: in-person therapists delivering WET via video because of the COVID-19 pandemic and telehealth therapists who regularly deliver PTSD therapies. We then used user-centered design methods in the Design and Build phase to brainstorm, develop, and iteratively refine potential workflows to address identified usability issues. All procedures were conducted remotely.

RESULTS

In the Discover phase, both groups had challenges delivering WET and other PTSD therapies via telehealth because of technology issues with videoconferencing software, environmental distractions, and workflow disruptions. Narrative transfer (ie, patients sending handwritten trauma accounts to therapists) was the first target for design solution development as it was deemed most critical to WET delivery. In the Design and Build phase, we identified design constraints and brainstormed solution ideas. This led to the development of 3 solution workflows that were presented to a subgroup of therapist users through cognitive walkthroughs. Meetings with this subgroup allowed workflow refinement to improve narrative transfers. Finally, to facilitate using these workflows, we developed PDF manuals that are being refined in subsequent phases of the implementation project (not mentioned in this paper).

CONCLUSIONS

The Discover, Design and Build, and Test framework can be a useful tool for understanding user needs in complex EBPI interventions and designing solutions to user-identified usability issues. Building on this work, an iterative evaluation of the 3 solution workflows and accompanying manuals with therapists and patients is underway as part of a nationwide WET implementation in telehealth settings.

Development of a method for Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI): A modified Delphi study

Background

Intervention adaptation is often necessary to improve the fit between evidence-based practices/programs and implementation contexts. Existing frameworks describe intervention adaptation processes but do not provide detailed steps for prospectively designing adaptations, are designed for researchers, and require substantial time and resources to complete. A pragmatic approach to guide implementers through developing and assessing adaptations in local contexts is needed. The goal of this project was to develop Making Optimal Decisions for Intervention Flexibility during Implementation (MODIFI), a method for intervention adaptation that leverages human centered design methods and is tailored to the needs of intervention implementers working in applied settings with limited time and resources.

Method

MODIFI was iteratively developed via a mixed-methods modified Delphi process. Feedback was collected from 43 implementation research and practice experts. Two rounds of data collection gathered quantitative ratings of acceptability (Round 1) and feasibility (Round 2), as well as qualitative feedback regarding MODIFI revisions analyzed using conventional content analysis.

Results

In Round 1, most participants rated all proposed components as essential but identified important avenues for revision which were incorporated into MODIFI prior to Round 2. Round 2 emphasized feasibility, where ratings were generally high and fewer substantive revisions were recommended. Round 2 changes largely surrounded operationalization of terms/processes and sequencing of content. Results include a detailed presentation of the final version of the three-step MODIFI method (Step 1: Learn about the users, local context, and intervention; Step 2: Adapt the intervention; Step 3: Evaluate the adaptation) along with a case example of its application.

Discussion

MODIFI is a pragmatic method that was developed to extend the contributions of other research-based adaptation theories, models, and frameworks while integrating methods that are tailored to the needs of intervention implementers. Guiding teams to tailor evidence-based interventions to their local context may extend for whom, where, and under what conditions an intervention can be effective.

Developing the Speaking Out and Allying Relationships Intervention on Videoconference for HIV-Positive GBMSM in Eastern Cape, South Africa

In South Africa, HIV prevalence for gay, bisexual, and men who have sex with men (GBMSM) is as high as 49.5%, yet only 25.7% are taking treatment, resulting in transmission risk between partners and the need for dyadic interventions for men in relationships. Through our preliminary research, we identified the evidence-based intervention Healthy Relationships (HR)-an HIV risk assessment and status disclosure intervention for those in relationships-to be tailored into videoconference format for partnered HIV-positive GBMSM in South Africa. The HR adaptation, called SOAR (Speaking Out & Allying Relationships), applied a human-centered design approach. In-depth interviews were conducted with HIV-positive GBMSM (N = 15) to refine intervention preferences with results used to establish a beta SOAR. A trained interventionist conducted SOAR functionality (n = 6) and usability (n = 7) tests with separate invited groups composed of the original participants interviewed. Field logs, focus group discussions, and a study-specific preference survey were administered. Thematic analysis and descriptive statistics were completed with a convergent analytical approach used to understand usability. Overall experience of GBMSM in SOAR was good (69%) or excellent (31%). More than half of the participants (61%) rated using videoconferencing for SOAR as good, with 38% rating it as fair. All participants stated that SOAR was understandable and satisfactory with willingness to recommend it to other GBMSM. This adaptation approach was effective, and the resultant SOAR intervention has the potential to improve individual coping and HIV communication skills with partners to engage with biomedical prevention and in turn support GBMSM couples.

Mobile Technologies for Supporting Mental Health in Youths: Scoping Review of Effectiveness, Limitations, and Inclusivity

BACKGROUND

Over the past decade, there has been growing support for the use of mobile health (mHealth) technologies to improve the availability of mental health interventions. While mHealth is a promising tool for improving access to interventions, research on the effectiveness and efficacy of mHealth apps for youths is limited, particularly for underrepresented populations, including youths of color and economically marginalized youths.

OBJECTIVE

This scoping review study sought to evaluate the following research questions: (1) What is the extent of the current literature on mHealth apps that provide intervention for mental health problems in children and adolescents? (2) What is known from the existing literature about the effectiveness or efficacy of delivering mental health services via mHealth apps? (3) What are the gaps in the knowledge base in the fields of technology and mental health? (4) Do the reviewed mHealth apps address issues of cultural sensitivity or have they been tested with underrepresented groups (ie, youths of color or economically marginalized groups)?

METHODS

An electronic database search was conducted using relevant search terms. Seven independent reviewers screened identified studies, including title and abstract review to determine if studies met the following inclusion criteria: (1) targeted samples with mental health symptomology or disorders, (2) studied youth participants aged 6-17 years, and (3) examined the use of a mobile app-based platform for intervention. Relevant studies were subjected to full-text review to extract and chart relevant data based on a priori research questions.

RESULTS

The initial database search yielded 304 papers published from 2010 to 2021. After screening and selection, the final review included 10 papers on the effectiveness and efficacy of mental health intervention apps for youths aged 8 to 17 years. Identified apps targeted a broad range of mental health challenges in youths (ie, depression, self-harm, autism spectrum disorder, anxiety, and obsessive-compulsive disorder). Results identified only a small number of studies suggesting that current effectiveness and efficacy research in this area are limited. While some studies provided general support for the effectiveness of mHealth apps in improving mental health outcomes in youths, several notable limitations were present across the literature, reducing the generalizability of findings. Additionally, considerations around racial, ethnic, and socioeconomic diversity were scarce across studies.

CONCLUSIONS

Although some studies cited in this scoping review provide support for the effectiveness and efficacy of mHealth apps targeting mental health concerns in youths, the overall body of literature remains quite limited. Moreover, mHealth apps expressly developed to be culturally responsive are almost nonexistent. Further efforts are needed to recruit youths who are typically underrepresented in research and invite stakeholder participation and collaborative input in the early stages of the mHealth app development process.

Increasing consumer engagement: tools to engage service users in quality improvement or implementation efforts

Introduction

Engaging service users or consumers in quality improvement or implementing a new service is important across settings and may reduce health inequities. Implementation strategies leveraging consumer engagement are neither commonly used nor robustly operationalized in implementation science. Implementers (e.g., middle managers, facilitators) want to involve consumers in implementation activities, but do not always feel confident in how to proceed. We developed a compendium of tools called Consumer Voice to guide others how to engage consumers in design/delivery of implementation strategies. Although generalizable to other settings, we developed Consumer Voice within the context of implementing suicide prevention treatments in healthcare to reach rural U.S. military veterans, as there are suicide inequities for people in rural areas.

Methods

We developed Consumer Voice using a multistep process and human-centered design methods. In between steps, a design team met to generate insights from data, and decide which prototypes to create/refine. In preliminary work, we conducted a scan of examples in healthcare of patient engagement in implementation activities and interviewed two implementation experts about preferred learning styles. In Step 1, we interviewed 26 participants with experience in community engagement, implementation, or lived experience as a rural U.S. veteran with suicidal thoughts/behavior. In Step 2, 11 implementers beta tested prototypes then share feedback in focus groups. In Step 3, we reconvened participants from prior steps to review tools and, using nominal group technique, prioritized remaining recommendations.

Results

Consumer Voice is online, modular, and nonlinear for self-guided learning tailored to beginner, intermediate, or advanced experience with consumer engagement. Tools consist of slides, audiovisual content with written text, and templates. Findings indicated there is not one "right" way to engage consumers in implementation activities, rather that implementers wanted tools showcasing core principles for consumer engagement and practical ideas.

Discussion

Consumer Voice can be used by implementers to reflect and decide on how to apply consumer engagement implementation strategies to improve equitable dissemination and uptake of evidence-based practices. Most insights generated by user data were explicitly to build trust between consumers and professionals representing institutions, which may be one component to reducing healthcare inequities.

Provider perspectives on patient-centredness: participatory formative research and rapid analysis methods to inform the design and implementation of a facility-based HIV care improvement intervention in Zambia

INTRODUCTION

Implementation of patient-centred care (PCC) practices in HIV treatment depends on healthcare workers' (HCWs) perceptions of the acceptability, appropriateness and feasibility of such practices (e.g. use of intentional, metric-driven activities to improve patient experiences).

METHODS

We applied rapid, rigorous formative research methods to refine a PCC intervention for future trial. In 2018, we conducted focus group discussions (FGDs) with 46 HCWs purposefully selected from two pilot sites. We elicited HCW perceptions of HIV service delivery, HCW motivation and perceived value of patient experience measures intended to improve PCC. FGDs utilized participatory methods to understand HCW responses to patient-reported care engagement challenges and Scholl's PCC Framework principles (e.g. seeing a patient as a unique person), enablers (e.g. care coordination) and activities (e.g. patient involvement). Our rapid analysis used analytic memos, thematic analysis, research team debriefs and HCW feedback to inform time-sensitive trial implementation.

RESULTS

While HCWs nearly universally identified with and supported principles of PCC in both facilities, they raised practical barriers given the practice environment. HCWs described motivation to help patients, attached value to seeing positive health outcomes and the importance of teamwork. However, HCWs reported challenges with enablers needed to deliver PCC. HCWs cited a work culture characterized by differential power dynamics between cadres and departments restricting HCW autonomy and resource access. Barriers included inflexibility in accommodating individual patient needs due to high patient volumes, limited human resources, laboratory capacity, infrastructure and skills translating patient perspectives into practice. HCW motivation was negatively influenced by encounters with "difficult patients," and feeling "unappreciated" by management, resulting in cognitive dissonance between HCW beliefs and behaviours. However, the enactment of PCC values also occurred. Results suggested that PCC interventions should reduce practice barriers, highlighting the value of mentors who could help HCWs dynamically engage with health system constraints, to facilitate PCC.

CONCLUSIONS

While HCWs perceived PCC principles as acceptable, they did not think it universally appropriate or feasible given the practice environment. Participatory and rapid methods provided timely insight that PCC interventions must provide clear and effective systems enabling PCC activities by measuring and mitigating relational and organizational constraints amenable to change such as inter-cadre coordination.

Bridging HCI and Implementation Science for Innovation Adoption and Public Health Impact

Human computer interaction (HCI) and implementation science (IS) each have been applied to improve the adoption and delivery of innovative health interventions, and the two fields have complementary goals, foci, and methods. While the IS community increasingly draws on methods from HCI, there are many unrealized opportunities for HCI to draw from IS and to catalyze bidirectional collaborations. This workshop will explore similarities and differences between fields, with a goal of articulating a research agenda at their intersection.

Integration of a Digital Health Intervention Into Immunization Clinic Workflows in Kenya: Qualitative, Realist Evaluation of Technology Usability

BACKGROUND

In an effort to increase vaccination coverage in low-resource settings, digital tools have been introduced to better track immunization records, improve data management practices, and provide improved access to vaccination coverage data for decision-making. Despite the potential of these electronic systems to improve the provision of health services, few digital health interventions have been institutionalized at scale in low- and middle-income countries.

OBJECTIVE

In this paper, we aimed to describe how health care workers in Kenya had integrated an electronic immunization registry into their immunization clinic workflows and to use these findings to inform the development of a refined program theory on the registry's usability.

METHODS

Informed by realist methodology, we developed a program theory to explain usability of the electronic immunization registry. We designed a qualitative study based on our theory to describe the barriers and facilitators influencing data entry and use. Qualitative data were collected through semistructured interviews with users and workflow observations of immunization clinic sessions. Our findings were summarized by context-mechanism-outcome relationships formed after analyzing our key themes across interviews and workflow observations. Using these relationships, we were able to identify common rules for future implementers.

RESULTS

Across the 12 facilities included in our study, 19 health care workers were interviewed, and 58 workflow sessions were observed. The common rules developed from our qualitative findings are as follows: rule 1-ensure that the users complete training to build familiarity with the system, understand the value of the system and data, and know where to find support; rule 2-confirm that the system captures all data needed for users to provide routine health care services and is easy to navigate; rule 3-identify work-arounds for poor network, system performance, and too few staff or resources; and rule 4-make users aware of expected changes to their workflow, and how these changes might differ over time and by facility size or number of patients. Upon study completion, we revised the program theory to reflect the importance of the goals and workflows of electronic immunization registries aligning with reality.

CONCLUSIONS

We created a deeper understanding of the underlying mechanisms for usability of the registry. We found that the electronic immunization registry had high acceptability among users; however, there were numerous barriers to using the system, even under ideal conditions, causing a misalignment between the system and the reality of the users' workflows and their environment. Human-centered design and human-factors methods can assist during pilot stages to better align systems with users' needs and again after scale-up to ensure that interventions are suitable for all user settings.

Study Protocol: Multi-level Determinants of Implementation and Sustainment in the Education Sector

Evidence-based programs (EBPs) delivered in elementary schools show great promise in reducing risk for emotional and behavioral disorders (EBDs). However, efforts to sustain EBPs in school face barriers. Improving EBP sustainment thus represents a priority, but little research exists to inform the development of sustainment strategies. To address this gap, the Sustaining Evidenced-Based Innovations through Multi-level Implementation Constructs (SEISMIC) project will: (a) Determine if malleable individual, intervention, and organizational factors predict EBP treatment fidelity and modifications during implementation, sustainment, or both; (b) Assess the impact of EBP fidelity and modifications on child outcomes during implementation and sustainment; and (c) Explore the mechanisms through which individual, intervention, and organizational factors influence sustainment outcomes. This protocol paper describes SEISMIC, which builds upon a federally-funded RCT evaluating BEST in CLASS, a teacher-delivered program for K-3rd grade children at risk for EBDs. The sample will include 96 teachers, 384 children, and 12 elementary schools. A multi-level, interrupted time series design will be used to examine the relationship between baseline factors, treatment fidelity, modifications, and child outcomes, followed by a mixed-method approach to elucidate the mechanisms that influence sustainment outcomes. Findings will be used to create a strategy to improve EBP sustainment in schools.

A machine learning approach to improve implementation monitoring of family-based preventive interventions in primary care

Background

Evidence-based parenting programs effectively prevent the onset and escalation of child and adolescent behavioral health problems. When programs have been taken to scale, declines in the quality of implementation diminish intervention effects. Gold-standard methods of implementation monitoring are cost-prohibitive and impractical in resource-scarce delivery systems. Technological developments using computational linguistics and machine learning offer an opportunity to assess fidelity in a low burden, timely, and comprehensive manner.

Methods

In this study, we test two natural language processing (NLP) methods [i.e., Term Frequency-Inverse Document Frequency (TF-IDF) and Bidirectional Encoder Representations from Transformers (BERT)] to assess the delivery of the Family Check-Up 4 Health (FCU4Health) program in a type 2 hybrid effectiveness-implementation trial conducted in primary care settings that serve primarily Latino families. We trained and evaluated models using 116 English and 81 Spanish-language transcripts from the 113 families who initiated FCU4Health services. We evaluated the concurrent validity of the TF-IDF and BERT models using observer ratings of program sessions using the COACH measure of competent adherence. Following the Implementation Cascade model, we assessed predictive validity using multiple indicators of parent engagement, which have been demonstrated to predict improvements in parenting and child outcomes.

Results

Both TF-IDF and BERT ratings were significantly associated with observer ratings and engagement outcomes. Using mean squared error, results demonstrated improvement over baseline for observer ratings from a range of 0.83-1.02 to 0.62-0.76, resulting in an average improvement of 24%. Similarly, results demonstrated improvement over baseline for parent engagement indicators from a range of 0.81-27.3 to 0.62-19.50, resulting in an approximate average improvement of 18%.

Conclusions

These results demonstrate the potential for NLP methods to assess implementation in evidence-based parenting programs delivered at scale. Future directions are presented.

Trial registration

NCT03013309 ClinicalTrials.gov.

Cocreation of a conversational agent to help patients adhere to their varenicline treatment: A study protocol

Objective

Varenicline is the most efficacious approved smoking cessation medication, making it one of the most cost-effective clinical interventions for reducing tobacco-related morbidity and mortality. Adhering to varenicline is strongly associated with smoking cessation. Healthbots have the potential to help people adhere to their medications by scaling up evidence-based behavioral interventions. In this protocol, we outline how we will follow the UK's Medical Research Council's guidance to codesign a theory-informed, evidence-based, and patient-centered healthbot to help people adhere to varenicline.

Methods

The study will utilize the Discover, Design and Build, and Test framework and will include three phases: (a) a rapid review and interviews with 20 patients and 20 healthcare providers to understand barriers and facilitators to varenicline adherence (Discover phase); (b) Wizard of Oz test to design the healthbot and get a sense of the questions that chatbot has to be able to answer (Design phase); and (c) building, training, and beta-testing the healthbot (Building and Testing phases) where the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability framework will be used to develop the healthbot using the simplest sensible solution, and 20 participants will beta test the healthbot. We will use the Capability, Opportunity, Motivation-Behavior (COM-B) model of behavior change and its associated framework, the Theoretical Domains Framework, to organize the findings.

Conclusions

The present approach will enable us to systematically identify the most appropriate features for the healthbot based on a well-established behavioral theory, the latest scientific evidence, and end users' and healthcare providers' knowledge.

The Implementation of Measurement-Based Care in the Context of Telemedicine: Qualitative Study

BACKGROUND

The Measurement Based Care in Mental Health Initiative launched by the Department of Veterans Affairs in 2016 is an example of an evidence-based practice that uses patient-reported outcome measures (PROMs) to improve patient outcomes. The acceptance of measurement-based care (MBC) among Veterans Affairs providers is relatively high. However, there are barriers to MBC for telehealth providers. Health information technologies might afford opportunities to address some of the barriers related to the uptake of MBC.

OBJECTIVE

This paper reports on an implementation effort to integrate MBC into mental health care telehealth practice using eHealth solutions.

METHODS

Qualitative data were generated from 22 semistructured interviews with psychiatrists (n=4), psychologists (n=3), social workers (n=3), nurses (n=6), a pharmacist (n=1), and administrative staff (n=5) who provide telemental health care through a community-based outpatient clinic in the rural Midwestern United States. The interviews were conducted during the pilot phase of an implementation initiative to increase the adoption of MBC by revising clinic workflows to integrate the use of eHealth technologies. Data were analyzed using thematic analysis.

RESULTS

Time burden and workflow issues were the most common barrier to provider adoption of MBC; sharing and reviewing pencil-and-paper measures and results in the same room was no longer possible in novel telehealth workflows necessitated by the COVID-19 pandemic. Providers voiced concerns about how long it would take to collect, adequately score, interpret, share, and document the PROMs during the telehealth visit. Concerns about time might also correspond to a gap in providers' familiarity with these assessments, greater comfort in assessing symptoms through clinical interviews, and being accustomed to using the assessments as screening tools more so than longitudinal outcome measures. Capacities associated with eHealth technologies may address workflow concerns and promote providers' understanding and use of the measures as tracking tools.

CONCLUSIONS

The need to use limited appointment time well was a top priority for telemental health providers. eHealth technologies provided operative supports that protect time in appointments by shifting when and how PROMs are collected. Bolstering providers' familiarity with how to use PROMs in the course of treatment may impact providers' buy-in by encouraging them to reconsider how sharing and acting on PROMs could be time well spent.

Development of a Web-Based Training Platform for School Clinicians in Evidence-Based Practices for ADHD

Lack of training for school clinicians in evidence-based practices (EBPs) contributes to underutilization of such services for youth with attention-deficit/hyperactivity disorder (ADHD). Advances in web-based technology and videoconferencing have allowed for expanded access to and optimization of training. We describe the development and outcomes of a novel web-based platform for training school clinicians to gain skills in EBPs for school-age youth with ADHD. The training platform is adapted from an empirically supported, in-person training for a school-home behavioral intervention (Collaborative Life Skills program) and includes skill modules for working with teachers, parents, and students. Training methods include web-accessed manuals/handouts, skill example video clips, automated progress monitoring tools, and consultation/in-session coaching via videoconferencing. We gathered stakeholder qualitative and quantitative feedback during discovery and design phases of the iterative development. We then evaluated the usability, acceptability, fidelity and clinician and student outcomes of the remote training program. Focus group themes and qualitative feedback identified clinician preferences for remote training features (e.g., interactive, brief, role-plays/coaching methods), video tools (recorded samples of skills and therapy sessions), and progress monitoring tools (e.g., clear, easy to use). Clinician usability ratings of the platform were high with most components rated as moderately to very useful/easy to use. Clinician ratings of usability, fidelity implementing the treatment, and their EBP knowledge and confidence following training were favorable. Student's outcomes were similar to those achieved in prior studies of clinician in-person training. Results support the promise of remote, web-based clinician training for the dissemination of evidence-based practices.

To Utility and Beyond! Specifying and Advancing the Utility of Measurement-Based Care for Youth

Mental health organizations that serve youth are under pressure to adopt measurement-based care (MBC), defined as the continuous collection of client-report data used to support clinical decision-making as part of standard care. However, few frameworks exist to help leadership ascertain how to select an MBC approach for a clinical setting. This paper seeks to define how an MBC approach can display clinical utility to provide such a framework. Broadly, we define clinical utility as evidence that an MBC approach assists stakeholders in fulfilling clinical goals related to care quality (i.e., improve client-clinician alliance and clinical outcomes) at the client (i.e., youth and caregiver), clinician, supervisor, and administrator levels. More specifically, our definition of clinical utility is divided into two categories relevant to the usability and usefulness of an MBC approach for a specific setting: (a) implementability (i.e., evidence indicating ease of use in a clinical setting) and (b) usefulness in aiding clinical activities (i.e., evidence indicating the potential to improve communication and make clinical activities related to care quality easier or more effective). These categories provide valuable information about how easy an MBC approach is to use and the potential benefits that the MBC data will confer. To detail how we arrived at this definition, we review prior definitions of clinical utility, discuss how previous definitions inform our definition of clinical utility for MBC, and provide examples of how the concept of clinical utility can be applied to MBC. We finish with a discussion of future research directions.

Development and evaluation of a remote training strategy for the implementation of mental health evidence-based practices in rural schools: pilot study protocol

Background

An increasing number of schools in rural settings are implementing multi-tier positive behavioral interventions and supports (PBIS) to address school-climate problems. PBIS can be used to provide the framework for the implementation of evidence-based practices (EBPs) to address children’s mental health concerns. Given the large service disparities for children in rural areas, offering EBPs through PBIS can improve access and lead to better long-term outcomes. A key challenge is that school personnel need technical assistance in order to implement EBPs with fidelity and clinical effectiveness. Providing ongoing on-site support is not feasible or sustainable in the majority of rural schools, due to their remote physical location. For this reason, remote training technology has been recommended for providing technical assistance to behavioral health staff (BHS) in under-served rural communities.

Objectives

The purpose of this study is to use the user-centered design, guided by an iterative process (rapid prototyping), to develop and evaluate the appropriateness, feasibility, acceptability, usability, and preliminary student outcomes of two online training strategies for the implementation of EBPs at PBIS Tier 2.

Methods

The study will employ a pragmatic design comprised of a mixed-methods approach for the development of the training platform, and a hybrid type 2, pilot randomized controlled trial to examine the implementation and student outcomes of two training strategies: Remote Video vs. Remote Video plus Coaching.

Discussion

There is a clear need for well-designed remote training studies focused on training in non-traditional settings. Given the lack of well-trained mental health professionals in rural settings and the stark disparities in access to services, the development and pilot-testing of a remote training strategy for BHS in under-served rural schools could have a significant public health impact.

Ethics and dissemination

The project was reviewed and approved by the institutional review board. Results will be submitted to ClinicalTrials.gov and disseminated to community partners and participants, peer-reviewed journals, and academic conferences.

Trial registration

ClinicialTrials.gov, NCT05034198 and NCT05039164

Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project Analysis

BACKGROUND

People often prefer evidence-based psychosocial interventions (EBPIs) for mental health care; however, these interventions frequently remain unavailable to people in nonspecialty or integrated settings, such as primary care and schools. Previous research has suggested that usability, a concept from human-centered design, could support an understanding of the barriers to and facilitators of the successful adoption of EBPIs and support the redesign of EBPIs and implementation strategies.

OBJECTIVE

This study aimed to identify and categorize usability issues in EBPIs and their implementation strategies.

METHODS

We adapted a usability issue analysis and reporting format from a human-centered design. A total of 13 projects supported by the National Institute of Mental Health-funded Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center at the University of Washington used this format to describe usability issues for EBPIs and implementation strategies with which they were working. Center researchers used iterative affinity diagramming and coding processes to identify usability issue categories. On the basis of these categories and the underlying issues, we propose heuristics for the design or redesign of EBPIs and implementation strategies.

RESULTS

The 13 projects reported a total of 90 usability issues, which we categorized into 12 categories, including complex and/or cognitively overwhelming, required time exceeding available time, incompatibility with interventionist preference or practice, incompatibility with existing workflow, insufficient customization to clients/recipients, intervention buy-in (value), interventionist buy-in (trust), overreliance on technology, requires unavailable infrastructure, inadequate scaffolding for client/recipient, inadequate training and scaffolding for interventionists, and lack of support for necessary communication. These issues range from minor inconveniences that affect a few interventionists or recipients to severe issues that prevent all interventionists or recipients in a setting from completing part or all of the intervention. We propose 12 corresponding heuristics to guide EBPIs and implementation strategy designers in preventing and addressing these usability issues.

CONCLUSIONS

Usability issues were prevalent in the studied EBPIs and implementation strategies. We recommend using the lens of usability evaluation to understand and address barriers to the effective use and reach of EBPIs and implementation strategies.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/14990.

A Perspective on How User-Centered Design Could Improve the Impact of Self-Applied Psychological Interventions in Low- or Middle-Income Countries in Latin America

Global technological progress has generated alternatives for psychological assistance, both for the evaluation and for the treatment of different emotional disorders. Evidence suggests that Internet-based treatments are effective for the treatment of anxiety and depression disorders. However, in Latin America online treatments are still scarce compared to developed countries and have similar problems as developed countries, such as high dropout rate. One possible solution to help decrease the dropout rate is to design and develop online interventions based on the needs and characteristics of the users. The user-centered design (UCD) is a fundamental concept to develop successful online interventions. The objective of this article is to provide a perspective overview on how UCD could improve the impact of self-applied psychological interventions in low- or middle-income countries in Latin America; however this proposal can also be applied in low- and middle-income countries in other regions of the world. The literature on UCD has demonstrated its efficacy when properly applied in online interventions; however, it is not common to see how this methodology has been applied in research in online interventions, and regarding Latin America, this is even more scarce with a very limited number of articles implementing the principles of UCD.

Lessons learned from designing an asynchronous remote community approach for behavioral activation intervention for teens

Adolescent depression is common; however, over 60% of depressed adolescents do not receive mental health care. Digitally-delivered evidence-based psychosocial interventions (EBPIs) may provide an opportunity to improve access and engagement in mental health care. We present a case study that reviews lessons learned from using the Discover - Design - Build - Test (DDBT) model to create, develop, and evaluate a high-fidelity prototype of an app to deliver an EBPI for depression, behavioral activation (BA), on an Asynchronous Remote Communities (ARC) platform (referred to as ActivaTeen). We review work at each stage of the DDBT framework, including initial formative work, iterative design and development work, and an initial feasibility study. We engaged teens with depression, mental health clinicians, and expert evaluators through the process. We found that the DDBT model supported the research team in understanding the requirements for our prototype system, ActivaTeen, and conceiving of and developing specific ideas for implementation. Our work contributes a case study of how the DDBT framework can be applied to adapting an EBPI to a new, scalable and digital format. We provide lessons learned from engaging teens and clinicians with an asynchronous approach to EBPIs and human centered design considerations for teen mental health.

Meeting Users Where They Are: User-centered Design of an Automated Text Messaging Tool to Support the Mental Health of Young Adults

Young adults have high rates of mental health conditions, but most do not want or cannot access formal treatment. We therefore recruited young adults with depression or anxiety symptoms to co-design a digital tool for self-managing their mental health concerns. Through study activities-consisting of an online discussion group and a series of design workshops-participants highlighted the importance of easy-to-use digital tools that allow them to exercise independence in their self-management. They described ways that an automated messaging tool might benefit them by: facilitating experimentation with diverse concepts and experiences; allowing variable depth of engagement based on preferences, availability, and mood; and collecting feedback to personalize the tool. While participants wanted to feel supported by an automated tool, they cautioned against incorporating an overtly human-like motivational tone. We discuss ways to apply these findings to improve the design and dissemination of digital mental health tools for young adults.

Iterative redesign of a caregiver-mediated intervention for use in educational settings

LAY ABSTRACT

Teachers often report concerns about behavior challenges in their students with autism spectrum disorder (ASD) in the school setting. Furthermore, teachers often report that they do not have adequate training in how to manage these challenging behaviors effectively. The RUBI program is an intervention initially developed for parents of children with ASD and co-occurring challenging behavior in clinic settings. The present project used school staff input to systematically redesign RUBI to be used with educators in schools. School staff gave input at multiple stages of development to ensure the adapted intervention was appropriate to use in a school setting. Responses were coded and analyzed to identify strengths and weaknesses of the RUBI manual in schools and adaptations were made accordingly. Scores of how appropriate, possible, likable, and usable RUBI would be in schools rose after the intervention was redesigned. The redesigned RUBIES manual may give school staff the tools they need to manage disruptive behaviors. In addition, collaborating with providers over multiple stages to redesign established interventions for new contexts may be a promising way to help bring research tools to practice in the future.

Generational Perspectives on Technology's Role in Mental Health Care: A Survey of Adults With Lived Mental Health Experience

Introduction

Personal technology (e.g., smartphones, wearable health devices) has been leveraged extensively for mental health purposes, with upwards of 20,000 mobile applications on the market today and has been considered an important implementation strategy to overcome barriers many people face in accessing mental health care. The main question yet to be addressed is the role consumers feel technology should play in their care. One underserved demographic often ignored in this discussion are people over the age of 60. The population of adults 60 and older is predicted to double by 2,050 signaling a need to address how older adults view technology for their mental health care.

Objective

The objective of this study is to better understand why digital mental health tools are not as broadly adopted as predicted, what role people with lived mental health experience feel technology should play in their care and how those results compare across age groups.

Method

In a mixed-methods approach, we analyzed results from a one-time cross-sectional survey that included 998 adults aged 18–83 with lived experience of mental health concerns recruited from Prolific, an online research platform. We surveyed participant's use of technology including their perspectives on using technology in conjunction with their mental health care. We asked participants about their previous use of digital mental health tools, their treatment preferences for mental health care, and the role technology should play in their mental health care.

Results

Across all age groups, respondents had favorable views of using digital mental health for managing mental health care. However, older adults rated their acceptability of digital mental health tools lower than middle-aged and younger adults. When asked what role technology should play in mental health care in an open-ended response, most participants responded that technology should play a complementary role in mental health care (723/954, 75.8%).

Conclusion

Digital mental health is seen as a valuable care management tool across all age groups, but preferences for its role in care remain largely administrative and supportive. Future development of digital mental health should reflect these preferences.

A skills-based sero-status HIV disclosure intervention for sexual minority men in South Africa: A protocol for intervention adaptation and pilot randomized controlled trial (Preprint)

Background

Gay, bisexual, and other men who have sex with men (GBMSM) living with HIV have low antiretroviral treatment adherence in South Africa due to limited skills in managing disclosure and prevention behaviors with sexual and romantic partners. As a result, there is a high HIV transmission risk within HIV-discordant partnerships, but an existing intervention may address these outcomes, if adapted effectively. Healthy Relationships (HR) is a behavioral intervention that was originally delivered in groups and in person over 5 sessions to develop coping skills for managing HIV-related stress and sexually risky situations, enhance decision-making skills for HIV disclosure to partners, and establish and maintain safer sex practices with partners. HR effectively improves prevention behaviors but has yet to be tailored to a non-US context.

Objective

We aim to adapt HR into a new culturally grounded intervention entitled Speaking Out & Allying Relationships for GBMSM and then assess its feasibility in Eastern Cape, South Africa.

Methods

The study will have 2 aims. For aim 1—adaptation—we will use a human-centered design approach. Initial intervention tailoring will involve integrating Undetectable=Untransmittable and pre-exposure prophylaxis education, developing intervention content for a videoconference format, and designing role-plays and movies for skill building based on preliminary data. Afterward, interviews and surveys will be administered to GBMSM to assess intervention preferences, and a focus group will be conducted with health care providers and information technology experts to assess the intervention’s design. Finally, a usability test will be performed to determine functionality and content understanding. Participants will be GBMSM living with HIV (n=15) who are in a relationship and health care providers and information technology (n=7) experts working in HIV care and programming with this population. For aim 2, we will examine the feasibility of the adapted intervention by using a pilot randomized control design. There will be 60 individuals per arm. Feasibility surveys and interviews will be conducted with the intervention arm, and behavioral and biomedical assessments for relationship and treatment adherence outcomes will be collected for both arms. All participants will be GBMSM living with HIV who are in a relationship with an HIV-negative or unknown status partner.

Results

Intervention adaptation began in August 2021. Initial tailoring and the refining of GBMSM intervention preferences were completed in December 2021. Usability and feasibility assessments are due to be completed by March 2022 and February 2024, respectively.

Conclusions

GBMSM need efficacious interventions that tackle partnership dynamics, HIV prevention, and treatment outcomes for antiretroviral treatment adherence and viral suppression in South Africa. Harnessing everyday technology use for social networking (eg, videoconferences), Undetectable=Untransmittable education, and pre-exposure prophylaxis to update an existing intervention for South African GBMSM has the potential to strengthen relationship communication about HIV treatment and prevention and, in turn, improve outcomes.

International Registered Report Identifier (IRRID)

DERR1-10.2196/36845

A Brief Online Implicit Bias Intervention for School Mental Health Clinicians

Clinician bias has been identified as a potential contributor to persistent healthcare disparities across many medical specialties and service settings. Few studies have examined strategies to reduce clinician bias, especially in mental healthcare, despite decades of research evidencing service and outcome disparities in adult and pediatric populations. This manuscript describes an intervention development study and a pilot feasibility trial of the Virtual Implicit Bias Reduction and Neutralization Training (VIBRANT) for mental health clinicians in schools-where most youth in the U.S. access mental healthcare. Clinicians (N = 12) in the feasibility study-a non-randomized open trial-rated VIBRANT as highly usable, appropriate, acceptable, and feasible for their school-based practice. Preliminarily, clinicians appeared to demonstrate improvements in implicit bias knowledge, use of bias-management strategies, and implicit biases (as measured by the Implicit Association Test [IAT]) post-training. Moreover, putative mediators (e.g., clinicians' VIBRANT strategies use, IAT D scores) and outcome variables (e.g., clinician-rated quality of rapport) generally demonstrated correlations in the expected directions. These pilot results suggest that brief and highly scalable online interventions such as VIBRANT are feasible and promising for addressing implicit bias among healthcare providers (e.g., mental health clinicians) and can have potential downstream impacts on minoritized youth's care experience.

Human-centered design methods to achieve preparation phase goals in the multiphase optimization strategy framework

Background

The public health impact of behavioral and biobehavioral interventions to prevent and treat mental health and substance use problems hinges on developing methods to strategically maximize their effectiveness, affordability, scalability, and efficiency.

Methods

The multiphase optimization strategy (MOST) is an innovative, principled framework that guides the development of multicomponent interventions. Each phase of MOST (Preparation, Optimization, Evaluation) has explicit goals and a range of appropriate research methods to achieve them. Methods for attaining Optimization and Evaluation phase goals are well-developed. However, methods used in the Preparation phase are often highly researcher-specific, and concrete ways to achieve Preparation phase goals are a priority area for further development.

Results

We propose that the discover, design, build, and test (DDBT) framework provides a theory-driven and methods-rich roadmap for achieving the goals of the Preparation phase of MOST, including specifying the conceptual model, identifying and testing candidate intervention components, and defining the optimization objective. The DDBT framework capitalizes on strategies from the field of human-centered design and implementation science to drive its data collection methods.

Conclusions

MOST and DDBT share many conceptual features, including an explicit focus on implementation determinants, being iterative and flexible, and designing interventions for the greatest public health impact. The proposed synthesized DDBT/MOST approach integrates DDBT into the Preparation phase of MOST thereby providing a framework for rigorous and efficient intervention development research to bolster the success of intervention optimization.

Plain Language Summary

1. What is already known about the topic? Optimizing behavioral interventions to balance effectiveness with affordability, scalability, and efficiency requires a significant investment in intervention development.2. What does this paper add? This paper provides a structured approach to integrating human-centered design principles into the Preparation phase of the multiphase optimization strategy (MOST).3. What are the implications for practice, research, or policy? The proposed synthesized model provides a framework for rigorous and efficient intervention development research in the Preparation phase of MOST that will ensure the success of intervention optimization and contribute to improving public health impact of mental health and substance use interventions.

Applying the lessons of implementation science to maximize feasibility and usability in team science intervention development

The Science of Team Science (SciTS) has generated a substantial body of work detailing characteristics of effective teams. However, that knowledge has not been widely translated into accessible, active, actionable, evidence-based interventions to help translational teams enhance their team functioning and outcomes. Over the past decade, the field of Implementation Science has rapidly developed methods and approaches to increase the translation of biomedical research findings into clinical care, providing a roadmap for mitigating the challenges of developing interventions while maximizing feasibility and utility. Here, we propose an approach to intervention development using constructs from two Implementation Science frameworks, Consolidated Framework for Implementation Research, and Reach, Effectiveness, Adoption, Implementation, and Maintenance, to extend the Wisconsin Interventions for Team Science framework described in Rolland et al. 2021. These Implementation Science constructs can help SciTS researchers design, build, test, and disseminate interventions that meet the needs of both adopters, the institutional leadership that decides whether to adopt an intervention, and implementers, those actually using the intervention. Systematically considering the impact of design decisions on feasibility and usability may lead to the design of interventions that can quickly move from prototype to pilot test to pragmatic trials to assess their impact.