Assessing the Usability of Complex Psychosocial Interventions: The Intervention Usability Scale

Feasibility study of a telehealth school-based behavioral parent training group program for attention-deficit/hyperactivity disorder

OBJECTIVE

To evaluate the feasibility and preliminary efficacy of Telehealth Behavioral Parent Training (T-BPT), a school telehealth group intervention for attention-deficit/hyperactivity disorder (ADHD) with a companion training program for school clinicians.

METHODS

T-BPT was developed in an iterative three-phase design in partnership with community stakeholders during the COVID-19 pandemic. School clinicians (N = 4) delivered T-BPT over 8 weeks to parents (N = 21, groups of 5-6 per school) of children (Grades 2-5) with ADHD while simultaneously receiving training and consultation from PhD-level study trainers. A single-arm open trial was used to assess feasibility, engagement, and preliminary efficacy.

RESULTS

Parents and school clinicians endorsed high feasibility, acceptability, and usability of T-BPT. Parent attendance was high (M = 94.6%) and a majority of parents (66.7%) attended all eight sessions. Preliminary outcomes indicate moderate to large reductions in parent-reported ADHD symptoms (ω2 = .36), functional and clinical global impairment (ω2s= .21 and .19, respectively), and distance learning challenges (ω2 = .22).

CONCLUSIONS

Results were in line with in-person delivery, indicating promising feasibility of school telehealth BPT groups. This study also provided further support for the feasibility of the remote training model for school clinicians. Implications of the commonly endorsed barriers and benefits beyond COVID-19 and relevance to under resourced communities are also discussed.

Reconceiving Domestic Burning Controls: Air Quality Alerts, Behavioural Responsive Regulation, and Designing for Compliance

Domestic combustion emissions pose a growing risk to public health, especially in the UK. Existing responses are polarised, with government advocating use of lower emission fuels and stoves while clean air campaigners call for blanket bans on burning. However, each approach is limited in its ability to control these emissions. An alternative can be found in the U.S.A., where 'burn alert' systems require stove and fireplace users to avoid lighting during periods of actual or projected poor air quality. Given the effectiveness of these regimes, the current study designs and evaluates the effectiveness and acceptability of a burn alert system in the UK for the first time, drawing on the theoretical perspective of behavioural responsive regulation. Fifty participants were recruited to use the system over 2 weeks in winter. The findings illustrate that a voluntary burn alert system can dissuade burning among users. Of those in receipt of an alert, 74% reduced burning frequency or burned for a shorter duration. In total, the alert system prevented at least 178 hours of burning for this group. Qualitative findings show that the consistency of the behavioural response is influenced by technical, structural, and environmental factors, providing key insight into how UK-based burn alert systems could be modified to increase the consistency of compliance in future. The overall conclusion is that burn alerts could be introduced in the UK and beyond, as a means of reducing domestic combustion emissions and their associated public health risks.

Implementation of a perioperative registry in Ethiopia to enhance surgical quality improvement

BACKGROUND

Despite a glaring need and proven efficacy, prospective surgical registries are lacking in low- and middle-income countries. The objective of this study was to design and implement a comprehensive prospective perioperative registry in a low-income country.

METHODS

This study was conducted at Hawassa University Comprehensive Specialized Hospital in Hawassa, Ethiopia. Design of the registry occurred from June 2021 to May 2022 and pilot implementation from May 2022 to May 2023. All patients undergoing elective or emergent general surgery were included. Following one year, operability and fidelity of the registry were analyzed by assessing capture rate, incidence of missing data, and accuracy.

RESULTS

A total of 67 variables were included in the registry including demographics, preoperative, operative, post-operative, and 30-day data. Of 440 eligible patients, 226 (51.4%) were successfully captured. Overall incidence of missing data and accuracy was 5.4% and 90.2% respectively. Post pilot modifications enhanced capture rate to 70.5% and further optimized data collection processes.

CONCLUSION

The establishment of a low-cost electronic prospective perioperative registry in a low-income country represents a significant step forward in enhancing surgical care in under-resourced settings. The initial success of this registry highlights the feasibility of such endeavors when strong partnerships and local context are at the center of implementation. Continuous efforts to refine this registry are ongoing, which will ultimately lead to enhanced surgical quality, research output, and expansion to other sites.

Applying the Cognitive Walkthrough for Implementation Strategies Methodology to Inform the Redesign of a Selection-Quality Implementation Toolkit for Use in Schools

Background

Implementation strategies are key to enhancing translation of new innovations but there is a need to systematically design and tailor strategies to match the targeted implementation context and address determinants. There are increasing methods to inform the redesign and tailoring of implementation strategies to maximize their usability, feasibility, and appropriateness in new settings such as the Cognitive Walkthrough for Implementation Strategies (CWIS) approach. The aim of the current project is to apply the CWIS approach to inform the redesign of a multifaceted selection-quality implementation toolkit entitled ACT SMARTS for use in middle and high schools.

Methods

We systematically applied CWIS as the second part of a community-partnered iterative redesign of ACT SMARTS for schools to evaluate the usability and inform further toolkit redesign areas. We conducted three CWIS user testing sessions with key end users of school district administrators, school principals, and educators.

Results

Our CWIS application revealed that end users found ACT SMARTS acceptable and relevant but anticipate usability issues engaging in the ACT SMARTS process. Results informed the identification of eleven usability issues and corresponding redesign solutions to enhance the usability of ACT SMARTS for use in middle and high schools.

Conclusions

Results indicated the utility of CWIS in assessing implementation strategy usability in service of informing strategy tailoring and redesign to improve alignment with user and setting needs. Recommendations regarding the use of this participatory approach are discussed.

An intervention to address nurse educators' occupational well-being: A process evaluation

BACKGROUND

There is a global need for more support for the occupational well-being of educators working in nurse education, where nurse educators experience challenges when managing their own occupational well-being. However, there is a lack of research studies into occupational well-being interventions. Aim To evaluate the usability and utility of the Self-Help INtervention for Educators in nurse education.

DESIGN

A process evaluation was conducted after the intervention in Spring 2022.

SETTINGS

Educational organisations providing national regulated practical nurse education in Finland.

PARTICIPANTS

Nurse educators (n = 37), completing the 8-workweek intervention.

METHODS

Data were collected with the self-reported electronic feedback questionnaire consisting of three sections: 1) 10-item System Usability Scale, 2) 7-item Utility scale developed for this study and 3) 4 open ended questions. The data were analysed statistically and with content analysis.

RESULTS

The intervention was found to be usable; especially the easy learnability and usage of the digital Smart Break-SHINE program and the applicable exercises. It was estimated to be moderately useful as a well-being and break promoter at work. The utility of the intervention to promote physical activity and recovery during working hours was statistically more positive for educators with <5 years of work experience than those with over 15 years. Usability and utility barriers were found especially regarding workload issues.

CONCLUSIONS

The Self-Help INtervention for Educators supports the occupational well-being of nurse educators and includes well-being actions suitable for different work surroundings (e.g., remote working) without the need for constant facilitating. The intervention was found to be most beneficial for early career nurse educators. The Self-Help INtervention for Educators needs more development to overcome the usability and utility barriers related to workload issues.

HCI Contributions in Mental Health: A Modular Framework to Guide Psychosocial Intervention Design

Many people prefer psychosocial interventions for mental health care or other concerns, but these interventions are often complex and unavailable in settings where people seek care. Intervention designers use technology to improve user experience or reach of interventions, and HCI researchers have made many contributions toward this goal. Both HCI and mental health researchers must navigate tensions between innovating on and adhering to the theories of change that guide intervention design. In this paper, we propose a framework that describes design briefs and evaluation approaches for HCI contributions at the scopes of capabilities, components, intervention systems, and intervention implementations. We show how theories of change (from mental health) can be translated into design briefs (in HCI), and that these translations can bridge and coordinate efforts across fields. It is our hope that this framework can support researchers in motivating, planning, conducting, and communicating work that advances psychosocial intervention design.

Support Through Remote Observation and Nutrition Guidance (STRONG), a digital health intervention to reduce malnutrition among pancreatic cancer patients: A study protocol for a pilot randomized controlled trial

Background

Malnutrition is a common and distressing condition among pancreatic cancer patients. Fewer than a quarter of pancreatic cancer patients receive medical nutrition therapy (MNT), important for improving nutritional status, weight maintenance, quality of life and survival. System, provider, and patient level barriers limit access to MNT. We propose to examine the feasibility of a 12-week multi-level, digital health intervention designed to expand MNT access among pancreatic cancer patients.

Methods

Individuals with advanced pancreatic cancer starting chemotherapy (N = 80) will be 1:1 randomized to the intervention or usual care. The Support Through Remote Observation and Nutrition Guidance (STRONG) intervention includes system-level (e.g., routine malnutrition and screening), provider-level (e.g., dietitian training and web-based dashboard), and patient-level strategies (e.g., individualized nutrition plan, self-monitoring of dietary intake via Fitbit, ongoing goal monitoring and feedback). Individuals receiving usual care will be referred to dietitians based on their oncologists' discretion. Study assessments will be completed at baseline, 4-, 8-, 12-, and 16-weeks.

Results

Primary outcomes will be feasibility (e.g., recruitment, retention, assessment completion) and acceptability. We will collect additional implementation outcomes, such as intervention adherence, perceived usability, and feedback on intervention quality via an exit interview. We will collect preliminary data on outcomes that may be associated with the intervention including malnutrition, quality of life, treatment outcomes, and survival.

Conclusion

This study will advance our knowledge on the feasibility of a digital health intervention to reduce malnutrition among individuals with advanced pancreatic cancer. Trial registration: NCT05675059, registered on December 9, 2022.

Formative Evaluation of a Comprehensive Self-Management Intervention for Irritable Bowel Syndrome, Comorbid Anxiety, and Depression: Mixed Methods Study

BACKGROUND

Irritable bowel syndrome (IBS) is a disorder of the gut-brain interaction that is associated with abdominal pain, altered bowel patterns, and reduced quality of life. Up to 50% of patients with IBS also report anxiety or depressive symptoms. Although effective self-management interventions exist for individuals with IBS, few have been effectively implemented, and most do not consider the unique needs of patients with comorbid IBS and anxiety or depression.

OBJECTIVE

This study aimed to determine the anticipated acceptability, appropriateness, feasibility, and usability of a comprehensive self-management intervention using an implementation science and human-centered design approach among individuals with comorbid IBS and anxiety or depression and health care providers.

METHODS

A convergent mixed methods design was used to elicit feedback on the comprehensive self-management intervention outline and content to identify refinement needs before testing. Patients with IBS and moderate to severe anxiety or depression and health care providers were purposefully sampled from primary care and gastroenterology settings. Participants completed semistructured interviews and surveys on anticipated acceptability, appropriateness, feasibility, and usability.

RESULTS

Patient participants (n=12) were on average 36.8 (SD 12.2) years of age, and 42% (5/12) were currently receiving psychological therapy. Health care providers (n=14) were from primary care (n=7) and gastroenterology (n=7) settings. The mean usability scores (out of 100) were 52.5 (SD 14.5) for patients and 45.6 (SD 11.6) for providers. For patients and providers, qualitative data expanded the quantitative findings for acceptability and appropriateness. Acceptability findings were the comprehensive nature of the intervention and discussion of the gut-brain interaction. For appropriateness, participants reported that the intervention provided structure, accountability, and support. Feasibility was confirmed for patients, but there was a divergence of findings between quantitative and qualitative measures for providers. Patients focused on intervention feasibility, while providers focused on implementation feasibility in the clinic. Identified usability issues to address before implementation included the intervention delivery format, length, and lack of integration into health care settings that, if not addressed, may limit the reach of the intervention.

CONCLUSIONS

Patients and health care providers found the intervention acceptable and appropriate. Several feasibility and usability issues were identified, including intervention delivery methods, length of intervention, and the best methods to implement in the clinic setting. The next steps are to refine the intervention to address the identified issues and test in a pilot study whether addressing usability issues leads to the anticipated improvements in implementation and uptake.

Participatory development of a manual for the implementation of diversity-sensitive palliative and hospice care in Germany: a mixed-methods study protocol

BACKGROUND

The diversity of the population is associated with different needs and expectations towards palliative and hospice care. Current approaches available in Germany generally fall short in addressing the role of diversity and intersectionality in this health care setting and healthcare facilities struggle with organizational difficulties and missing information on how to implement corresponding diversity-sensitive measures. The present study aims to develop a hands-on manual that enables providers of hospice and palliative care to implement measures and strategies for diversity-sensitive care, while taking into account the perspective of healthcare users and explicitly including vulnerable and minority patient groups.

METHODS

A participatory approach is used to co-create the aforementioned manual using an explanatory sequential mixed-methods design. First, based on a systematic analysis of existing measures, an initial draft of the manual will be developed. Subsequently, an online survey will be conducted among all hospice and palliative care providers in Germany (n = 2,823). Based on the results of the survey, 12 to 15 qualitative problem-centered interviews will be conducted with employees of selected providers who took part in the survey. Results of the survey and the qualitative interviews will be integrated and analyzed. In parallel to the development and research process, a comprehensive dissemination strategy will be developed.

DISCUSSION

The manual will assist providers of palliative and hospice care in determining goals, needs, and available resources in order to utilize patient-centered and diversity-sensitive measures to meet a wide range of expectations. It can also be informative for providers in other countries. The participatory co-development approach ensures the practical relevance of the manual, while the mixed-methods design allows for targeted input on the manual's usability, acceptance, and viability as a supportive tool.

Mixed-methods evaluation of a group psychosocial intervention for refugee, migrant and host community women in Ecuador and Panamá: Results from the Entre Nosotras cluster randomized feasibility trial

Community-based psychosocial interventions are key elements of mental health and psychosocial support; yet evidence regarding their effectiveness and implementation in humanitarian settings is limited. This study aimed to assess the appropriateness, acceptability, feasibility and safety of conducting a cluster randomized trial evaluating two versions of a group psychosocial intervention. Nine community clusters in Ecuador and Panamá were randomized to receive the standard version of the Entre Nosotras intervention, a community-based group psychosocial intervention co-designed with community members, or an enhanced version of Entre Nosotras that integrated a stress management component. In a sample of 225 refugees, migrants and host community women, we found that both versions were safe, acceptable and appropriate. Training lay facilitators to deliver the intervention was feasible. Challenges included slow recruitment related to delays caused by the COVID-19 pandemic, high attrition due to population mobility and other competing priorities, and mixed psychometric performance of psychosocial outcome measures. Although the intervention appeared promising, a definitive cluster randomized comparative effectiveness trial requires further adaptations to the research protocol. Within this pilot study we identified strategies to overcome these challenges that may inform adaptations. This comparative effectiveness design may be a model for identifying effective components of psychosocial interventions.

Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya

Opportunities exist to leverage mobile phones to replace or supplement in-person supervision of lay counselors. However, contextual variables, such as network connectivity and provider preferences, must be considered. Using an iterative and mixed methods approach, we co-developed implementation guidelines to support the implementation of mobile phone supervision with lay counselors and supervisors delivering a culturally adapted trauma-focused cognitive behavioral therapy in Western Kenya. Guidelines were shared and discussed with lay counselors in educational outreach visits led by supervisors. We evaluated the impact of guidelines and outreach on the acceptability, feasibility, and usability of mobile phone supervision. Guidelines were associated with significant improvements in acceptability and usability of mobile phone supervision. There was no evidence of a significant difference in feasibility. Qualitative interviews with lay counselors and supervisors contextualized how guidelines impacted acceptability and feasibility - by setting expectations for mobile phone supervision, emphasizing importance, increasing comfort, and sharing strategies to improve mobile phone supervision. Introducing and discussing co-developed implementation guidelines significantly improved the acceptability and usability of mobile phone supervision. This approach may provide a flexible and scalable model to address challenges with implementing evidence-based practices and implementation strategies in lower-resourced areas.

Artificial Intelligence-Based Chatbots for Promoting Health Behavioral Changes: Systematic Review

BACKGROUND

Artificial intelligence (AI)-based chatbots can offer personalized, engaging, and on-demand health promotion interventions.

OBJECTIVE

The aim of this systematic review was to evaluate the feasibility, efficacy, and intervention characteristics of AI chatbots for promoting health behavior change.

METHODS

A comprehensive search was conducted in 7 bibliographic databases (PubMed, IEEE Xplore, ACM Digital Library, PsycINFO, Web of Science, Embase, and JMIR publications) for empirical articles published from 1980 to 2022 that evaluated the feasibility or efficacy of AI chatbots for behavior change. The screening, extraction, and analysis of the identified articles were performed by following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

Of the 15 included studies, several demonstrated the high efficacy of AI chatbots in promoting healthy lifestyles (n=6, 40%), smoking cessation (n=4, 27%), treatment or medication adherence (n=2, 13%), and reduction in substance misuse (n=1, 7%). However, there were mixed results regarding feasibility, acceptability, and usability. Selected behavior change theories and expert consultation were used to develop the behavior change strategies of AI chatbots, including goal setting, monitoring, real-time reinforcement or feedback, and on-demand support. Real-time user-chatbot interaction data, such as user preferences and behavioral performance, were collected on the chatbot platform to identify ways of providing personalized services. The AI chatbots demonstrated potential for scalability by deployment through accessible devices and platforms (eg, smartphones and Facebook Messenger). The participants also reported that AI chatbots offered a nonjudgmental space for communicating sensitive information. However, the reported results need to be interpreted with caution because of the moderate to high risk of internal validity, insufficient description of AI techniques, and limitation for generalizability.

CONCLUSIONS

AI chatbots have demonstrated the efficacy of health behavior change interventions among large and diverse populations; however, future studies need to adopt robust randomized control trials to establish definitive conclusions.

Formative Evaluation of a Student Symptom Decision Tree for COVID-19

Objectives

In support of schools restarting during the COVID-19 pandemic, some schools partnered with local experts in academia, education, community, and public health to provide decision-support tools for determining what actions to take when presented with students at risk for spreading infection at school.

Methods

The Student Symptom Decision Tree, developed in Orange County, California, is a flow chart consisting of branching logic and definitions to assist school personnel in making decisions regarding possible COVID-19 cases in schools which was repeatedly updated to reflect evolving evidence-based guidelines. A survey of 56 school personnel evaluated the frequency of use, acceptability, feasibility, appropriateness, usability, and helpfulness of the Decision Tree.

Results

The tool was used at least 6 times a week by 66% of respondents. The Decision Tree was generally perceived as acceptable (91%), feasible (70%), appropriate (89%), usable (71%) and helpful (95%). Suggestions for improvement included reducing the complexity in content and formatting of the tool.

Conclusions

The data suggest that school personnel found value in the Decision Tree, which was intended to assist them with making decisions in a challenging and rapidly evolving pandemic.

Development of a Web-Based Training Platform for School Clinicians in Evidence-Based Practices for ADHD

Lack of training for school clinicians in evidence-based practices (EBPs) contributes to underutilization of such services for youth with attention-deficit/hyperactivity disorder (ADHD). Advances in web-based technology and videoconferencing have allowed for expanded access to and optimization of training. We describe the development and outcomes of a novel web-based platform for training school clinicians to gain skills in EBPs for school-age youth with ADHD. The training platform is adapted from an empirically supported, in-person training for a school-home behavioral intervention (Collaborative Life Skills program) and includes skill modules for working with teachers, parents, and students. Training methods include web-accessed manuals/handouts, skill example video clips, automated progress monitoring tools, and consultation/in-session coaching via videoconferencing. We gathered stakeholder qualitative and quantitative feedback during discovery and design phases of the iterative development. We then evaluated the usability, acceptability, fidelity and clinician and student outcomes of the remote training program. Focus group themes and qualitative feedback identified clinician preferences for remote training features (e.g., interactive, brief, role-plays/coaching methods), video tools (recorded samples of skills and therapy sessions), and progress monitoring tools (e.g., clear, easy to use). Clinician usability ratings of the platform were high with most components rated as moderately to very useful/easy to use. Clinician ratings of usability, fidelity implementing the treatment, and their EBP knowledge and confidence following training were favorable. Student's outcomes were similar to those achieved in prior studies of clinician in-person training. Results support the promise of remote, web-based clinician training for the dissemination of evidence-based practices.

Effectiveness of Mental Health Apps for Distress During COVID-19 in US Unemployed and Essential Workers: Remote Pragmatic Randomized Clinical Trial

BACKGROUND

During the COVID-19 pandemic, the general public was concerned about the mental health impacts of unemployment due to COVID-19 and the stress essential workers experienced during this time. Several reports indicated that people in distress were turning to digital technology, but there was little evidence about the impact of these tools on mitigating distress.

OBJECTIVE

This study seeks to determine the acceptability, feasibility, usability, and effectiveness of mobile mental health apps for decreasing mental health symptoms in essential workers and unemployed individuals with suicide risk.

METHODS

We recruited participants who indicated that they were unemployed because of COVID-19 or were COVID-19-designated essential workers. Participants were randomized to 1 of 4 free commercial mobile apps for managing distress that were (1) highly rated by PsyberGuide and (2) met the criteria for intervention features these participants indicated were desirable in a previous survey. Participants used the apps for 4 weeks and completed baseline and 4-week self-assessments of depression, anxiety emotional regulation, and suicide risk.

RESULTS

We found no differences between the apps in any outcome but did find significant changes in depression and anxiety over time (Patient Health Questionnaire [PHQ]-9: estimate=-1.5, SE 0.2, 95% CI -1.1 to -1.8, P<.001; Generalized Anxiety Disorder Scale [GAD]-7: estimate=-1.3, SE 0.2, 95% CI -1.0 to -1.6, P<.001). We found no significant changes in suicidal behavior (Suicide Behaviors Questionnaire-Revised [SBQ-R]) or emotional regulation (Difficulties in Emotion Regulation Scale - Short Form [DERS-SF]) for the 4 weeks. We did find a significant dose-response pattern for changes in depression and anxiety. Using the app at least once a week resulted in greater improvements in treatment conditions over time on depression (estimate=-0.6, SE 0.2, 95% CI 1.0-0.2, P=.003) and anxiety (estimate=0.1, SE 0.2, 95% CI 0.4-0.6, P=.78). There was no association between app frequency and changes in suicidal behavior (SBQ-R) or emotional regulation (DERS-SF). We further found a significant difference between the conditions with regard to app usability, with the control app being the most usable (mean_{Beautiful Mood} 72.9, SD 16.7; mean_{COVID Coach} 71.2, SD 15.4; mean_Calm 66.8, SD 17.3; mean_{7 Cups} 65.2, SD 17.7). We found no significant differences for app acceptability or appropriateness.

CONCLUSIONS

Few studies have evaluated prospectively the utility and usability of commercial apps for mood. This study found that free, self-guided commercial mobile mental health apps are seen as usable, but no one app is superior to the other. Although we found that regular use is indicated for effects on depression and anxiety to occur in those who are more symptomatic, regression to the mean cannot be ruled out.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04536935; https://tinyurl.com/mr36zx3s.

Evaluating the feasibility of a group psychosocial intervention for migrant and host community women in Ecuador and Panamá: protocol for a multi-site feasibility cluster trial

BACKGROUND

Community- and strengths-based psychosocial interventions are central to mental health and psychosocial support guidelines, but rigorous evidence regarding the effectiveness of these interventions is limited. The complexity and variability that is inherent to many community-based psychosocial interventions requires innovative strategies in order to facilitate the comparability and synthesis across research studies without compromising the fit and appropriateness of interventions to specific study populations and context. Entre Nosotras is a community-based psychosocial intervention developed for migrant and host community women that is designed to be flexible enough to enable integration of external intervention components and adaptable to diverse study contexts and populations. This protocol describes a study that aims to evaluate the appropriateness, acceptability, and feasibility of integrating a standardized stress management intervention into Entre Nosotras.

METHODS

This study will evaluate the appropriateness, acceptability, feasibility, and safety of intervention and research procedures for a cluster randomized comparative effectiveness trial conducted in Ecuador and Panamá with migrant and host community women. In this feasibility trial, we will allocate communities nested within the three study sites to the integrated Entre Nosotras + stress management intervention versus Entre Nosotras alone through stratified randomization. Migrant and host community women residing in these study communities who report low to moderate levels of distress will be allocated to the intervention condition that their community is assigned (n = 220 total). We will collect quantitative measures of psychosocial wellbeing, psychological distress, coping, social support, and functioning from study participants. We will collect quantitative measures of fidelity and facilitator competencies through observation and facilitator self-assessment. Data on appropriateness, acceptability, feasibility, and safety will be gathered from participants and facilitators through quantitative assessments at 0, 5, and 10 weeks post-enrollment and qualitative interviews conducted with all facilitators and a subset of 70 study participants during the post-intervention follow-up period.

DISCUSSION

Results from this feasibility trial will determine whether a multi-site cluster randomized comparative effectiveness trial of an adaptable community-based psychosocial intervention for migrant and host community women is relevant, acceptable, and feasible.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT05130944 . Registered November 23, 2021-retrospectively registered.

Usability Issues in Evidence-Based Psychosocial Interventions and Implementation Strategies: Cross-project Analysis

BACKGROUND

People often prefer evidence-based psychosocial interventions (EBPIs) for mental health care; however, these interventions frequently remain unavailable to people in nonspecialty or integrated settings, such as primary care and schools. Previous research has suggested that usability, a concept from human-centered design, could support an understanding of the barriers to and facilitators of the successful adoption of EBPIs and support the redesign of EBPIs and implementation strategies.

OBJECTIVE

This study aimed to identify and categorize usability issues in EBPIs and their implementation strategies.

METHODS

We adapted a usability issue analysis and reporting format from a human-centered design. A total of 13 projects supported by the National Institute of Mental Health-funded Accelerating the Reach and Impact of Treatments for Youth and Adults with Mental Illness Center at the University of Washington used this format to describe usability issues for EBPIs and implementation strategies with which they were working. Center researchers used iterative affinity diagramming and coding processes to identify usability issue categories. On the basis of these categories and the underlying issues, we propose heuristics for the design or redesign of EBPIs and implementation strategies.

RESULTS

The 13 projects reported a total of 90 usability issues, which we categorized into 12 categories, including complex and/or cognitively overwhelming, required time exceeding available time, incompatibility with interventionist preference or practice, incompatibility with existing workflow, insufficient customization to clients/recipients, intervention buy-in (value), interventionist buy-in (trust), overreliance on technology, requires unavailable infrastructure, inadequate scaffolding for client/recipient, inadequate training and scaffolding for interventionists, and lack of support for necessary communication. These issues range from minor inconveniences that affect a few interventionists or recipients to severe issues that prevent all interventionists or recipients in a setting from completing part or all of the intervention. We propose 12 corresponding heuristics to guide EBPIs and implementation strategy designers in preventing and addressing these usability issues.

CONCLUSIONS

Usability issues were prevalent in the studied EBPIs and implementation strategies. We recommend using the lens of usability evaluation to understand and address barriers to the effective use and reach of EBPIs and implementation strategies.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/14990.

It's all in the name: why exposure therapy could benefit from a new one

Exposure therapy for anxiety and related disorders is the psychological intervention with the strongest support for its efficacy and effectiveness to date. Yet, it is the least used evidence-based intervention in routine clinical practice, with a long-acknowledged public relations problem. Despite a wealth of research aimed at improving uptake of exposure, exposure's marketing and branding remains an untapped target. We first introduce principles from the marketing literature to propose that the field take steps toward a rebranding and repackaging of exposure therapy to support efforts to implement it widely. Second, we present preliminary data on clinician preferences for the use of alternative terminology developed to be more palatable and marketable - "Supported Approach of Feared Experiences - Cognitive Behavioral Therapy (SAFE-CBT)" - compared to traditional terminology. This initial survey indicated that most clinicians preferred use of the SAFE-CBT term when talking to patients, whereas only a minority preferred it for use among training clinicians. We conclude by discussing implications of these results for future efforts to implement exposure therapy more widely and set an agenda for future research in this space.

Iterative redesign of a caregiver-mediated intervention for use in educational settings

LAY ABSTRACT

Teachers often report concerns about behavior challenges in their students with autism spectrum disorder (ASD) in the school setting. Furthermore, teachers often report that they do not have adequate training in how to manage these challenging behaviors effectively. The RUBI program is an intervention initially developed for parents of children with ASD and co-occurring challenging behavior in clinic settings. The present project used school staff input to systematically redesign RUBI to be used with educators in schools. School staff gave input at multiple stages of development to ensure the adapted intervention was appropriate to use in a school setting. Responses were coded and analyzed to identify strengths and weaknesses of the RUBI manual in schools and adaptations were made accordingly. Scores of how appropriate, possible, likable, and usable RUBI would be in schools rose after the intervention was redesigned. The redesigned RUBIES manual may give school staff the tools they need to manage disruptive behaviors. In addition, collaborating with providers over multiple stages to redesign established interventions for new contexts may be a promising way to help bring research tools to practice in the future.

A Brief Online Implicit Bias Intervention for School Mental Health Clinicians

Clinician bias has been identified as a potential contributor to persistent healthcare disparities across many medical specialties and service settings. Few studies have examined strategies to reduce clinician bias, especially in mental healthcare, despite decades of research evidencing service and outcome disparities in adult and pediatric populations. This manuscript describes an intervention development study and a pilot feasibility trial of the Virtual Implicit Bias Reduction and Neutralization Training (VIBRANT) for mental health clinicians in schools-where most youth in the U.S. access mental healthcare. Clinicians (N = 12) in the feasibility study-a non-randomized open trial-rated VIBRANT as highly usable, appropriate, acceptable, and feasible for their school-based practice. Preliminarily, clinicians appeared to demonstrate improvements in implicit bias knowledge, use of bias-management strategies, and implicit biases (as measured by the Implicit Association Test [IAT]) post-training. Moreover, putative mediators (e.g., clinicians' VIBRANT strategies use, IAT D scores) and outcome variables (e.g., clinician-rated quality of rapport) generally demonstrated correlations in the expected directions. These pilot results suggest that brief and highly scalable online interventions such as VIBRANT are feasible and promising for addressing implicit bias among healthcare providers (e.g., mental health clinicians) and can have potential downstream impacts on minoritized youth's care experience.

Human-centered design methods to achieve preparation phase goals in the multiphase optimization strategy framework

Background

The public health impact of behavioral and biobehavioral interventions to prevent and treat mental health and substance use problems hinges on developing methods to strategically maximize their effectiveness, affordability, scalability, and efficiency.

Methods

The multiphase optimization strategy (MOST) is an innovative, principled framework that guides the development of multicomponent interventions. Each phase of MOST (Preparation, Optimization, Evaluation) has explicit goals and a range of appropriate research methods to achieve them. Methods for attaining Optimization and Evaluation phase goals are well-developed. However, methods used in the Preparation phase are often highly researcher-specific, and concrete ways to achieve Preparation phase goals are a priority area for further development.

Results

We propose that the discover, design, build, and test (DDBT) framework provides a theory-driven and methods-rich roadmap for achieving the goals of the Preparation phase of MOST, including specifying the conceptual model, identifying and testing candidate intervention components, and defining the optimization objective. The DDBT framework capitalizes on strategies from the field of human-centered design and implementation science to drive its data collection methods.

Conclusions

MOST and DDBT share many conceptual features, including an explicit focus on implementation determinants, being iterative and flexible, and designing interventions for the greatest public health impact. The proposed synthesized DDBT/MOST approach integrates DDBT into the Preparation phase of MOST thereby providing a framework for rigorous and efficient intervention development research to bolster the success of intervention optimization.

Plain Language Summary

1. What is already known about the topic? Optimizing behavioral interventions to balance effectiveness with affordability, scalability, and efficiency requires a significant investment in intervention development.2. What does this paper add? This paper provides a structured approach to integrating human-centered design principles into the Preparation phase of the multiphase optimization strategy (MOST).3. What are the implications for practice, research, or policy? The proposed synthesized model provides a framework for rigorous and efficient intervention development research in the Preparation phase of MOST that will ensure the success of intervention optimization and contribute to improving public health impact of mental health and substance use interventions.

The Cognitive Walkthrough for Implementation Strategies (CWIS): a pragmatic method for assessing implementation strategy usability

BACKGROUND

Implementation strategies have flourished in an effort to increase integration of research evidence into clinical practice. Most strategies are complex, socially mediated processes. Many are complicated, expensive, and ultimately impractical to deliver in real-world settings. The field lacks methods to assess the extent to which strategies are usable and aligned with the needs and constraints of the individuals and contexts who will deliver or receive them. Drawn from the field of human-centered design, cognitive walkthroughs are an efficient assessment method with potential to identify aspects of strategies that may inhibit their usability and, ultimately, effectiveness. This article presents a novel walkthrough methodology for evaluating strategy usability as well as an example application to a post-training consultation strategy to support school mental health clinicians to adopt measurement-based care.

METHOD

The Cognitive Walkthrough for Implementation Strategies (CWIS) is a pragmatic, mixed-methods approach for evaluating complex, socially mediated implementation strategies. CWIS includes six steps: (1) determine preconditions; (2) hierarchical task analysis; (3) task prioritization; (4) convert tasks to scenarios; (5) pragmatic group testing; and (6) usability issue identification, classification, and prioritization. A facilitator conducted two group testing sessions with clinician users (N = 10), guiding participants through 6 scenarios and 11 associated subtasks. Clinicians reported their anticipated likelihood of completing each subtask and provided qualitative justifications during group discussion. Following the walkthrough sessions, users completed an adapted quantitative assessment of strategy usability.

RESULTS

Average anticipated success ratings indicated substantial variability across participants and subtasks. Usability ratings (scale 0-100) of the consultation protocol averaged 71.3 (SD = 10.6). Twenty-one usability problems were identified via qualitative content analysis with consensus coding, and classified by severity and problem type. High-severity problems included potential misalignment between consultation and clinical service timelines as well as digressions during consultation processes.

CONCLUSIONS

CWIS quantitative usability ratings indicated that the consultation protocol was at the low end of the "acceptable" range (based on norms from the unadapted scale). Collectively, the 21 resulting usability issues explained the quantitative usability data and provided specific direction for usability enhancements. The current study provides preliminary evidence for the utility of CWIS to assess strategy usability and generate a blueprint for redesign.