Empirical Examinations of Modifications and Adaptations to Evidence-Based Psychotherapies: Methodologies, Impact, and Future Directions

The Interrater Reliability of a Coding System for Measuring Mental Health Professionals' Decisions and Actions

OBJECTIVE

The clinical decisions and actions of evidence-based practice in psychology (EBPP) are largely underspecified and poorly understood, in part due to the lack of measurement methods. We tested the reliability of a behavioral coding system that characterizes a flow of interrelated activities that includes problem detection and prioritization, intervention selection and implementation, and review of intervention integrity and impact.

METHOD

The context included two publicly funded youth mental health service organizations located in geographically distinct and underresourced communities in the U.S. where service inequities are common. We sampled 84 digitally recorded and transcribed supervision events that included a sample of professionals who were mostly women (93.02%) and BIPOC (86.04%) whose self-reported race/ethnicity matched the youth populations they served. We coded these events for activities (e.g., considering) and their predicate content (i.e., problems or practices) and examined reliability of these codes applied to excerpts (i.e., small contiguous units of dialogue) as well as to complete events.

RESULTS

Interrater reliability estimates showed that, overall, coders reliably rated the occurrence and extensiveness of activities and content. Excerpt coding was generally more reliable than event coding. However, mathematical aggregation of excerpt coding offered a superior method for estimating event codes reliably, reducing individual subjectivity while providing event level synthesis of activities that are grounded in excerpt level details.

CONCLUSIONS

The assessment of clinical decisions and actions has the potential to unpack the black box of EBPP, with different methods best suited to different research questions and resource considerations.

Application of the FRAME-IS to a multifaceted implementation strategy

BACKGROUND

Research demonstrates the importance of documenting adaptations to implementation strategies that support integration of evidence-based interventions into practice. While studies have utilized the FRAME-IS [Framework for Reporting Adaptations and Modifications for Implementation Strategies] to collect structured adaptation data, they are limited by a focus on discrete implementation strategies (e.g., training), which do not reflect the complexity of multifaceted strategies like practice facilitation. In this paper, we apply the FRAME-IS to our trial evaluating the effectiveness of PF on implementation fidelity of an evidence-based technology-facilitated team care model for improved hypertension control within a federally qualified health center (FQHC).

METHODS

Three data sources are used to document adaptations: (1) implementation committee meeting minutes, (2) narrative reports completed by practice facilitators, and (3) structured notes captured on root cause analysis and Plan-Do-Study-Act worksheets. Text was extracted from the data sources according to the FRAME-IS modules and inputted into a master matrix for content analysis by two authors; a third author conducted member checking and code validation.

RESULTS

We modified the FRAME-IS to include part 2 of module 2 (what is modified) to add greater detail of the modified strategy, and a numbering system to track adaptations across the modules. This resulted in identification of 27 adaptations, of which 88.9% focused on supporting practices in identifying eligible patients and referring them to the intervention. About half (52.9%) of the adaptations were made to modify the context of the PF strategy to include a group-based format, add community health workers to the strategy, and to shift the implementation target to nurses. The adaptations were often widespread (83.9%), affecting all practices within the FQHC. While most adaptations were reactive (84.6%), they resulted from a systematic process of reviewing data captured by multiple sources. All adaptations included the FQHC in the decision-making process.

CONCLUSION

With modifications, we demonstrate the ability to document our adaptation data across the FRAME-IS modules, attesting to its applicability and value for a range of implementation strategies. Based on our experiences, we recommend refinement of tracking systems to support more nimble and practical documentation of iterative, ongoing, and multifaceted adaptations.

TRIAL REGISTRATION

Clinicaltrials.gov NCT03713515, Registration date: October 19, 2018.

Responding to Key Process Markers as a Focus of Psychotherapy Training and Practice

Historically, evidence-based psychotherapy training has favored the standardized application of discrete treatment packages, with key outcomes being the therapist's adherence to and competent delivery of theory-prescribed ingredients. However, this model often fails to align with the priorities and values of clinicians, and research casts doubt on the notion that a therapist's faithful application of treatment protocols is a valid index of clinical expertise. Considering this, training and practice models that emphasize evidence-based clinician flexibility and patient-centered tailoring of interventions are receiving increased attention. In this article, we outline one such model informed by the context-responsive psychotherapy integration (CRPI) framework. Consistent with CRPI principles, we describe several "if this/then try that" marker-response sequences that could become a centerpiece of a more nuanced, clinically representative, and evidence-based psychotherapy training paradigm. Finally, we offer several recommendations for future work on CRPI.

Getting More Out of Clinical Documentation: Can Clinical Dashboards Yield Clinically Useful Information?

This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.

A systematic review of written exposure therapy for the treatment of posttraumatic stress symptoms

OBJECTIVE

Written exposure therapy (WET) is a brief treatment for posttraumatic stress disorder (PTSD), with an increasing number of studies published over the past several years. The current study conducted a systematic review to evaluate the current state of evidence for WET as a treatment for PTSD symptom severity.

METHOD

Four databases were searched: PsycInfo, PTSDpubs, MEDLINE, and PubMed. Inclusion criteria included a peer-reviewed study of WET, a PTSD treatment outcome measure at pre- and posttreatment, and full-text available in English.

RESULTS

Seventeen studies were identified for inclusion, seven of which were randomized controlled trials. Study sample sizes ranged from three to 277, with most studies (88%) examining adults. Five studies used a language translation version of WET, two studies examined a group format, and three studies examined PTSD symptom outcome when WET was delivered via telehealth. Within condition effect sizes for PTSD treatment outcome were moderate to large (d range = 0.48-6.45), and between condition effect sizes were large (d range = 1.05-5.25), except for three studies that included a trauma-focused treatment comparison condition (d range = 0.01-0.31). Dropout rates for WET were generally low, and less when compared with other trauma-focused treatments.

CONCLUSIONS

The published studies indicate that WET is an efficacious and effective treatment for PTSD symptoms across a variety of samples, settings, and counties. Future work in this area should include investigation of the implementation and dissemination of WET. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Implementing interventions to start HPV vaccination at age 9: Using the evidence we have

Human papillomavirus (HPV) vaccination is routinely recommended for adolescents aged 11 or 12 years but can begin at age 9. On-time HPV vaccination by the thirteenth birthday has proven to be effective in preventing HPV cancer and pre-cancer. However, HPV coverage rates continue to lag behind other routinely recommended vaccinations for adolescents. A promising approach to improving coverage is to start HPV vaccination at age 9. This approach has been endorsed by the American Academy of Pediatrics and the American Cancer Society. Benefits of this approach include increased time to complete vaccination series by the thirteenth birthday, additional spacing of recommended vaccines, and a more concentrated focus on cancer prevention messaging. While promising, little is known about how and if existing evidence-based interventions and approaches can be used to promote starting HPV vaccination at age 9. Implementation science frameworks offer scientific direction in how to adapt current and develop new interventions to promote starting HPV vaccination at age 9 and accelerate dissemination and prevent HPV cancers.

Adapting and scaling a proven diabetes prevention program across 11 worksites in India: the INDIA-WORKS trial

BACKGROUND

Structured lifestyle change education reduces the burden of cardiometabolic diseases such as diabetes. Delivery of these programs at worksites could overcome barriers to program adoption and improve program sustainability and reach; however, tailoring to the worksite setting is essential.

METHODS

The Integrating Diabetes Prevention in Workplaces (INDIA-WORKS) study tested the implementation and effectiveness of a multi-level program for reducing cardiometabolic disease risk factors at 11 large and diverse worksites across India. Herein, we describe and classify program adaptations reported during in-depth interviews and focus group discussions with worksite managers, program staff, and peer educators involved in program delivery, and program participants and drop-outs. We used thematic analysis to identify key themes in the data and classified reported program adaptations using the FRAME classification system.

RESULTS

Adaptations were led by worksite managers, peer educators, and program staff members. They occurred both pre- and during program implementation and were both planned (proactive) and unplanned (proactive and reactive). The most frequently reported adaptations to the individual-level intervention were curriculum changes to tailor lessons to the local context, make the program more appealing to the workers at the site, or add a wider variety of exercise options. Other content adaptations included improvements to the screening protocol, intervention scheduling, and outreach plans to tailor participant recruitment and retention to the sites. Environment-level content adaptations included expanding or leveraging healthy food and exercise options at the worksites. Challenges to adaptation included scheduling and worksite-level challenges. Participants discussed the need to continue adapting the program in the future to continue making it relevant for worksite settings and engaging for employees.

CONCLUSION

This study describes and classifies site-specific modifications to a structured lifestyle change education program with worksite-wide health improvements in India. This adds to the literature on implementation adaptation in general and worksite wellness in India, a country with a large and growing workforce with, or at risk of, serious cardiometabolic diseases. This information is key for program scale-up, dissemination, and implementation in other settings.

TRIAL REGISTRATION

Clinicaltrials.gov NCT02813668. Registered June 27, 2016.

Assessing ad-hoc adaptations' alignment with therapeutic goals: a qualitative study of lay counselor-delivered family therapy in Eldoret, Kenya

BACKGROUND

A key question in implementation science is how to balance adaptation and fidelity in translating interventions to new settings. There is growing consensus regarding the importance of planned adaptations to deliver interventions in contextually sensitive ways. However, less research has examined ad-hoc adaptations, or those that occur spontaneously in the course of intervention delivery. A key question is whether ad-hoc adaptations ultimately contribute to or detract from intervention goals. This study aimed to (a) identify ad-hoc adaptations made during delivery of a family therapy intervention and (b) assess whether they promoted or interrupted intervention goals.

METHODS

Tuko Pamoja (Swahili: "We are Together") is an evidence-informed family therapy intervention aiming to improve family dynamics and mental health in Kenya. Tuko Pamoja employs a task-shifting model, delivered by lay counselors who are afforded a degree of flexibility in presenting content and in practices they use in sessions. We used transcripts of therapy sessions with 14 families to examine ad-hoc adaptations used by counselors. We first identified and characterized ad-hoc adaptations through a team-based code development, coding, and code description process. Then, we evaluated to what extent ad-hoc adaptations promoted the principles and strategies of the intervention ("TP-promoting"), disrupted them ("TP-interrupting"), or neither ("TP-neutral"). To do this, we first established inter-coder agreement on application of these categories with verification by the intervention developer. Then, coders categorized ad-hoc adaptation text segments as TP-promoting, TP-interrupting, or TP-neutral.

RESULTS

Ad-hoc adaptations were frequent and included (in decreasing order): incorporation of religious content, exemplars/role models, community dynamics and resources, self-disclosure, and metaphors/proverbs. Ad-hoc adaptations were largely TP-promoting (49%) or neutral (39%), but practices were TP-interrupting 12% of the time. TP-interrupting practices most often occurred within religious content and exemplars/role models, which were also the most common practices overall.

CONCLUSION

Extra attention is needed during planned adaptation, training, and supervision to promote intervention-aligned use of common ad-hoc adaptation practices. Discussing them in trainings can provide guidance for lay providers on how best to incorporate ad-hoc adaptations during delivery. Future research should evaluate whether well-aligned ad-hoc adaptations improve therapeutic outcomes.

TRIAL REGISTRATION

Pilot trial registered at clinicaltrials.gov (C0058).

Adapting family-based treatment for adolescent anorexia nervosa delivered in the home: A novel approach for improving access to care and generalizability of skill acquisition

BACKGROUND

Anorexia nervosa (AN) is a serious mental illness associated with high rates of morbidity and mortality. Family-based treatment (FBT) is a well-established treatment for adolescent AN, yet it is underutilized in community settings and is unavailable to many families, particularly those from lower income and racial and ethnic minority backgrounds. Furthermore, some families do not respond optimally to FBT, possibly because of challenges translating skills acquired in office-based treatment settings to naturalistic settings. Home-based treatment could reduce barriers to access and enhance generalization of newly learned treatment skills. Home-based models demonstrate initial feasibility, acceptability, and efficacy for adolescent AN, however, FBT principles have yet to be applied as a stand-alone intervention in a home-based level of care. This paper describes the rationale for and process of adapting FBT principles/interventions to improve fit within a home-based model delivered in the context of community mental health, and discusses potential strengths and opportunities associated with this approach.

RESULTS

Adaptations were made through consultation with collaborating community agencies and were guided by the complex interventions framework. The primary modifications included: (1) altered dose; (2) multiple family meals; (3) additional support for meal preparation and supervision; (4) clinician attendance at medical appointments; (5) cultural adaptation; and (6) introduction of distress tolerance and emotion regulation skills.

CONCLUSIONS

Implementing FBT in the home may present one promising and novel approach to enhance engagement and treatment outcomes for adolescents with restrictive eating disorders, particularly those who are underserved, but evaluation of efficacy/effectiveness is needed.

Evidence-Based Intervention Adaptations Within the Veterans Health Administration: a Scoping Review

BACKGROUND

Veterans receiving care within the Veterans Health Administration (VA) are a unique population with distinctive cultural traits and healthcare needs compared to the civilian population. Modifications to evidence-based interventions (EBIs) developed outside of the VA may be useful to adapt care to the VA healthcare system context or to specific cultural norms among veterans. We sought to understand how EBIs have been modified for veterans and whether adaptations were feasible and acceptable to veteran populations.

METHODS

We conducted a scoping review of EBI adaptations occurring within the VA at any time prior to June 2021. Eligible articles were those where study populations included veterans in VA care, EBIs were clearly defined, and there was a comprehensive description of the EBI adaptation from its original context. Data was summarized by the components of the Framework for Reporting Adaptations and Modifications to Evidence-based interventions (FRAME).

FINDINGS

We retrieved 922 abstracts based on our search terms. Following review of titles and abstracts, 49 articles remained for full-text review; eleven of these articles (22%) met all inclusion criteria. EBIs were adapted for mental health (n = 4), access to care and/or care delivery (n = 3), diabetes prevention (n = 2), substance use (n = 2), weight management (n = 1), care specific to cancer survivors (n = 1), and/or to reduce criminal recidivism among veterans (n = 1). All articles used qualitative feedback (e.g., interviews or focus groups) with participants to inform adaptations. The majority of studies (55%) were modified in the pre-implementation, planning, or pilot phases, and all were planned proactive adaptations to EBIs.

IMPLICATIONS FOR D&I RESEARCH

The reviewed articles used a variety of methods and frameworks to guide EBI adaptations for veterans receiving VA care. There is an opportunity to continue to expand the use of EBI adaptations to meet the specific needs of veteran populations.

Introduction to the Special Issue: Advancing Racial Justice in Clinical Child and Adolescent Psychology

Relative to White youth, racially and ethnically marginalized youth in the U.S. are less likely to initiate treatment, stay in treatment, and receive adequate care. This special issue attends to racial injustice in clinical child and adolescent psychology. While numerous factors drive these racial disparities, this special issue focuses specifically on opportunities and responsibilities we have as mental health providers, teachers, mentors, researchers, and gatekeepers to make our field more racially just. In this introduction to the special issue, we review barriers and solutions across multiple contexts including structural, institutional, and practice-based. We also discuss challenges and opportunities to diversify our field and increase the representation of racially and ethnically marginalized practitioners and scholars in clinical child and adolescent psychology. We then briefly review the special issue articles and make final recommendations for how to move the field forward.

Optimizing ATTAIN implementation in a federally qualified health center guided by the FRAME-IS

Implementation strategies are methods or techniques used to adopt, implement, and sustain evidence-based practices (EBPs). Implementation strategies are dynamic and may require adaptation to fit implementation contexts, especially in low-resource settings, which are most likely to serve racially and ethnically diverse patients. The framework for reporting adaptations and modifications to evidence-based implementation strategies (FRAME-IS) was used to document adaptations to implementation strategies to inform an optimization pilot of Access to Tailored Autism Integrated Care (ATTAIN; an integrated care model for children with autism and co-occurring mental health needs) in a federally qualified health center (FQHC) near the United States/Mexico border. Quantitative and qualitative data were collected from 36 primary care providers who participated in the initial ATTAIN feasibility pilot to inform adaptations. Adaptations were mapped to the FRAME-IS through an iterative template analysis to inform an optimization pilot at a FQHC 1-year from the start of the COVID-19 pandemic. Four implementation strategies (training and workflow reminders, provider/clinic champions, periodic reflections, and technical assistance) were employed during the feasibility pilot and were adapted for the optimization pilot to fit the needs of the FQHC and service delivery changes prompted by the pandemic. Findings demonstrate the utility of using the FRAME-IS to systematically inform EBP optimization in a FQHC providing care to underserved communities. Findings will inform future research studies implementing integrated mental health models in low-resourced primary care settings. Implementation outcomes and provider perceptions of ATTAIN at the FQHC are also reported. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Implementation of a community health worker-focused team-based model of care: What modifications do clinics make?

Background

Team-based care offers potential for integrating non-clinicians, such as community health workers (CHWs), into the primary care team to ensure that patients and families receive culturally relevant care to address their physical, social, and behavioral health and wellness needs. We describe how two federally qualified health center (FQHC) organizations adapted an evidence-based, team-based model of well-child care (WCC) designed to ensure that the parents of young children, aged 0-3, have their comprehensive preventive care needs met at WCC visits.

Methods

Each FQHC formed a Project Working Group composed of clinicians, staff, and parents to determine what adaptations to make in the process of implementation of PARENT (Parent-Focused Redesign for Encounters, Newborns to Toddlers), a team-based care intervention that uses a CHW in the role of a preventive care coach. We use the Framework for Reporting Adaptations and Modifications to Evidence-based interventions (FRAME) to chronicle the various intervention modifications and the adaptation process, focusing on when and how modifications occurred, whether it was planned or unplanned, and the reasons and goals for the modification.

Results

The Project Working Groups adapted several elements of the intervention in response to clinic priorities, workflow, staffing, space, and population need. Modifications were planned and proactive, and were made at the organization, clinic, and individual provider level. Modification decisions were made by the Project Working Group and operationalized by the Project Leadership Team. Examples of modifications include the following: (1) changing the parent coach educational requirement from a Master's degree to a bachelor's degree or equivalent experience to reflect the needs of the coach role; (2) the use of FQHC-specific templates for the coach's documentation of the pre-visit screening in the electronic health record; and (3) the use of electronic social needs referral tools to help the coach track and follow up on social need referrals. The modifications did not change the core elements (i.e., parent coach provision of preventive care services) or intervention goals.

Conclusions

For clinics implementing team-based care interventions, the engagement of key clinical stakeholders early and often in the intervention adaptation and implementation process, and planning for intervention modifications at both at an organizational level and at a clinical level are critical for local implementation.

Scaling Out Virtual Interview Training for Transition-Age Youth: A Quasi-Experimental Hybrid Effectiveness-Implementation Study

In a previous randomized controlled trial (RCT), Virtual Interview Training for Transition Age Youth (VIT-TAY) enhanced employment for autistic youth receiving transition services. For this study, a non-randomized hybrid effectiveness-implementation trial evaluated whether VIT-TAY maintained its core implementation and effectiveness functions when scaled out to students with disabilities in 32 schools. Also, we compared the implementation and effectiveness between VIT-TAY and Virtual Reality Job Interview Training (VR-JIT; developed for adults and previously evaluated in students with disabilities). Core implementation functions did not differ between VIT-TAY and VR-JIT. Greater engagement with either training was significantly associated with employment (both p < 0.05). Overall, VIT-TAY was feasibly implemented with high adherence and may be effective at enhancing employment for students with disabilities. Limitations and implications for research and practice are discussed.

Electronic health record closed-loop referral ("eReferral") to a state tobacco quitline: a retrospective case study of primary care implementation challenges and adaptations

BACKGROUND

Health system change can increase the reach of evidence-based smoking cessation treatments. Proactive electronic health record (EHR)-enabled, closed-loop referral ("eReferral") to state tobacco quitlines increases the rates at which patients who smoke accept cessation treatment. Implementing such system change poses many challenges, however, and adaptations to system contexts are often required, but are understudied. This retrospective case study identified adaptations to eReferral EHR tools and implementation strategies in two healthcare systems.

METHODS

In a large clustered randomized controlled trial (C-RCT; NCT02735382) conducted in 2016-2017, 11 primary care clinics in two healthcare systems implemented quitline eReferral, starting with 1 pilot clinic per system followed by 2 phases of implementation (an experimental phase in 5-6 test clinics per system and then a system-wide dissemination phase in both systems). Adaptations were informed by stakeholder input from live trainings, follow-up calls and meetings in the first month after eReferral launch, emails, direct observation by researchers, and clinic staff survey responses. Retrospective, descriptive analysis characterized implementation strategy modifications and adaptations using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS). A pre- and post-implementation survey assessed staff ratings of eReferral acceptability and implementation barriers and facilitators.

FINDINGS

Major modifications to closed-loop eReferral implementation strategies included aligning the eReferral initiative with other high-priority health system objectives, modifying eReferral user interfaces and training in their use, modifying eReferral workflows and associated training, and maintaining and enhancing interoperability and clinician feedback functions. The two health systems both used Epic EHRs but used different approaches to interfacing with the quitline vendor and integrating eReferral into clinician workflows. Both health systems engaged in iterative refinement of the EHR alert prompting eReferral, the eReferral order, trainings, and workflows. Staff survey comments suggested moderate acceptability of eReferral processes and identified possible targets for future modifications in eReferral, including reducing clinician burden related to EHR documentation and addressing clinicians' negative beliefs about patient receptivity to cessation treatment.

CONCLUSIONS

System-wide implementation of tobacco quitline eReferral in primary care outpatient clinics is feasible but requires extensive coordination across stakeholders, tailoring to local health system EHR configurations, and sensitivity to system- and clinic-specific workflows.

TRIAL REGISTRATION

www.

CLINICALTRIALS

gov, NCT02735382 . Registered on 12 August 2016.

How Flexible are Treatments for Youth Internalizing Disorders? Examining Modification Guidelines Included across Supported Treatments

OBJECTIVE

Treatment protocols for youth-internalizing disorders have been developed, however these protocols have yielded mixed findings in routine care settings. Despite increased recognition of the importance of flexibility when delivering evidence-based treatments (EBTs), little is known about the extent to which protocols offer guidance to providers in flexible EBT implementation. The current study examined the extent to which supported EBTs for youth internalizing disorders explicitly incorporate guidance for treatment modification.

METHODS

Supported treatment protocols for youth internalizing disorders were identified (N = 44), from which 4,021 modification guidelines were extracted and coded using a structured coding system to classify modification strategies (i.e., the forms that recommended modifications take), and associated tailoring factors (i.e., the rationale for which modifications are recommended).

RESULTS

Across all EBTs, modification guidelines were quite common, with the average protocol including almost 91 text passages providing guidance for modification. The majority of modification guidelines functionally increase session or treatment length by recommending the addition or repetition of material, whereas less than 5% of modification guidelines provided strategies for condensing or streamlining care. Strikingly, less than 2% of modification guidelines in EBT protocols address patient cultural factors, and rarely address provider or setting issues that can challenge standard implementation.

CONCLUSIONS

Findings highlight critical gaps in the available guidance to modify EBTs for youth internalizing disorders, and suggest EBT protocols may not be optimally poised to flexibly address the broad diversity of children and adolescents across varied settings in need of mental health care.

A Mixed-Methods Study of Clinician Adaptations to Parent-Child Interaction Therapy - What about Culture?

Parent-Child Interaction Therapy (PCIT) is an evidence-based practice (EBP) for young children with challenging behaviors. PCIT has been adapted to treat varying presentations and culturally diverse families. Although efforts have been made to disseminate PCIT into community settings, which often serve clinically complex, socio-culturally diverse, and marginalized communities, barriers to disseminating adapted models remain. An alternative strategy to understanding how to increase access to appropriately adapted PCIT is to learn from community clinicians' practice-based adaptations to meet their clients' diverse needs related to clinical presentation, culture, and language. This mixed-method study investigated community clinician adaptations of PCIT. Clinicians (N = 314) were recruited via PCIT listservs to complete a survey collecting background information, and adaptations to PCIT. Most clinicians had a master's degree (72.1%), were licensed (74.2%), and were PCIT-certified (70.7%). Qualitative interviews were conducted with a purposeful sample of 23 community clinicians, who were 39% Spanish-speaking, were 30% Latinx, and 30% reported serving a ≥50% Latinx clientele. Clinicians reported engaging in adaptations aimed at augmenting PCIT more extensively than adaptations involving removing core components. Themes from qualitative interviews converged with quantitative findings, with clinicians most frequently describing augmenting adaptations, and highlighted reasons for adapting PCIT. Clinicians primarily augmented treatment to address clients' clinical presentations. Clinicians rarely adapted treatment specifically for culture, but when mentioned, clinicians discussed tailoring idioms and phrases to match clients' culture for Spanish-speaking clients. Implications for training PCIT clinicians in intervention adaptations will be discussed.

Implementing School-Based Mental Health Services: A Scoping Review of the Literature Summarizing the Factors That Affect Implementation

Background: Mental illness in children and youths has become an increasing problem. School-based mental health services (SBMHS) are an attempt to increase accessibility to mental health services. The effects of these services seem positive, with some mixed results. To date, little is known about the implementation process of SBMHS. Therefore, this scoping review synthesizes the literature on factors that affect the implementation of SBMHS. Methods: A scoping review based on four stages: (a) identifying relevant studies; (b) study selection; (c) charting the data; and (d) collating, summarizing, and reporting the results was performed. From the searches (4414 citations), 360 were include in the full-text screen and 38 in the review. Results: Implementation-related factors were found in all five domains of the Consolidated Framework for Implementation Research. However, certain subfactors were mentioned more often (e.g., the adaptability of the programs, communication, or engagement of key stakeholders). Conclusions: Even though SBMHS differed in their goals and way they were conducted, certain common implementation factors were highlighted more frequently. To minimize the challenges associated with these types of interventions, learning about the implementation of SBMHS and using this knowledge in practice when introducing SBMHS is essential to achieving the best possible effects with SMBHSs.

The Urban Youth Trauma Center: A Trauma-Informed Continuum for Addressing Community Violence Among Youth

Contemporary community violence has escalated into a national public health crisis with urban youth, particularly ethnic minorities, suffering disproportionate negative impacts. The Urban Youth Trauma Center (UYTC) promotes a trauma-informed continuum of prevention-to-intervention services that combines community-based and clinic-based manualized protocols designed to reduce and prevent community violence for youth and families. Based on a socio-ecological model, UYTC has the main goals of addressing community violence and related traumatic stress as well as co-occurring conditions of substance abuse and disruptive behavior problems in urban youth by: (1) raising public awareness; (2) disseminating specialized trauma-informed training; and (3) mobilizing service system coalitions. UYTC employs this evidence-based yet flexible structure for disseminating, implementing, and evaluating trauma-informed training as a means of contributing to the reduction and prevention of community violence for low-income urban minority youth and families who bear the biggest burden of this current crisis.

Planning implementation and scale-up of physical activity interventions for people with walking difficulties: study protocol for the process evaluation of the ComeBACK trial

Background

There is currently little evidence of planning for real-world implementation of physical activity interventions. We are undertaking the ComeBACK (Coaching and Exercise for Better Walking) study, a 3-arm hybrid Type 1 randomised controlled trial evaluating a health coaching intervention and a text messaging intervention. We used an implementation planning framework, the PRACTical planning for Implementation and Scale-up (PRACTIS), to guide the process evaluation for the trial. The aim of this paper is to describe the protocol for the process evaluation of the ComeBACK trial using the framework of the PRACTIS guide.

Methods

A mixed methods process evaluation protocol was developed informed by the Medical Research Council (MRC) guidance on process evaluations for complex interventions and the PRACTIS guide. Quantitative data, including participant questionnaires, health coach and administrative logbooks, and website and text message usage data, is being collected over the trial period. Semi-structured interviews and focus groups with trial participants, health coaches and health service stakeholders will explore expectations, factors influencing the delivery of the ComeBACK interventions and potential scalability within existing health services. These data will be mapped against the steps of the PRACTIS guide, with reporting at the level of the individual, provider, organisational and community/systems. Quantitative and qualitative data will elicit potential contextual barriers and facilitators to implementation and scale-up. Quantitative data will be reported descriptively, and qualitative data analysed thematically.

Discussion

This process evaluation integrates an evaluation of prospective implementation and scale-up. It is envisaged this will inform barriers and enablers to future delivery, implementation and scale-up of physical activity interventions. To our knowledge, this is the first paper to describe the application of PRACTIS to guide the process evaluation of physical activity interventions.

Trial registration

Australian and New Zealand Clinical Trials Registry (ANZCTR) Registration date: 10/12/2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05990-3.

Therapist adaptations to evidence-based practices and associations with implementation outcomes in child therapy sessions

OBJECTIVE

Community therapists inevitably adapt evidence-based practices (EBPs) to meet the needs of their clients and practice settings. Yet, the implications of spontaneous, therapist-driven adaptations for EBP implementation outcomes are not well understood. We used a sequential QUAN → qual mixed-methods design to examine how different types of therapist-described adaptations were associated with observer-rated extensiveness of therapist delivery of EBP content and technique strategies at the session level.

METHOD

Data were drawn from an observational study of a system-driven implementation of multiple EBPs into public children's mental health services. Community therapists (n = 103) described adaptations they made in 680 sessions with 273 clients (50.92% female, 49.08% male, M_age = 9.72 years, 70.70% Hispanic/Latinx). Coders classified therapist-described adaptations into five types: (a) Modifying Presentation, (b) Integrating, (c) Extending, (d) Reducing, and (e) Generalizing. Independent observers rated the extensiveness of EBP strategy delivery from session recordings using the EBP Concordant Care Assessment (ECCA) Observational Coding System.

RESULTS

Quantitative analyses using multilevel regression revealed that Modifying Presentation adaptations were associated with higher extensiveness of EBP technique delivery, whereas Extending adaptations were associated with lower extensiveness of EBP content and technique delivery. Qualitative analysis of adaptation descriptions identified explanations for the quantitative findings.

CONCLUSIONS

Findings suggest that Modifying Presentation adaptations, associated with higher extensiveness, involved creative use of activities and materials, language modification, and personalization of EBP content to meet clients' diverse needs, whereas Extending adaptations, associated with lower extensiveness, involved slowing EBP pacing in response to client challenges. Implications for provider training are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Practitioner use of and attitudes towards videoconferencing for the delivery of evidence-based telemental health interventions: A mixed methods study

The implementation of evidence-based psychosocial interventions using video-conference delivery (VCD) has the potential to increase accessibility to effective treatments, although its use remains limited and understudied. This study employed a mixed methods approach in surveying mental health practitioners about their attitudes regarding VCD of interventions that are considered evidence-based (i.e., have been shown to improve targeted outcomes in rigorous research). One hundred and eleven practitioners were sampled from several national and regional U.S. practice organizations and were administered quantitative surveys about their use of and attitudes towards VCD of evidence-based interventions (EBI). We examined the relationship between practitioner-level technology access, experience, and training with technology fluency and acceptability of using VCD. Quantitative results indicated the most frequently used adaptation for VCD was Tailoring and that practitioner education predicted attitudes towards EBIs. A subset (n = 20) of respondents were then purposively selected for qualitative interviews to further investigate accessibility, appropriateness, and feasibility of delivering EBIs via video conference. A conventional content analysis revealed that VCD was appropriate and acceptable for EBIs; however, many practitioners also described barriers related to feasibility of implementation. The results of this study have important implications for telemental health dissemination efforts which seek to extend services to populations not served well by traditional, in-person mental health services.

Implementation case study: Multifamily group intervention in first-episode psychosis programs

AIM

Family interventions are a core component of first-episode psychosis (FEP) treatment; however, low implementation rates are consistently reported. As such, work is needed to understand the factors impacting real-world treatment delivery. The present paper describes the implementation of the McFarlane-model multifamily psychoeducational groups (MFG) in established FEP early intervention programs within a single state. The aims were to examine (a) training participation and implementation of MFG, (b) barriers and facilitators to implementation, and (c) modifications made to MFG.

METHODS

Practitioners from six established FEP early intervention programs received in-person training and ongoing consultation in MFG. Training participation data were obtained via attendance and implementation outcomes were obtained from practitioner reports. Fifteen months following the initial training, practitioners reported on clinic-specific barriers, facilitators, and modifications across four categories (context, intervention, practitioner, and recipient).

RESULTS

Twenty-three practitioners across six clinics received in-person training and were offered ongoing consultation to support implementation. Difficulties in starting MFG were salient as the earliest group was run 7 months after the initial training, thereby resulting in low overall frequency of groups. A number of barriers spanning context, intervention, practitioner, and recipient domains were noted, the majority of which were clinic-specific. Despite challenges, practitioners identified several facilitators and made modifications to the intervention and its delivery in service of implementation.

CONCLUSIONS

Results from this implementation case study highlighted the challenges of delivering MFG in real-world FEP early intervention programs. Further, this paper emphasizes the value in identifying and addressing clinic-specific factors when implementing MFG.

A Systematic Review of Mental Health Interventions for ASD: Characterizing Interventions, Intervention Adaptations, and Implementation Outcomes

Youth with autism spectrum disorder (ASD) have high rates of co-occurring mental health needs that necessitate mental health interventions. Given the unique clinical characteristics of youth with ASD, there have been significant efforts to adapt and test mental health interventions for this population. Yet, characterization of the nature and types of interventions adaptations is limited, especially across the wide range of interventions tested for youth with ASD with a focus on implementation factors. Additionally, understanding how these interventions may be implemented in community services is limited. The aims of this systematic review are to characterize the (1) types of interventions tested for co-occurring mental health conditions for youth with ASD; (2) adaptations to mental health interventions for use with youth with ASD; and (3) implementation strategies, outcomes, and determinants of mental health interventions to inform their translation to community service settings. Eighty-three articles testing interventions targeting mental health symptoms in youth with ASD that included implementation factors in analyses were reviewed. The Stirman et al. (2013; 2019) FRAME adaptation, Powell et al. (2012;2015) implementation strategies, and Proctor et al. (2011) implementation outcomes taxonomies were applied to characterize the nature and types of adaptations for use with youth with ASD and types of implementation strategies, outcomes, and determinants used, when available, respectively. Of the interventions examined, the majority (64.1%) were originally designed to target youth mental health concerns and were then adapted to be used with ASD. The most common adaptations included those to the intervention content, particularly adding elements with tailoring or refining aspects of the intervention while maintaining core functions. Half of the articles described at least one implementation strategy used during intervention testing. Fidelity and acceptability were the most frequently examined implementation outcomes, with some examination of appropriateness and feasibility. Nineteen percent of articles described implementation determinants (i.e. barriers/facilitators) of these implementation outcomes. The common adaptations for ASD provide direction for future intervention development and for training community therapists. Further examination, specification, and reporting of implementation strategies and outcomes within ongoing efforts to adapt and interventions to meet the co-occurring mental health needs of youth ASD are needed to facilitate their translation to community settings. Areas for future research as well as clinical implications are discussed.

Understanding the value of adhering to or adapting evidence-based interventions: a study protocol of a discrete choice experiment

Background

Whereas the value of an evidence-based intervention (EBI) is often determined by its effect on clinical outcomes, the value of implementing and using EBIs in practice is broader, reflecting qualities such as appropriateness, equity, costs, and impact. Reconciling these value conflicts involves a complicated decision process that has received very limited scholarly attention. Inspired by studies on decision-making, the objective of this project is to explore how practitioners appraise the values of different outcomes and to test how this appraisal influences their decisions surrounding the so-called fidelity–adaptation dilemma. This dilemma is related to the balance between using an EBI as it was designed (to ensure its effectiveness) and making appropriate adaptations (to ensure alignment with constraints and possibilities in the local context).

Methods

This project consists of three sub-studies. The participants will be professionals leading evidence-based parental programs in Sweden and, in Sub-study 1, parents and decision-makers. Sub-study 1 will use sequential focus groups and individual interviews to explore parameters that influence fidelity and adaptation decisions—the dilemmas encountered, available options, how outcomes are valued by practitioners as well as other stakeholders, and value trade-offs. Sub-study 2 is a discrete choice experiment that will test how value appraisals influence decision-making using data from Sub-study 1 as input. Sub-study 3 uses a mixed-method design, with findings from the two preceding sub-studies as input in focus group interviews to investigate how practitioners make sense of findings from optimal decision situations (experiment) and constrained, real-world decision situations.

Discussion

The project will offer unique insights into decision-making processes that influence how EBIs are used in practice. Such knowledge is needed for a more granular understanding of how practitioners manage the fidelity–adaptation dilemma and thus, ultimately, how the value of EBI implementation can be optimized. This study contributes to our knowledge of what happens once EBIs are adopted—that is, the gap between the way in which EBIs are intended to be used and the way in which they are used in practice.

Correction to: A Systematic Review of Mental Health Interventions for ASD: Characterizing Interventions, Intervention Adaptations, and Implementation Outcomes

Due to the errors occurred in the originally published version, this article is being reprinted in its entirety as Correction. All errors have been corrected. It is the correct version.

Situational analysis of breast health care systems: Why context matters

BACKGROUND

Implementation of evidence-based, resource-appropriate guidelines for breast cancer control should be preceded by a baseline assessment or situational analysis to assess breast health infrastructure, workforce capacity, patient pathways, existing practices, accessibility, and costs.

METHODS

To support the assessment of breast health care systems within the broader context in which they exist, the Breast Health Global Initiative (BHGI) developed, tested, and refined a set of situational analysis tools with which to guide the assessment of breast health care capacity, identify the relative strengths and weaknesses of the health system, and support stakeholders in prioritizing actionable items to advance breast cancer care using evidence-based strategies tailored to their setting. The tools address 6 domains of breast health care delivery: 1) breast cancer early detection practices; 2) breast cancer awareness programs; 3) the availability of breast cancer surgery; 4) the availability of pathology; 5) the availability of radiotherapy, and 6) the availability of systemic therapy services. The current study also describes the more comprehensive International Atomic Energy Agency Programme of Action for Cancer Therapy (PACT) integrated missions for PACT (imPACT) review.

RESULTS

As of 2020, 5 formal BHGI situational analyses have been performed in India, Brazil, Panama, Tanzania, and Uganda. As of August 2019, a total of 100 imPACT reviews have been conducted in 91 countries. These assessments can contribute to more informed policymaking.

CONCLUSIONS

Situational analyses are a prerequisite for the development of resource-appropriate strategies with which to advance breast cancer control in any setting and should assess services across the entire breast health care continuum as well as the broader structural, sociocultural, personal, and financial contexts within which they operate.

The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare

BACKGROUND

Implementation strategies are necessary to ensure that evidence-based practices are successfully incorporated into routine clinical practice. Such strategies, however, are frequently modified to fit local populations, settings, and contexts. While such modifications can be crucial to implementation success, the literature on documenting and evaluating them is virtually nonexistent. In this paper, we therefore describe the development of a new framework for documenting modifications to implementation strategies.

DISCUSSION

We employed a multifaceted approach to developing the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS), incorporating multiple stakeholder perspectives. Development steps included presentations of initial versions of the FRAME-IS to solicit structured feedback from individual implementation scientists ("think-aloud" exercises) and larger, international groups of researchers. The FRAME-IS includes core and supplementary modules to document modifications to implementation strategies: what is modified, the nature of the modification (including the relationship to core elements or functions), the primary goal and rationale for the modification, timing of the modification, participants in the modification decision-making process, and how widespread the modification is. We provide an example of application of the FRAME-IS to an implementation project and provide guidance on how it may be used in future work.

CONCLUSION

Increasing attention is being given to modifications to evidence-based practices, but little work has investigated modifications to the implementation strategies used to implement such practices. To fill this gap, the FRAME-IS is meant to be a flexible, practical tool for documenting modifications to implementation strategies. Its use may help illuminate the pivotal processes and mechanisms by which implementation strategies exert their effects.

Professionals' management of the fidelity-adaptation dilemma in the use of evidence-based interventions-an intervention study

Background

Evidence-based interventions (EBIs) can be effective tools for the prevention of disease and health promotion. However, their implementation often requires a delicate balance between the need to adjust the intervention to the context in which it is implemented and the need to keep the core components that make the intervention effective. This so-called dilemma between fidelity and adaptation is often handled by health professionals in the sustainment phase of an implementation (i.e., once the intervention has been adopted and institutionalized in an organization), but not much is known about how and to what extent health professionals are affected by this dilemma. Focusing on the sustainment phase, this project aims to study (1) how fidelity and adaptation are managed by professionals using an EBI, (2) how the fidelity–adaptation dilemma affects professionals’ psychosocial working conditions, and (3) how a structured decision support influences professionals’ management of the dilemma and their psychosocial working conditions.

Methods

The study is set in Sweden, and the EBI in focus is a parental program (All Children in Focus). A longitudinal within-person intervention design is used, combined with a cross-sectional survey design. Data sources include web-based questionnaires, brief interviews, fidelity ratings, paper-and-pen questionnaires, and written documentation, collected at multiple time points with both group leaders and parents as respondents.

Discussion

This project approaches fidelity and adaptation from the perspective of the professionals that manage EBIs during the sustainment phase of implementation. Although it is well known that EBIs continue to change over time, it remains to be understood how the fidelity–adaptation dilemma can be managed so that the effectiveness of interventions is retained or improved, not diluted. Moreover, the project adds to the literature by presenting an occupational health perspective on the fidelity–adaptation dilemma. It is acknowledged that fidelity and adaptation may have consequences for not only clients but also the occupational wellbeing of the professionals managing the dilemma, and subsequently, their willingness and ability to deliver EBIs in a sustainable way.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-021-00131-y.

Virtual Reality Job Interview Training in Transition Services: Results of a Single-Arm, Noncontrolled Effectiveness-Implementation Hybrid Trial

This study reports the results of a single-arm non-controlled, Type 3 hybrid effectiveness-implementation trial evaluating virtual reality job-interview training (VR-JIT) delivered in five pre-employment transition programs comprising 15 schools, 10 administrators, 23 teachers, and 279 youth ages 16-21 years receiving special education pre-employment transition services. Fidelity, expected implementation feasibility, and teacher and student acceptance of VR-JIT were high. Youth completed virtual interviews over six to eight weeks (M = 10.8, SD = 7.4). At the six-month follow-up, teachers reported that youth using VR-JIT had employment rates higher than current national employment rates for youth with disabilities. A multinomial logistic regression revealed VR-JIT engagement was associated with greater employment rates by six-month follow-up (OR=1.63, p=0.002). This study provides promising evidence that VR-JIT may be feasibly implemented with high fidelity in special education pre-employment transition services and can potentially enhance employment outcomes among transition-age youth receiving special education services.

Applying an Equity Lens to Characterizing the Process and Reasons for an Adaptation to an Evidenced-based Practice

BACKGROUND

Adaptations to evidence-based practices (EBPs) are common but can impact implementation and patient outcomes. In our prior research, providers in routine care made a fidelity-inconsistent adaptation to an EBP that improved health outcomes in people with serious mental illness (SMI). The purpose of this study was to characterize the process and reasons for the adaptation using a framework for reporting adaptations and modifications to EBPs, with a focus on equity.

METHODS

This study used qualitative data collected during a national implementation of the InSHAPE EBP addressing obesity in persons with SMI. We reviewed transcripts from five behavioral health organizations that made a successful fidelity-inconsistent adaptation to a core component of InSHAPE that was associated with cardiovascular risk reduction. We coded the data using the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) with an emphasis on exploring whether the adaptation addressed inequities in using the EBP related to social determinants of health.

RESULTS

Across the five agencies, the fidelity-inconsistent adaptation was characterized as unplanned and reactive in response to challenges InSHAPE teams experienced delivering the intervention in community fitness facilities as intended. In all cases, the goal of the adaptation was to improve intervention access, feasibility and fit. Social and economic disadvantage were noted obstacles to accessing fitness facilities or gyms among participants with SMI, which led agencies to adapt the program by offering sessions at the mental health center.

CONCLUSIONS

Findings from this study show the advantages of applying a health equity lens to evaluate how obstacles such as poverty and discrimination influence EBP adaptations. Recommendations can also assist researchers and community partners in making proactive decisions about allowable adaptations to EBPs.

Incorporators, Early Investors, and Learners: a longitudinal study of organizational adaptation during EBP implementation and sustainment

BACKGROUND

The majority of literature on evidence-based practice (EBP) adaptation focuses on changes to clinical practices without explicitly addressing how organizations must adapt to accommodate a new EBP. This study explores the process of organizational-rather than EBP-adaptation during implementation and sustainment. To the authors' knowledge, there are no previous implementation studies that focus on organizational adaptation in this way.

METHODS

This analysis utilizes a case study approach to examine longitudinal qualitative data from 17 community-based organizations (CBOs) in one state and seven county-based child welfare systems. The CBOs had sustained a child-neglect intervention EBP (SafeCare®) for 2 to 10 years. The unit of analysis was the organization, and each CBO represented a case. Organizational-level profiles were created to describe the organizational adaptation process.

RESULTS

Three organizational-level adaptation profiles were identified as follows: incorporators (n = 7), early investors (n = 6), and learners (n = 4). Incorporators adapted by integrating SafeCare into existing operations to meet contractual or EBP fidelity requirements. Early Investors made substantial organizational adaptations during the early implementation period, then operated relatively consistently as the EBP became embedded in the organization and service system. Learners were characterized by steady and continuous attention to new ways that the organization could adapt to support the EBP.

CONCLUSION

The profiles demonstrated that there can be multiple effective paths to EBP sustainment. Organizational adaptation was calibrated to fit a CBO's operations (e.g., size of the program) and immediate environmental constraints (e.g., funding levels). Additionally, organizations fulfilled different functional roles in the network of entities involved in EBP implementation. Knowing organizational roles and adaptation profiles can guide implementation planning and help to structure contract designs that bridge the outer (system) and inner (organizational) contexts. Adaptation profiles can also inform the intensity of the implementation strategy tailoring process and the way that strategies are marketed to organizations.

Preliminary Cross-Sectional Validation of the CROPS 14-Item for Juvenile Offenders

The Child Report of Posttraumatic Symptoms (CROPS) is an effective 26-item trauma assessment tool. Research has indicated a 14-item version of the CROPS for juvenile offenders with improved predictive accuracy for detecting trauma exposure among male offenders and commensurate accuracy for female offenders. However, the 14-item scale has yet to be validated for juvenile offenders with an established trauma measure. Cross-sectional retrospective data of 74 adjudicated youth (59.5% male) from the original CROPS 14-item psychometric study sample were used to examine the factor structure, internal consistency, and convergent and divergent validity of the 14-item scale. Findings indicated strong internal consistency and significant correlations with all scales of an established and more extensive trauma measure for children and adolescents. Findings also revealed a CROPS 12-item model explaining 36.9% of variance. Results supported both convergent and divergent validity, suggesting both the CROPS 14-item and 12-item may be used as valid trauma symptom screeners for juvenile offenders.

Enhancing vocational training in corrections: A type 1 hybrid randomized controlled trial protocol for evaluating virtual reality job interview training among returning citizens preparing for community re-entry

The Michigan Department of Corrections operates the Vocational Villages, which are skilled trades training programs set within prisons that include an immersive educational community using virtual reality, robotics, and other technologies to develop employable trades. An enhancement to the Vocational Villages could be an evidence-based job interview training component. Recently, we conducted a series of randomized controlled trials funded by the National Institute of Mental Health to evaluate the efficacy of virtual reality job interview training (VR-JIT). The results suggested that the use of VR-JIT was associated with improved job interview skills and a greater likelihood of receiving job offers within 6 months. The primary goal of this study is to report on the protocol we developed to evaluate the effectiveness of VR-JIT at improving interview skills, increasing job offers, and reducing recidivism when delivered within two Vocational Villages via a randomized controlled trial and process evaluation. Our aims are to: (1) evaluate whether services-as-usual in combination with VR-JIT, compared to services-as-usual alone, enhances employment outcomes and reduces recidivism among returning citizens enrolled in the Vocational Villages; (2) evaluate mechanisms of employment outcomes and explore mechanisms of recidivism; and (3) conduct a multilevel, mixed-method process evaluation of VR-JIT implementation to assess the adoptability, acceptability, scalability, feasibility, and implementation costs of VR-JIT.

Building the Evidence Base to Inform Planned Intervention Adaptations by Practitioners Serving Health Disparity Populations

Many evidence-based interventions (EBIs) have been developed to prevent or treat major health conditions. However, many EBIs have exhibited limited adoption, reach, and sustainability when implemented in diverse community settings. This limitation is especially pronounced in low-resource settings that serve health disparity populations. Often, practitioners identify problems with existing EBIs originally developed and tested with populations different from their target population and introduce needed adaptations to make the intervention more suitable. Although some EBIs have been extensively adapted for diverse populations and evaluated, most local adaptations to improve fit for health disparity populations are not well documented or evaluated. As a result, empirical evidence is often lacking regarding the potential effectiveness of specific adaptations practitioners may be considering. We advocate an expansion in the emphasis of adaptation research from researcher-led interventions to research that informs practitioner-led adaptations. By presenting a research vision and strategies needed to build this area of science, we aim to inform research that facilitates successful adaptation and equitable implementation and delivery of EBIs that reduce health disparities.

Reframing implementation science to address inequities in healthcare delivery

BACKGROUND

Research has generated valuable knowledge in identifying, understanding, and intervening to address inequities in the delivery of healthcare, yet these inequities persist. The best available interventions, programs and policies designed to address inequities in healthcare are not being adopted in routine practice settings. Implementation science can help address this gap by studying the factors, processes, and strategies at multiple levels of a system of care that influence the uptake, use, and the sustainability of these programs for vulnerable populations. We propose that an equity lens can help integrate the fields of implementation science and research that focuses on inequities in healthcare delivery.

MAIN TEXT

Using Proctor et al.' (12) framework as a case study, we reframed five elements of implementation science to study inequities in healthcare. These elements include: 1) focus on reach from the very beginning; 2) design and select interventions for vulnerable populations and low-resource communities with implementation in mind; 3) implement what works and develop implementation strategies that can help reduce inequities in care; 4) develop the science of adaptations; and 5) use an equity lens for implementation outcomes.

CONCLUSIONS

The goal of this paper is to continue the dialogue on how to critically infuse an equity approach in implementation studies to proactively address healthcare inequities in historically underserved populations. Our examples provide ways to operationalize how we can blend implementation science and healthcare inequities research.

Reducing mental health disparities by increasing the personal relevance of interventions

One of the most persistent health disparities is the underutilization of mental health services by people of color. Neither evidence-based treatments (universal focus) nor culturally adapted treatments (group focus) have reduced these disparities. We propose the personal relevance of psychotherapy (PROP) model, which integrates universal, group, and individual dimensions to determine the personal relevance of interventions. A cultural example of personal relevance among people of East Asian ancestry involves "face" (i.e., one's prestige and position in society), which may moderate treatment outcomes. Pragmatic intervention approaches focused on helping individuals cope with specific external problems, compared to managing a "personal" disease, can effectively "restore" face. Thus, social problem-solving interventions may be more personally relevant to many people of East Asian ancestry than are approaches that are internally focused. In addition, we posit that social neuroscience can offer unique opportunities above and beyond self-report measures when assessing the impact of PROP and the personal relevance of interventions for diverse populations. Our preliminary evidence upon testing this hypothesis indicated that among Asian Americans, exposure to problem-solving therapy content elicited significantly greater neural activity in brain areas associated with personal relevance compared to exposure to cognitive-behavioral therapy content. Identifying personally relevant interventions has the potential to reduce mental health disparities by increasing engagement with mental health services for diverse groups. The increased client engagement produced by personally relevant interventions also has the potential to make mental health services more effective for diverse groups. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Using community-engaged methods to adapt virtual reality job-interview training for transition-age youth on the autism spectrum

BACKGROUND

Virtual Reality Job-Interview Training (VR-JIT) is an efficacious Internet-based intervention for adults with severe mental illness (SMI). Evaluations of VR-JIT have shown improved interview skill and access to employment in several cohorts of adults with SMI and with autism spectrum disorders (ASD). VR-JIT trains participants how to fill out job applications and handle job interviews through e-learning content and applied practice. Trainees receive feedback through in-the-moment nonverbal cues, critiques, and recommendations for improving performance. Our study sought to adapt VR-JIT for transition-age youth with ASD (TAY-ASD).

METHODS

We recruited TAY-ASD and adult stakeholders from public and charter schools, transition programs, and community service providers. Participants provided feedback on VR-JIT to enhance its applicability to TAY-ASD. We used community-engaged methods to process and analyze data from TAY-ASD and stakeholders, presented their quantitative and qualitative responses to community and scientific advisory boards for review and recommendations, and adapted the intervention design and content.

RESULTS

Our adaptations included adding diversity (gender; race/ethnicity) to the virtual hiring manager; shortening the interview by reducing response options; increasing social storytelling to enhance engagement with VR-JIT core components; adding employment opportunities more relevant to younger workers; reducing the reading level; and making the e-learning content more accessible by adding bullet points, voiceover, and imagery/video; and adding new learning goals.

CONCLUSIONS

This study presents a rigorous and innovative community-engaged methodology for adapting VR-JIT to meet the needs of TAY-ASD. We review our engagement with TAY-ASD and stakeholders, and discuss the standardized coding scheme we used to adapt VR-JIT and the usefulness and limitations of employing this methodology in adapting other behavioral interventions.

Multi-method process maps: An interdisciplinary approach to investigate ad hoc modifications in protocol-driven interventions

INTRODUCTION

Implementation scientists increasingly recognize that the process of implementation is dynamic, leading to ad hoc modifications that may challenge fidelity in protocol-driven interventions. However, limited attention to ad hoc modifications impairs investigators' ability to develop evidence-based hypotheses about how such modifications may impact intervention effectiveness and cost. We propose a multi-method process map methodology to facilitate the systematic data collection necessary to characterize ad hoc modifications that may impact primary intervention outcomes.

METHODS

We employ process maps (drawn from systems science), as well as focus groups and semi-structured interviews (drawn from social sciences) to investigate ad hoc modifications. Focus groups are conducted with the protocol's developers and/or planners (the implementation team) to characterize the protocol "as envisioned," while interviews conducted with frontline administrators characterize the process "as realized in practice." Process maps with both samples are used to identify when modifications occurred across a protocol-driven intervention. A case study investigating a multistage screening protocol for autism spectrum disorders (ASD) is presented to illustrate application and utility of the multi-method process maps.

RESULTS

In this case study, frontline administrators reported ad hoc modifications that potentially influenced the primary study outcome (e.g., time to ASD diagnosis). Ad hoc modifications occurred to accommodate (1) whether providers and/or parents were concerned about ASD, (2) perceptions of parental readiness to discuss ASD, and (3) perceptions of family service delivery needs and priorities.

CONCLUSION

Investigation of ad hoc modifications on primary outcomes offers new opportunities to develop empirically based adaptive interventions. Routine reporting standards are critical to provide full transparency when studying ad hoc modifications.

How can evidence-based interventions give the best value for users in social services? Balance between adherence and adaptations: a study protocol

Background

Using evidence-based interventions (EBIs) is a basic premise of contemporary social services (e.g., child and family social services). However, EBIs seldom fit seamlessly into a specific setting but often need to be adapted. Although some adaptions might be necessary, they can cause interventions to be less effective or even unsafe. The challenge of balancing adherence and adaptations when using EBIs is often referred to as the adherence and adaptation dilemma. Although the current literature identifies professionals’ management of this dilemma as problematic, it offers little practical guidance for professionals. This research aims to investigate how the adherence and adaptation dilemma is handled in social services and to explore how structured decision support can impact the management of the dilemma.

Methods

The design is a prospective, longitudinal intervention with a focus on the feasibility and usefulness of the structured decision support. The project is a collaboration between academic researchers, embedded researchers at three research and development units, and social service organizations. A multi-method data collection will be employed. Initially, a scoping review will be performed, and the results will be used in the development of a structured decision support. The decision support will be further developed and tested during a series of workshops with social service professionals. Different forms of data—focus group interviews, questionnaires, and documentation—will be used on several occasions to evaluate the impact of the structured decision support. Qualitative and quantitative analysis will be performed and usefulness for practice prioritized throughout the study.

Discussion

The study will contribute with knowledge on how the adherence and adaption dilemma is handled and experienced by social service professionals. Most importantly, the study will generate rich empirical data on how a structured decision support impacts professionals’ management of adherence and adaptions. The goal is to produce more strategic and context-sensitive implementation of EBIs in social service, which will increase value for service users.

Does increased implementation support improve community clinics' guideline-concordant care? Results of a mixed methods, pragmatic comparative effectiveness trial

Background

Disseminating care guidelines into clinical practice remains challenging, partly due to inadequate evidence on how best to help clinics incorporate new guidelines into routine care. This is particularly true in safety net community health centers (CHCs).

Methods

This pragmatic comparative effectiveness trial used a parallel mixed methods design. Twenty-nine CHC clinics were randomized to receive increasingly intensive implementation support (implementation toolkit (arm 1); toolkit + in-person training + training webinars (arm 2); toolkit + training + webinars + offered practice facilitation (arm 3)) targeting uptake of electronic health record (EHR) tools focused on guideline-concordant cardioprotective prescribing for patients with diabetes. Outcomes were compared across study arms, to test whether increased support yielded additive improvements, and with 137 non-study CHCs that share the same EHR as the study clinics. Quantitative data from the CHCs’ EHR were used to compare the magnitude of change in guideline-concordant ACE/ARB and statin prescribing, using adjusted Poisson regressions. Qualitative data collected using diverse methods (e.g., interviews, observations) identified factors influencing the quantitative outcomes.

Results

Outcomes at CHCs receiving higher-intensity support did not improve in an additive pattern. ACE/ARB prescribing did not improve in any CHC group. Statin prescribing improved overall and was significantly greater only in the arm 1 and arm 2 CHCs compared with the non-study CHCs. Factors influencing the finding of no additive impact included: aspects of the EHR tools that reduced their utility, barriers to providing the intended implementation support, and study design elements, e.g., inability to adapt the provided support. Factors influencing overall improvements in statin outcomes likely included a secular trend in awareness of statin prescribing guidelines, selection bias where motivated clinics volunteered for the study, and study participation focusing clinic staff on the targeted outcomes.

Conclusions

Efforts to implement care guidelines should: ensure adaptability when providing implementation support and conduct formative evaluations to determine the optimal form of such support for a given clinic; consider how study data collection influences adoption; and consider barriers to clinics’ ability to use/accept implementation support as planned. More research is needed on supporting change implementation in under-resourced settings like CHCs.

Trial registration

ClinicalTrials.gov, NCT02325531. Registered 15 December 2014.

If we build it, will they come? And will they stay? Commentary on Holmes et al. "Examining patterns of dose response for clients who do and do not complete Cognitive Processing Therapy"

Although evidence-based practice is becoming more widely accepted, the issue of patient preference has been relatively ignored. As noted by Holmes and colleagues (2019), when delivered in a community setting, Cognitive Processing Therapy (CPT) can have a relatively high dropout rate (42 % in the Holmes et al., 2019). In this commentary, issues about the conceptualization of treatment dropout as one index of patient preference are discussed. Dropout can be conceptualized as a potential reflection of poor fit between the patient and a specific empirically-supported treatment. Consideration of ways in which an empirically-supported treatment can be personalized, while remaining true to its underlying principles, are discussed using CPT as an example.

The Value Equation: Three complementary propositions for reconciling fidelity and adaptation in evidence-based practice implementation

BACKGROUND

There has long been debate about the balance between fidelity to evidence-based interventions (EBIs) and the need for adaptation for specific contexts or particular patients. The debate is relevant to virtually all clinical areas. This paper synthesises arguments from both fidelity and adaptation perspectives to provide a comprehensive understanding of the challenges involved, and proposes a theoretical and practical approach for how fidelity and adaptation can optimally be managed.

DISCUSSION

There are convincing arguments in support of both fidelity and adaptations, representing the perspectives of intervention developers and internal validity on the one hand and users and external validity on the other. Instead of characterizing fidelity and adaptation as mutually exclusive, we propose that they may better be conceptualized as complimentary, representing two synergistic perspectives that can increase the relevance of research, and provide a practical way to approach the goal of optimizing patient outcomes. The theoretical approach proposed, the "Value Equation," provides a method for reconciling the fidelity and adaptation debate by putting it in relation to the value (V) that is produced. The equation involves three terms: intervention (IN), context (C), and implementation strategies (IS). Fidelity and adaptation determine how these terms are balanced and, in turn, the end product - the value it produces for patients, providers, organizations, and systems. The Value Equation summarizes three central propositions: 1) The end product of implementation efforts should emphasize overall value rather than only the intervention effects, 2) implementation strategies can be construed as a method to create fit between EBIs and context, and 3) transparency is vital; not only for the intervention but for all of the four terms of the equation. There are merits to arguments for both fidelity and adaptation. We propose a theoretical approach, a Value Equation, to reconciling the fidelity and adaptation debate. Although there are complexities in the equation and the propositions, we suggest that the Value Equation be used in developing and testing hypotheses that can help implementation science move toward a more granular understanding of the roles of fidelity and adaptation in the implementation process, and ultimately sustainability of practices that provide value to stakeholders.

Depression self-management in people with epilepsy: Adapting project UPLIFT for underserved populations

Data from the 2015 National Health Interview Survey found that the prevalence of active epilepsy has increased to three million adults. Although findings have been mixed, some research indicates that Blacks and Hispanics share a higher burden of epilepsy prevalence compared with non-Hispanic whites. Moreover, depression is a common comorbid condition among people with epilepsy (PWE), affecting up to 55% of the epilepsy population. Widespread use and increased public health impact of evidence-based self-management interventions is critical to reducing disease burden and may require adapting original interventions into more culturally relevant versions for racial and ethnic minority groups. Project UPLIFT provides access to mental health self-management skills training that is distance-delivered, does not interfere with medication management, and has been shown to be effective in reducing depressive symptoms. This paper presents the process of exploring the adaptation of Project UPLIFT for Black and Hispanic PWE and herein suggests that evidence-based interventions can be successfully adapted for new populations or cultural settings through a careful and systematic process. Additional key lessons learned include the importance of community engagement and that language matters. Ultimately, if the adapted Project UPLIFT intervention produces positive outcomes for diverse populations of PWE, it will extend the strategies available to reduce the burden of depression. Implementing evidence-based interventions such as Project UPLIFT is critical to reducing disease burden; however, their delivery may need to be tailored to the needs and culture of the populations of interest.

Adapting implementation strategies: a case study of how to support implementation of healthy canteen policies

OBJECTIVES

Although evidence-based interventions (EBIs) and effective strategies to implement them exist, they cannot be used by policy makers and practitioners if they do not align with end users' needs. As such, adaptations to EBIs and implementation approaches are likely to occur to increase 'fit' with end users' capacity. This article describes an approach undertaken by a population health service delivery unit in one Australian state to develop an adapted implementation strategy to support the implementation of the mandatory healthy canteen policy (EBI) to all schools located in the service delivery region.

STUDY DESIGN

This is a case study of adapting an intervention to improve implementation of the healthy canteen policy.

METHODS AND RESULTS

This is a six-step pragmatic, empirically driven approach. The steps include (i) adapt, where appropriate, the EBI to facilitate implementation; (ii) identify end users' capacity for implementation; (iii) identify opportunities to adapt the implementation interventions while preserving meaningful intervention impact; (iv) undertake program adaptation; (v) develop training and resources to support delivery of implementation strategies and; (vi) evaluate the adapted intervention. This article describes the application of these steps by the authors to develop an adapted support strategy consistent with end users' needs.

CONCLUSIONS

This study provides some guidance on how to adapt implementation support approaches particularly when EBIs cannot be adapted. Future empirical research providing guidance on making practical adaptation decisions are needed.

The FRAME: an expanded framework for reporting adaptations and modifications to evidence-based interventions

BACKGROUND

This paper describes the process and results of a refinement of a framework to characterize modifications to interventions. The original version did not fully capture several aspects of modification and adaptation that may be important to document and report. Additionally, the earlier framework did not include a way to differentiate cultural adaptation from adaptations made for other reasons. Reporting additional elements will allow for a more precise understanding of modifications, the process of modifying or adapting, and the relationship between different forms of modification and subsequent health and implementation outcomes.

DISCUSSION

We employed a multifaceted approach to develop the updated FRAME involving coding documents identified through a literature review, rapid coding of qualitative interviews, and a refinement process informed by multiple stakeholders. The updated FRAME expands upon Stirman et al.'s original framework by adding components of modification to report: (1) when and how in the implementation process the modification was made, (2) whether the modification was planned/proactive (i.e., an adaptation) or unplanned/reactive, (3) who determined that the modification should be made, (4) what is modified, (5) at what level of delivery the modification is made, (6) type or nature of context or content-level modifications, (7) the extent to which the modification is fidelity-consistent, and (8) the reasons for the modification, including (a) the intent or goal of the modification (e.g., to reduce costs) and (b) contextual factors that influenced the decision. Methods of using the framework to assess modifications are outlined, along with their strengths and weaknesses, and considerations for research to validate these measurement strategies.

CONCLUSION

The updated FRAME includes consideration of when and how modifications occurred, whether it was planned or unplanned, relationship to fidelity, and reasons and goals for modification. This tool that can be used to support research on the timing, nature, goals and reasons for, and impact of modifications to evidence-based interventions.

Provider fidelity and modifications to cognitive processing therapy in a diverse community health clinic: Associations with clinical change

OBJECTIVE

The purpose of this study is to examine associations between therapist adherence, competence, and modifications of an evidence-based protocol (EBP) delivered in routine clinical care and client outcomes.

METHOD

Data were derived from a NIMH-funded implementation-effectiveness hybrid study of Cognitive Processing Therapy (CPT) for PTSD in a diverse community health center. Providers (n = 19) treated clients (n = 58) as part of their routine clinical care. Clients completed the PCL-S and PHQ-9 at baseline, after each CPT session, and posttreatment. CPT sessions were rated for treatment fidelity and therapist modifications.

RESULTS

Overall, therapist adherence was high, although it decreased across sessions suggesting potential drift. Therapist competence ratings varied widely. Therapists made on average 1.6 fidelity-consistent and 0.4 fidelity-inconsistent modifications per session. Results show that higher numbers of fidelity-consistent modifications were associated with larger reductions in posttraumatic stress and depressive symptoms. High adherence ratings were associated with greater reductions in depressive symptoms, whereas higher competence ratings were associated with greater reduction in posttraumatic stress symptoms.

CONCLUSIONS

The results highlight the importance of differentially assessing therapist adherence, competence, and modifications to EBP in usual care settings. The findings also suggest that effective EBP delivery in routine care may require minor adaptations to meet client needs, consistent with previous studies. Greater attention to fidelity and adaptation can enhance training so providers can tailor while retaining core components of the intervention. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Evaluating the effectiveness and implementation of evidence-based treatment: A multisite hybrid design

The gap between treatment development and efficacy testing to scaled up implementations of evidence-based treatment (EBT) is an estimated 20 years, and hybrid research designs aim to reduce the gap. One was used for a multisite study in cancer control, testing coprimary aims: (a) determine the feasibility and utility of a flexible EBT implementation strategy and (b) determine the clinical effectiveness of an EBT as implemented by newly trained providers. Therapists from 15 diverse sites implemented the biobehavioral intervention (BBI) for cancer patients (N = 158) as part of standard care. For implementation, therapists determined treatment format, number of sessions, and so forth and reported session-by-session fidelity. Patients completed fidelity and outcome assessments. Results showed therapists BBI implementation was done with fidelity, for example, session "dose" (59%), core content coverage (60-70%), and others. Patient reported fidelity was favorable and comparable to the BBI efficacy trial. Effectiveness data show the primary outcome, patients' scores on the Profile of Mood States total mood disturbance, significantly improved (R² = 0.06, β = -0.24, p < .01) as did a secondary outcome, physical activity (R² = 0.02, β = 0.13, p < .05). This first use of a hybrid design in health psychology provided support for a novel strategy that allowed providers implementation flexibility. Still, the EBT was delivered with fidelity and in addition, therapists generated novel procedures to enhance setting-specific usage of BBI and its ultimate effectiveness with patients. This research is an example of translational research spanning theory and efficacy tests to dissemination and implementation. (PsycINFO Database Record (c) 2019 APA, all rights reserved).