Digital Engagement Strategy and Health Care Worker Mental Health: A Randomized Clinical Trial

Importance

The health care workforce continues to experience high rates of depression and anxiety. Finding ways to effectively support the mental health and well-being of health care workers is challenging.

Objective

To test the effectiveness of remote, pushed digital assessments and engagement to improve depression and anxiety among health care workers compared with usual care.

Design, Setting, and Participants

This was a 9-month randomized clinical trial with a 6-month intervention period. Participants were health care workers with self-reported daily access to a smartphone and at least 4 clinical hours per week. Participants were randomized to usual care or the intervention between January 2022 and March 2023. Data analyses were conducted between May and July 2023.

Interventions

All participants completed baseline, 6-month, and 9-month mental health, well-being, and burnout assessments. The control group had open access to a web-based mental health platform. Participants in the intervention group received monthly text messaging about mental health, mental health assessments, and linkages to care.

Main Outcomes and Measures

The primary outcomes were mean change in depression and anxiety scores at 6 months from baseline. Secondary outcomes include mean change in well-being, burnout, and self-reported workplace productivity.

Results

In this study, 1275 participants were randomized (642 [50.4%] to the intervention group and 633 [49.6%] to control group). Participants had a mean (SD) age of 38.6 (10.9) years, 1063 participants (83.4%) were female, 320 (25.1%) self-identified as Black, and 793 (62.2%) self-identified as White. Across the groups, the mean difference in depression score was significantly different at 6 months (-0.96 [95% CI, -1.52 to -0.40]) and at 9 months (-1.14 [95% CI, -1.69 to -0.58]). The mean difference in anxiety score from baseline to 6 months was statistically significantly larger for those in the intervention group vs usual care (-0.71 [95% CI, -1.25 to -0.17]) and held true at 9 months (-1.06 [95% CI, -1.59 to -0.52]).

Conclusions and Relevance

In a trial of health care workers, a proactive digital engagement strategy, including pushed text messaging, mobile mental health assessments, and connection to care, improved depression and anxiety over a 6-month period compared with simply making the same resources available for individuals to find and use.

Trial Registration

ClinicalTrials.gov Identifier: NCT05028075.

Mobile Crisis Services: A Clinician Survey of Current Suicide Prevention Practices and Barriers to Care Delivery

Mobile crisis teams (MCTs) deploy clinicians to assist individuals in acute crisis in the community. Little is known about the extent to which these teams provide evidence-based practices (EBPs) for suicide prevention nor the barriers they face. We surveyed 120 MCT clinicians across the United States about their: (1) use of suicide risk screening and assessment tools; (2) strategies used to address suicide risk (both EBPs and non-EBPs); and (3) perceived barriers to high-quality MCT services. Nearly all clinicians reported use of validated suicide screening tools and generic "safety planning." However, a sizeable minority also reported use of non-EBPs. Open-ended responses suggested many client/family-, clinician-, and systems-level barriers to MCT use of EBPs for suicide prevention. We identified several targets for future implementation efforts, including the need for de-implementation strategies to reduce use of ineffective and potentially harmful practices, and unique aspects of MCTs that require tailored implementation supports.

Implementation of an Adapted Fidelity Scale for Pennsylvania Coordinated Specialty Care Programs for First Episode Psychosis

Coordinated Specialty Care (CSC) is an evidence-based model for early intervention of first episode psychosis (FEP). Monitoring fidelity to CSC models is essential for proper evaluation of program outcomes and quality improvement. To address variability across CSC programs and fidelity assessment-associated burden, an adapted fidelity scale was developed and implemented statewide in Pennsylvania. This report describes the design and preliminary performance of the adapted scale. The Pennsylvania FEP Fidelity Scale (PA-FEP-FS) assesses adherence to the CSC model by focusing on essential model components of other established fidelity scales, in tandem with program evaluation data. Initial data from fourteen PA-FEP sites from 2018 to 2021 were examined as preliminary validation. Assessment-associated burdens and costs were also estimated. PA-FEP-FS captured essential components of CSC models and proved feasible for implementation across varying program structures, while minimizing burden and cost. Programs utilized annual feedback as CSC benchmarks, evidenced by increased scores over time. PA-FEP-FS provides a model for adapting CSC fidelity scales to meet state- or local-level requirements while reducing assessment burdens and costs that may be associated with existing scales.

The Relationship Between Trauma Exposure and Obsessive-Compulsive Disorder in Youth: A Systematic Review

Extant literature suggests a possible relationship between childhood trauma exposure and the development and trajectory of obsessive-compulsive symptoms (OCS) and obsessive-compulsive disorder (OCD); however, this relationship is poorly understood. We conducted a systematic review that examined trauma exposure and OCS/OCD in youth. Primary inclusion criteria were English-language articles that addressed a sample with participants under 18 years of age. 46 articles met criteria for review, and were categorized into three groups of overlap: Post-traumatic Stress Disorder (PTSD) and OCS or OCD (n = 3), clearly defined Criterion A event trauma exposure and OCS or OCD (n = 30), and less well-defined potential trauma exposure and OCS or OCD (n = 13). There was mixed evidence linking trauma exposure and the development of OCS or OCD in youth. Major methodological limitations preclude formal conclusions. More research on co-occurring trauma exposure and OCD/OCS in youth is needed to advance research and improve treatment.

An exposure-based implementation strategy to decrease clinician anxiety about implementing suicide prevention evidence-based practices: protocol for development and pilot testing (Project CALMER)

BACKGROUND

Clinicians often report that their own anxiety and low self-efficacy inhibit their use of evidence-based suicide prevention practices, including gold-standard screening and brief interventions. Exposure therapy to reduce clinician maladaptive anxiety and bolster self-efficacy use is a compelling but untested approach to improving the implementation of suicide prevention evidence-based practices (EBPs). This project brings together an interdisciplinary team to leverage decades of research on behavior change from exposure theory to design and pilot test an exposure-based implementation strategy (EBIS) to target clinician anxiety to improve suicide prevention EBP implementation.

METHODS

We will develop, iteratively refine, and pilot test an EBIS paired with implementation as usual (IAU; didactic training and consultation) in preparation for a larger study of the effect of this strategy on reducing clinician anxiety, improving self-efficacy, and increasing use of the Columbia Suicide Severity Rating Scale and the Safety Planning Intervention in outpatient mental health settings. Aim 1 of this study is to use participatory design methods to develop and refine the EBIS in collaboration with a stakeholder advisory board. Aim 2 is to iteratively refine the EBIS with up to 15 clinicians in a pilot field test using rapid cycle prototyping. Aim 3 is to test the refined EBIS in a pilot implementation trial. Forty community mental health clinicians will be randomized 1:1 to receive either IAU or IAU + EBIS for 12 weeks. Our primary outcomes are EBIS acceptability and feasibility, measured through questionnaires, interviews, and recruitment and retention statistics. Secondary outcomes are the engagement of target implementation mechanisms (clinician anxiety and self-efficacy related to implementation) and preliminary effectiveness of EBIS on implementation outcomes (adoption and fidelity) assessed via mixed methods (questionnaires, chart-stimulated recall, observer-coded role plays, and interviews).

DISCUSSION

Outcomes from this study will yield insight into the feasibility and utility of directly targeting clinician anxiety and self-efficacy as mechanistic processes informing the implementation of suicide prevention EBPs. Results will inform a fully powered hybrid effectiveness-implementation trial to test EBIS' effect on implementation and patient outcomes.

TRIAL REGISTRATION

Clinical Trials Registration Number: NCT05172609 . Registered on 12/29/2021.

COVID-19 Impacts on Pennsylvania Coordinated Specialty Care for Early Psychosis Participants

OBJECTIVES

The coronavirus disease (COVID-19) pandemic produced swift, extensive changes in daily life, including for first-episode psychosis (FEP) clients. This study examined pandemic-related psychosocial impacts to clients while engaged in Coordinated Specialty Care (CSC). We also examined FEP client vaccination rates, as vaccinations can reduce hospitalizations/deaths, and related worries.

METHODS

Thirty-one clients (45% female; ages 13-39; 26% black, 61% white) from Pennsylvania (PA) CSC outpatient programs completed an online survey evaluating exposure to COVID-19, associated worries, coping, and safety strategies. Descriptive statistics characterized responses and demographic group differences. Additional program evaluation data informed vaccination rates for PA FEP clients.

RESULTS

Participants reported substantial pandemic-related impacts to daily life. Many clients reported improved safety measures to protect themselves/others from COVID-19. Clients largely denied substantial worries about infection for themselves, reporting greater concern for loved ones. Multiple coping strategies were endorsed, which, with few exceptions, did not differ among demographic groups. FEP clients had a low reported rate of vaccination (28.6%) as of September 2021.

CONCLUSIONS

Observed prolonged pandemic effects may alter FEP client progress in CSC. Stakeholders should be prepared to adjust FEP treatment accordingly in the event of a similar disaster. Concentrated vaccination efforts may be necessary for this population.

Implementing a resilience coach program to support first year housestaff during the COVID-19 pandemic: early pilot results and comparison with non-housestaff sessions

BACKGROUND

In response to the COVID-19 pandemic, we launched the Penn Medicine Coping First Aid program to provide psychosocial supports to our health system community. Our approach leveraged lay health worker volunteers trained in principles of Psychological First Aid to deliver coaching services through a centralized virtual platform.

METHODS

We emailed all (n = 408) first year housestaff (i.e., residents and fellows) with an invitation to schedule a session with a resilience coach. We compared the mental health concerns, symptoms, and Psychological First Aid techniques recorded in (n = 67) first year housestaff sessions with (n = 91) sessions of other employees in the health system.

RESULTS

Between June and November 2020, forty-six first year housestaff attended at least one resilience coaching session. First year housestaff most commonly presented with feelings of anxiety and sadness and shared concerns related to the availability of social support. Resilience coaches most frequently provided practical assistance and ensured safety and comfort to first year housestaff. First year housestaff reported fewer physical or mental health symptoms and held shorter sessions with resilience coaches than non-housestaff.

CONCLUSIONS

This work offers insights on how to address psychosocial functioning through low-intensity interventions delivered by lay personnel. More research is needed to understand the efficacy of this program and how best to engage housestaff in wellness and resilience programs throughout training, both during and beyond COVID-19.

An Assessment of Faculty Perceptions of Wellness Needs and Strategies to Optimize Engagement With Wellness Resources in Academic Medicine

OBJECTIVE

Faculty at academic medical institutions are at increased risk for burnout. This study aimed to assess faculty perceptions of wellness needs and identify strategies to optimize engagement with individualized wellness resources.

METHODS

Semistructured interviews were conducted with 37 faculty members in one US academic medical center.

RESULTS

Participants identified significant barriers to achieving emotional health and wellness goals. Areas where participants identified needing the most support included interpersonal relationships, accountability for wellness goals, career support, financial resources, and mentorship. Most participants were unaware of all wellness resources available at their institution. Participants recommended regular marketing and emphasizing confidentiality of employer-sponsored programs. They also provided feedback on specific dissemination and marketing methods.

CONCLUSIONS

This research underscores the need for wellness resources for faculty and the importance of intentional dissemination of these resources to optimize uptake.

Identifying Common and Unique Barriers and Facilitators to Implementing Evidence-Based Practices for Suicide Prevention across Primary Care and Specialty Mental Health Settings

OBJECTIVE

We identified common and unique barriers and facilitators of evidence-based suicide prevention practices across primary care practices with integrated behavioral health services and specialty mental health settings to identify generalizable strategies for enhancing future implementation efforts.

METHOD

Twenty-six clinicians and practice leaders from behavioral health (n = 2 programs) and primary care (n = 4 clinics) settings participated. Participation included a semi-structured qualitative interview on barriers and facilitators to implementing evidence-based suicide prevention practices. Within that interview, clinicians participated in a chart-stimulated recall exercise to gather additional information about decision making regarding suicide screening. Interview guides and qualitative coding were informed by leading frameworks in implementation science and behavioral science, and an integrated approach to interpreting qualitative results was used.

RESULTS

There were a number of similar themes associated with implementation of suicide prevention practices across settings and clinician types, such as the benefits of inter-professional collaboration and uncertainties about managing suicidality once risk was disclosed. Clinicians also highlighted barriers unique to their settings. For primary care settings, time constraints and competing demands were consistently described as barriers. For specialty mental health settings, difficulties coordinating care with schools and other providers in the community made implementation of suicide prevention practices challenging.

CONCLUSION

Findings can inform the development and testing of implementation strategies that are generalizable across primary care and specialty mental health settings, as well as those tailored for unique site needs, to enhance use of evidence-based suicide prevention practices in settings where individuals at risk for suicide are especially likely to present.HIGHLIGHTSWe examined barriers and facilitators to suicide prevention across health settings.Common and unique barriers and facilitators across health-care settings emerged.Findings can enhance suicide prevention implementation across health-care settings.

To Proceed Via Telehealth or Not? Considerations for Pediatric Anxiety and Related Disorders Beyond COVID-19

The COVID-19 pandemic accelerated a widespread shift to telehealth among mental health professionals to prioritize both providers' and clients' safety. Telehealth is likely here to stay; however, there is limited practical guidance for clinicians about how to make decisions regarding who should proceed with care via telehealth versus in-person. There also is virtually no data on the effectiveness of hybrid approaches to care; yet this can be an attractive option with potential clinical benefit. This paper provides practice-informed guidance to support shared clinical decision-making between clinicians and families to decide whether to engage in therapy services in-person or via telehealth. We specifically focus on decision-making guidance relevant for youth with anxiety or related disorders, given the unique implications of telehealth for these youth. Guided by the three-legged stool of evidence-based practice, we discuss how clinicians can use principles of shared decision-making to inform clinical recommendations about treatment modality.

Youth Trauma Histories are Associated with Under-diagnosis and Under-treatment of Co-occurring Youth Psychiatric Symptoms

OBJECTIVE

We examined whether in the presence of trauma exposure, non-traumatic stress-related symptoms are interpreted by mental health clinicians as less salient than the trauma exposure and are de-emphasized as a treatment target, consistent with a diagnostic overshadowing bias.

METHODS

Using an adapted version of a diagnostic overshadowing bias experimental paradigm, mental health clinicians (N = 266, M age = 34.4 years, 82% female) were randomly assigned to receive two of six clinical vignette variations. Vignette 1 described an adolescent with obsessive-compulsive disorder (OCD). Vignette 2 described a pre-adolescent with oppositional defiant disorder (ODD). Vignettes were identical except for whether the youth reported exposure to a potentially traumatic event (PTE; no PTE, sexual PTE, or physical PTE). Clinicians received one vignette with a PTE and one without, counterbalancing order. Clinicians rated the likelihood the youth met criteria for various diagnoses and the appropriateness of various treatments on 7-point scales.

RESULTS

Across both vignettes, clinicians rated the target diagnosis (OCD in Vignette 1, ODD in Vignette 2) as less likely for vignettes with a PTE than for the same vignettes without a PTE. Clinicians also rated evidence-based treatment modalities for target diagnoses as less appropriate in the presence of a PTE than when a PTE was present.

CONCLUSIONS

Consistent with possible bias, clinicians may under-recognize and under-treat non-traumatic stress-related mental health symptoms in youth with a co-occurring trauma history. Future work to validate this bias in real-world practice is indicated.

Tele-mental Health Transitions for Pennsylvania Coordinated Specialty Care Programs for Early Psychosis During the COVID-19 Pandemic

This study examined provider and client perspectives of tele-mental health (TMH) in early psychosis care during the COVID-19 pandemic. To achieve this goal, thirty-three mental health providers and 31 clients from Pennsylvania Coordinated Specialty Care (CSC) programs completed web-based surveys assessing TMH usage, experiences, and perceptions between May and September 2020. Three additional TMH-related questions were asked two years later of PA CSC Program Directors between Feb and March 2022. Descriptive statistics characterized responses. Open-ended items were coded and grouped into themes for qualitative synthesis. As early as mid-2020, participants reported extensive use of TMH technologies, including telephone and video visits. Although most providers and clients preferred in-person care to TMH, most clients still found TMH to be comparable to or better than in-person care; 94% of clients indicated interest in future TMH services. Providers also noted more successes than challenges with TMH. Nine themes emerged regarding provider-perceived client characteristics that could benefit from TMH and were grouped into two categories: client-level (access to technology, comfort with technology, transportation, young age, symptom severity, functioning level, motivation for treatment adherence) and interpersonal-level (external support systems and engagement with program prior to the pandemic) characteristics. Two years later, program directors reported continued perceived advantages of TMH in CSCs, although some barriers persisted. Despite the unexpected shift to TMH in early psychosis programs during the COVID-19 pandemic, findings indicated a relatively positive transition to TMH and perceived promise of TMH as a sustained part of routine care.

A systematic review of brief, freely accessible, and valid self-report measures for substance use disorders and treatment

BACKGROUND

Self-report measures can improve evidence-based assessment practices in substance use disorder treatment, but many measures are burdensome and costly, limiting their utility in community practice and non-specialty healthcare settings. This systematic review identified and evaluated the psychometric properties of brief, free, and readily accessible self-report measures of substance use and related factors.

METHODS

We searched two electronic databases (PsycINFO and PubMed) in May 2021 for published literature on scales, measures, or instruments related to substance use, substance use treatment, and recovery, and extracted the names of all measures. Measures were included if they were: (1) brief (25 items or fewer), (2) freely accessible in a ready-to-use format, and (3) had published psychometric data.

RESULTS

An initial search returned 411 measures, of which 73 (18%) met criteria for inclusion. Included measures assessed a variety of substances (e.g., alcohol, nicotine, opioids, cannabinoids, cocaine) and measurement domains (e.g., use, severity, expectancies, withdrawal). Among these measures, 14 (19%) were classified as psychometrically "excellent," 27 (37%) were rated as "good," 32 (44%) were "adequate."

CONCLUSIONS

Despite the shift toward evidence-based assessment in substance use disorder treatment in the last twenty years, key areas of public health concern are lacking pragmatic, psychometrically valid measures. Among the brief measures we reviewed, less than a fifth met criteria for psychometric "excellence" and most of these instruments fell into one measurement domain: screening for problematic substance use. Future research should focus both on improving the evidence base for existing brief self-report measures and creating new low-burden measures for specific substances and treatment constructs.

Preliminary Implementation Outcomes of a Free Online Toolkit to Support Exposure Therapy Implementation for Youth

Background

Exposure therapy ("exposure") for youth anxiety is highly underutilized in clinical practice. Asynchronous, online implementation strategies such as online toolkits hold promise as pragmatic approaches for extending the sustainability of evidence-based interventions, but their long-term usage, perceived utility, and impact are rarely studied.

Objective

This study presents three-year preliminary implementation outcomes for a free, online toolkit to support exposure therapy use with youth: the Resource for Exposure for Anxiety Disordered Youth (READY; www.bravepracticeforkids.com). Implementation outcomes of interest included READY usage statistics, adoption, perceived utility, and clinician exposure use.

Methods

Web analytics characterized usage patterns. A survey of READY users (N = 49; M age = 34.2, 82.9% female, 71% White) assessed adoption, perceived utility, clinician exposure use, and persistent barriers to exposure use.

Results

In its first three years, READY had 13,543 page views across 1,731 unique users; 442 (25.6%) registered as a site user to access specialized content. Survey data suggested variability in usage and perceived utility across toolkit components. Qualitative analyses highlighted persistent exposure barriers that pointed to potential READY refinements.

Conculsions

Overall, READY has been accessed by hundreds of clinicians, but its impact was limited by low return to the site. This study highlights strengths and limitations of standalone online implementation supports and identifies additional steps needed to optimally support clinicians to deliver exposure to youth in need.

What is the Status of Multi-Informant Treatment Fidelity Research?

OBJECTIVE

The precise measurement of treatment fidelity (quantity and quality in the delivery of treatment strategies in an intervention) is essential for intervention development, evaluation, and implementation. Various informants are used in fidelity assessment (e.g., observers, practitioners [clinicians, teachers], clients), but these informants often do not agree on ratings. This scoping review aims to ascertain the state of science around multi-informant assessment of treatment fidelity.

METHOD

A literature search of articles published through December 2021 identified 673 articles. Screening reduced the number of articles to 44, and the final study set included 35 articles.

RESULTS

There was substantial variability across studies regarding study design, how fidelity was operationalized, and how reliability was defined and assessed. Most studies evaluated the agreement between independent observers and practitioner-report, though several other informant pairs were assessed. Overall, findings suggest that concordance across fidelity informants was low to moderate, with a few key exceptions.

CONCLUSIONS

It is difficult to draw clear conclusions about the degree to which single versus multiple informant assessment is needed to produce an accurate and complete picture of treatment fidelity. The field needs to take steps to determine how to leverage multi-informant assessment to accurately assess treatment fidelity.

Metrics to evaluate implementation scientists in the USA: what matters most?

Background

Implementation science has grown rapidly as a discipline over the past two decades. An examination of how publication patterns and other scholarly activities of implementation scientists are weighted in the tenure and promotion process is needed given the unique and applied focus of the field.

Methods

We surveyed implementation scientists (mostly from the USA) to understand their perspectives on the following matters: (1) factors weighted in tenure and promotion for implementation scientists, (2) how important these factors are for success as an implementation scientist, (3) how impact is defined for implementation scientists, (4) top journals in implementation science, and (5) how these journals are perceived with regard to their prestige. We calculated univariate descriptive statistics for all quantitative data, and we used Wilcoxon signed-rank tests to compare the participants’ ratings of various factors. We analyzed open-ended qualitative responses using content analysis.

Results

One hundred thirty-two implementation scientists completed the survey (response rate = 28.9%). Four factors were rated as more important for tenure and promotion decisions: number of publications, quality of publication outlets, success in obtaining external funding, and record of excellence in teaching. Six factors were rated as more important for overall success as an implementation scientist: presentations at professional meetings, involvement in professional service, impact of the implementation scientist’s scholarship on the local community and/or state, impact of the implementation scientist’s scholarship on the research community, the number and quality of the implementation scientist’s community partnerships, and the implementation scientist’s ability to disseminate their work to non-research audiences. Participants most frequently defined and described impact as changing practice and/or policy. This expert cohort identified Implementation Science as the top journal in the field.

Conclusions

Overall, there was a significant mismatch between the factors experts identified as being important to academic success (e.g., tenure and promotion) and the factors needed to be a successful implementation scientist. Findings have important implications for capacity building, although they are largely reflective of the promotion and tenure process in the USA.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00323-0.

A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy

Clinician fidelity to cognitive behavioral therapy (CBT) is an important mechanism by which desired clinical outcomes are achieved and is an indicator of care quality. Despite its importance, there are few fidelity measurement methods that are efficient and have demonstrated reliability and validity. Using a randomized trial design, we compared three methods of assessing CBT adherence-a core component of fidelity-to direct observation, the gold standard. Clinicians recruited from 27 community mental health agencies (n = 126; M age = 37.69 years, SD = 12.84; 75.7% female) were randomized 1:1:1 to one of three fidelity conditions: self-report (n = 41), chart-stimulated recall (semistructured interviews with the chart available; n = 42), or behavioral rehearsal (simulated role-plays; n = 43). All participating clinicians completed fidelity assessments for up to three sessions with three different clients that were recruited from clinicians' caseloads (n = 288; M age = 13.39 years SD = 3.89; 41.7% female); sessions were also audio-recorded and coded for comparison to determine the most accurate method. All fidelity measures had parallel scales that yielded an adherence maximum score (i.e., the highest-rated intervention in a session), a mean of techniques observed, and a count total of observed techniques. Results of three-level mixed effects regression models indicated that behavioral rehearsal produced comparable scores to observation for all adherence scores (all ps > .01), indicating no difference between behavioral rehearsal and observation. Self-report and chart-stimulated recall overestimated adherence compared to observation (ps < .01). Overall, findings suggested that behavioral rehearsal indexed CBT adherence comparably to direct observation, the gold-standard, in pediatric populations. Behavioral rehearsal may at times be able to replace the need for resource-intensive direct observation in implementation research and practice.

Comparing output from two methods of participatory design for developing implementation strategies: traditional contextual inquiry vs. rapid crowd sourcing

Background

Participatory design methods are a key component of designing tailored implementation strategies. These methods vary in the resources required to execute and analyze their outputs. No work to date has examined the extent to which the output obtained from different approaches to participatory design varies.

Methods

We concurrently used two separate participatory design methods: (1) field observations and qualitative interviews (i.e., traditional contextual inquiry) and (2) rapid crowd sourcing (an innovation tournament). Our goal was to generate and compare information to tailor implementation strategies to increase the use of evidence-based data collection practices among one-to-one aides working with children with autism. Each method was executed and analyzed by study team members blinded to the output of the other method. We estimated the personnel time and monetary costs associated with each method to further facilitate comparison.

Results

Observations and interviews generated nearly double the number of implementation strategies (n = 26) than did the innovation tournament (n = 14). When strategies were classified into implementation strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy, there was considerable overlap in the content of identified strategies. However, strategies derived from observations and interviews were more specific than those from the innovation tournament. Five strategies (13%) reflected content unique to observations and interviews and 3 (8%) strategies were unique to the innovation tournament. Only observations and interviews identified implementation strategies related to adapting and tailoring to context; only the innovation tournament identified implementation strategies that used incentives. Observations and interviews required more than three times the personnel hours than the innovation tournament, but the innovation tournament was more costly overall due to the technological platform used.

Conclusions

There was substantial overlap in content derived from observations and interviews and the innovation tournament, although there was greater specificity in the findings from observations and interviews. However, the innovation tournament yielded unique information. To select the best participatory design approach to inform implementation strategy design for a particular context, researchers should carefully consider unique advantages of each method and weigh the resources available to invest in the process.

Economic Precarity, Financial Strain, and Job-Related Stress Among Philadelphia's Public Mental Health Clinicians

OBJECTIVE

Efforts to improve mental health treatment delivery come at a time of rising inequality and cuts or insufficient increases to mental health funding. Public mental health clinicians face increased demands, experience economic stress, and treat underresourced patients disproportionately burdened by trauma. The authors sought to understand clinicians' current economic and psychological conditions and the relationship of these conditions to the delivery of an evidence-based intervention (EBI) designed to treat posttraumatic stress disorder among youths.

METHODS

In July 2020, 49 public mental health clinicians from 16 Philadelphia clinics who were trained in an EBI, trauma-focused cognitive-behavioral therapy (TF-CBT), were surveyed by e-mail. Respondents reported on their economic precarity, financial strain, burnout, secondary traumatic stress (i.e., the stress response associated with caring for people exposed to trauma), and TF-CBT use. Associations between clinicians' job-related stressors and their use of TF-CBT were examined with mixed models. Content coding was used to organize clinicians' open-ended responses to questions regarding financial strain related to the COVID-19 pandemic.

RESULTS

Clinicians' economic precarity, financial strain, and job-related stress were high; 37% of clinicians were independent contractors, 44% of whom wanted a salaried position. Of 37 clinicians with education debt, 38% reported owing ≥$100,000. In the past year, 29% of clinicians reported lack of personal mental health care because of cost, and 22% met the cutoff for experiencing secondary traumatic stress symptoms. Education debt was negatively associated with use of TF-CBT (p<0.001).

CONCLUSIONS

The stress of providing care in underresourced clinical settings may interfere with efforts to integrate scientific evidence into mental health care.

Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: a lagged analysis

Background

Theory and empirical research suggest organizational climate for evidence-based practice (EBP) implementation may be an important and malleable target to improve clinician use of EBPs in healthcare; however, this work has been criticized for overreliance on self-report measures of implementation outcomes and cross-sectional designs. This study combines data from two studies spanning 7 years to test the hypothesis that higher levels of organizational EBP implementation climate prospectively predicts improved clinician adherence to an EBP, cognitive behavioral therapy (CBT), as rated by expert observers.

Methods

Biennial assessments of EBP implementation climate collected in 10 community mental health agencies in Philadelphia as part of a systemwide evaluation (time 1) were linked to subsequent observer ratings of clinician adherence to CBT in clinical encounters with 108 youth (time 2). Experts rated clinician adherence to CBT using the Therapy Process Observation Coding System which generated two primary outcomes (a) maximum CBT adherence per session (i.e., highest rated CBT intervention per session; depth of delivery) and (b) average CBT adherence per session (i.e., mean rating across all CBT interventions used; depth and breadth of delivery).

Results

On average, time 2 clinician adherence observations occurred 19.8 months (SD = 10.15) after time 1 organizational climate assessments. Adjusting for organization, clinician, and client covariates, a one standard deviation increase in organizational EBP implementation climate at time 1 predicted a 0.63-point increase in clinicians’ maximum CBT adherence per session at time 2 (p = 0.000), representing a large effect size (d = 0.93; 95% CI = 0.63–1.24) when comparing organizations in the upper (k = 3) versus lower tertiles (k = 3) of EBP implementation climate. Higher levels of time 1 organizational EBP implementation climate also predicted higher time 2 average CBT adherence per session (b = 0.23, p < 0.001, d = 0.72). Length of time between assessments of climate and adherence did not moderate these relationships.

Conclusions

Organizational EBP implementation climate is a promising predictor of clinicians’ subsequent observed adherence to CBT. Implementation strategies that target this antecedent may improve the delivery of EBPs in healthcare settings.

It's all in the name: why exposure therapy could benefit from a new one

Exposure therapy for anxiety and related disorders is the psychological intervention with the strongest support for its efficacy and effectiveness to date. Yet, it is the least used evidence-based intervention in routine clinical practice, with a long-acknowledged public relations problem. Despite a wealth of research aimed at improving uptake of exposure, exposure's marketing and branding remains an untapped target. We first introduce principles from the marketing literature to propose that the field take steps toward a rebranding and repackaging of exposure therapy to support efforts to implement it widely. Second, we present preliminary data on clinician preferences for the use of alternative terminology developed to be more palatable and marketable - "Supported Approach of Feared Experiences - Cognitive Behavioral Therapy (SAFE-CBT)" - compared to traditional terminology. This initial survey indicated that most clinicians preferred use of the SAFE-CBT term when talking to patients, whereas only a minority preferred it for use among training clinicians. We conclude by discussing implications of these results for future efforts to implement exposure therapy more widely and set an agenda for future research in this space.

Clinician Maladaptive Anxious Avoidance in the Context of Implementation of Evidence-Based Interventions: A Commentary

This paper posits that a clinician's own anxious reaction to delivering specific evidence-based interventions (EBIs) should be better accounted for within implementation science frameworks. A key next step for implementation science is to delineate the causal processes most likely to influence successful implementation of evidence-based interventions (EBIs). This is critical for being able to develop tailored implementation strategies that specifically target mechanisms by which implementation succeeds or fails. First, we review the literature on specific EBIs that may act as negatively valenced stimuli for clinicians, leading to a process of clinician maladaptive anxious avoidance that can negatively impact EBI delivery. In the following sections, we argue that there are certain EBIs that can cause emotional distress or discomfort in a clinician, related to either: (1) a clinicians' fear of the real or predicted short-term distress the EBI can cause patients, or (2) fears that the clinician will inadvertently cause the patient harm and/or face liability. This distress experienced by the clinician can perpetuate a cycle of maladaptive anxious avoidance by the clinician, contributing to lack of or suboptimal EBI implementation. We illustrate how this cycle of maladaptive anxious avoidance can influence implementation by providing several examples from leading EBIs in the psychosocial literature. To conclude, we discuss how leveraging decades of treatment literature aimed at mitigating maladaptive anxious avoidance can inform the design of more tailored and effective implementation strategies for EBIs that are negatively valenced.

A case example of community-based supervision to overcome barriers and support the implementation of exposure therapy

Supervision in community mental health (CMH) settings is critically important for supporting clinical competency and optimizing clinical outcomes. However, CMH supervision is hindered by numerous administrative and fiscal challenges, which makes aligning supervision practices with gold-standard recommendations notoriously difficult. This article describes a case example of one multicomponent supervision approach within a CMH clinic used to support the delivery of exposure therapy ("exposure") to youth with publicly funded insurance. We first review current recommendations for supervision and highlight the primary barriers to the use of these supervisory practices in CMH settings. We then present three low-burden supervisory approaches used to supplement traditional individual supervision. Each strategy presented aims to mitigate known barriers and maximize alignment with gold-standard supervision practices for exposure while being sustainable within a community setting. Finally, we review the strengths and limitations of each supervision strategy. The multicomponent strategies presented represent a creative approach to CMH supervision that maximizes the limited resources and infrastructure in CMH settings to support the delivery of gold-standard supervision. We conclude by offering insights into how these strategies could support supervision for a broader range of evidence-based practices (EBPs) and make recommendations for future research. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Making Progress Monitoring Easier and More Motivating: Developing a Client Data Collection App Incorporating User-Centered Design and Behavioral Economics Insights

Data collection is an important component of evidence-based behavioral interventions for children with autism, but many one-to-one aides (i.e., behavioral support staff) do not systemically collect quantitative data that are necessary for best-practice client progress monitoring. Data collection of clients' behaviors often involves labor-intensive pen-and-paper practices. In addition, the solitary nature of one-to-one work limits opportunities for timely supervisor feedback, potentially reducing motivation to collect data. We incorporated principles from behavioral economics and user-centered design to develop a phone-based application, Footsteps, to address these challenges. We interviewed nine one-to-one aides working with children with autism and seven supervisors to ask for their app development ideas. We then developed the Footsteps app prototype and tested the prototype with 10 one-to-one aides and supervisors through three testing cycles. At each cycle, one-to-one aides rated app usability. Participants provided 76 discrete suggestions for improvement, including 29 new app features (e.g., behavior timer), 20 feature modifications (e.g., numeric type-in option for behavior frequency), four flow modifications (e.g., deleting a redundant form), and 23 out-of-scope suggestions. Of the participants that tested the app, 90% rated usability as good or excellent. Results support continuing to develop Footsteps and testing its impact in a clinical trial.

Predictors of Clinician-Reported Self-Efficacy in Treating Trauma-Exposed Youth

Clinicians' self-efficacy with regard to delivering evidence-based interventions (EBIs) to youth is an important target for both improving EBI use in the community and mitigating the risk of clinician burnout and turnover. Examining predictors of clinician self-efficacy to treat trauma-exposed youth is, therefore, an important step for informing the design of implementation strategies to enhance the mental health workforce's capacity to deliver EBIs in this population. We examined predictors of clinician self-efficacy in working with trauma-exposed youth in a sample of practicing mental health clinicians (N = 258, M age = 34.4 years, 85.0% female). Clinicians were recruited and surveyed as part of a larger study examining how clients' exposure to potentially traumatic events influences clinician decision-making. Results of regression models indicated that training in any trauma treatment model, being trained via a variety of formats (e.g., in-person training, online, supervision), and training in a variety of treatment models were all associated with higher perceived self-efficacy regarding effectively treating trauma-exposed youth. Of the treatment models and training formats examined, receiving in-person training, R2 = .10, and training in trauma-focused cognitive behavioral therapy, R2 = .10, were the strongest predictors of higher self-efficacy ratings. Clinician discipline, R2 = .04, and clinical practice factors, R2 = .20, were also related to self-efficacy. Collectively, the R2 indicated a large effect, with the predictors explaining 25.4% of the variance in self-efficacy ratings. Implications for designing implementation strategies targeting clinician self-efficacy and future research are discussed.

Assessing practical implementation of modular psychotherapy for youth in community-based settings using benchmarking

Background

Achieving high quality outcomes in a community context requires the strategic coordination of many activities in a service system, involving families, clinicians, supervisors, and administrators. In modern implementation trials, the therapy itself is guided by a treatment manual; however, structured supports for other parts of the service system may remain less well-articulated (e.g., supervision, administrative policies for planning and review, information/feedback flow, resource availability). This implementation trial investigated how a psychosocial intervention performed when those non-therapy supports were not structured by a research team, but were instead provided as part of a scalable industrial implementation, testing whether outcomes achieved would meet benchmarks from published research trials.

Method

In this single-arm observational benchmarking study, a total of 59 community clinicians were trained in the Modular Approach to Therapy for Children (MATCH) treatment program. These clinicians delivered MATCH treatment to 166 youth ages 6 to 17 naturally presenting for psychotherapy services. Clinicians received substantially fewer supports from the treatment developers or research team than in the original MATCH trials and instead relied on explicit process management tools to facilitate implementation. Prior RCTs of MATCH were used to benchmark the results of the current initiative. Client improvement was assessed using the Top Problems Assessment and Brief Problem Monitor.

Results

Analysis of client symptom change indicated that youth experienced improvement equal to or better than the experimental condition in published research trials. Similarly, caregiver-reported outcomes were generally comparable to those in published trials.

Conclusions

Although results must be interpreted cautiously, they support the feasibility of using process management tools to facilitate the successful implementation of MATCH outside the context of a formal research or funded implementation trial. Further, these results illustrate the value of benchmarking as a method to evaluation industrial implementation efforts. Plain Language Summary: Randomized effectiveness trials are inclusive of clinicians and cases that are routinely encountered in community-based settings, while continuing to rely on the research team for both clinical and administrative guidance. As a result, the field still struggles to understand what might be needed to support sustainable implementation and how interventions will perform when brought to scale in community settings without those clinical trial supports. Alternative approaches are needed to delineate and provide the clinical and operational support needed for implementation and to efficiently evaluate how evidence-based treatments perform. Benchmarking findings in the community against findings of more rigorous clinical trials is one such approach. This paper offers two main contributions to the literature. First, it provides an example of how benchmarking is used to evaluate how the Modular Approach to Therapy for Children (MATCH) treatment program performed outside the context of a research trial. Second, this study demonstrates that MATCH produced comparable symptom improvements to those seen in the original research trials and describes the implementation strategies associated with this success. In particular, although clinicians in this study had less rigorous expert clinical supervision as compared with the original trials, clinicians were provided with process management tools to support implementation. This study highlights the importance of evaluating the performance of intervention programs when brought to scale in community-based settings. This study also provides support for the use of process management tools to assist providers in effective implementation.

Stakeholder intention to engage in fidelity measurement methods in community mental health settings: A mixed methods study

Background

The current gold standard for measuring fidelity (specifically, adherence) to cognitive behavioral therapy (CBT) is direct observation, a costly, resource-intensive practice that is not feasible for many community organizations to implement regularly. Recent research indicates that behavioral rehearsal (i.e., role-play between clinician and individual with regard to session delivery) and chart-stimulated recall (i.e., brief structured interview between clinician and individual about what they did in session; clinicians use the client chart to prompt memory) may provide accurate and affordable alternatives for measuring adherence to CBT in such settings, with behavioral rehearsal yielding greater correspondence with direct observation.

Methods

Drawing on established causal theories from social psychology and leading implementation science frameworks, this study evaluates stakeholders’ intention to use behavioral rehearsal and chart-stimulated recall. Specifically, we measured attitudes, self-efficacy, and subjective norms toward using each, and compared these factors across the two methods. We also examined the relationship between attitudes, self-efficacy, subjective norms, and intention to use each method. Finally, using an integrated approach we asked stakeholders to discuss their perception of contextual factors that may influence beliefs about using each method. These data were collected from community-based supervisors ( n = 17) and clinicians ( n = 66).

Results

Quantitative analyses suggest moderately strong intention to use both methods across stakeholders. There were no differences in supervisors’ or clinicians’ attitudes, self-efficacy, subjective norms, or intention across methods. More positive attitudes and greater reported subjective norms were associated with greater reported intention to use either measure. Qualitative analyses identified participants’ specific beliefs about using each fidelity measure in their organization, and results were organized using the Consolidated Framework for Implementation Research.

Conclusions

Strategies are warranted to overcome or minimize potential barriers to using fidelity measurement methods and to further increase the strength of intention to use them. Plain Language Summary: The best way to measure fidelity, or how closely a clinician follows the protocol, to Cognitive Behavioral Therapy (CBT) is watching the session. This is an expensive practice that is not feasible for many community organizations to do regularly. Recent research indicates that behavioral rehearsal, or a role-play between the clinician and individual with regard to session delivery, and chart-stimulated recall, or a brief discussion between an individual and the clinician about what they did in session with the clinician having access to the chart to help them remember, may provide accurate and affordable alternatives for measuring fidelity to CBT. We just completed a study demonstrating that both methods are promising, with behavioral rehearsal offering scores that are the most similar to watching the session. Drawing on established theories from social psychology and leading implementation science frameworks, this study evaluates future supervisor and clinician motivation to use these fidelity measurement methods. Specifically, we measured supervisor (n = 17) and clinician (n = 66) attitudes, norms, self-efficacy, intentions, and anticipated barriers and facilitators to using each of these fidelity measurement tools. Quantitative and qualitative analyses suggest similar intention to use both methods, and concerns about barriers to using each method. Further research is warranted to minimize the burden associated with implementing fidelity measurement methods and deploying strategies to increase use.

Increasing access to mental health supports for healthcare workers during the COVID-19 pandemic and beyond through a novel coaching program

The COVID-19 pandemic led to heightened anxiety, distress, and burnout among healthcare workers and faculty in academic medicine. Penn Medicine launched Coping First Aid (CFA) in March 2020 in response to the pandemic. Informed by Psychological First Aid principles and therapeutic micro skills, CFA was designed as a tele-mental healthcare service for health system employees and their families delivered by trained lay volunteer coaches under the supervision of licensed mental health clinicians. We present an overview of the model, feasibility and utilization data, and preliminary implementation and effectiveness outcomes based on cross sectional coach (n = 22) and client (n = 57) self-report surveys with a subset of program users in the first year. A total of 44 individuals completed training and were certified to coach. Over the first 24 months of the program, 513 sessions occurred with 273 clients (119 sessions were no-shows or canceled). Follow-up appointments were recommended in 52.6% (n = 270) of sessions and 21.2% (n = 109) of clients were referred for professional mental health care. Client survey respondents reported CFA was helpful; 60% were very or extremely satisfied, and 74% indicated they would recommend the program. Our preliminary findings suggest that CFA was feasible to implement and most clients found the service beneficial. CFA provides a model for rapidly developing and scaling mental health supports during and beyond the pandemic.

Assessing Causal Pathways and Targets of Implementation Variability for EBP use (Project ACTIVE): a study protocol

Background

Advancing causal implementation theory is critical for designing tailored implementation strategies that target specific mechanisms associated with evidence-based practice (EBP) use. This study will test the generalizability of a conceptual model that integrates organizational constructs and behavioral theory to predict clinician use of cognitive-behavioral therapy (CBT) techniques in community mental health centers. CBT is a leading psychosocial EBP for psychiatric disorders that remains underused despite substantial efforts to increase its implementation.

Methods

We will leverage ongoing CBT implementation efforts in two large public health systems (Philadelphia and Texas) to recruit 300 mental health clinicians and 600 of their clients across 40 organizations. Our primary implementation outcomes of interest are clinician intentions to use CBT and direct observation of clinician use of CBT. As CBT comprises discrete components that vary in complexity and acceptability, we will measure clinician use of six discrete components of CBT. After finishing their CBT training, participating clinicians will complete measures of organizational and behavior change constructs delineated in the model. Clinicians also will be observed twice via audio recording delivering CBT with a client. Within 48 h of each observation, theorized moderators of the intention-behavior gap will be collected via survey. A subset of clinicians who report high intentions to use CBT but demonstrate low use will be purposively recruited to complete semi-structured interviews assessing reasons for the intention-behavior gap. Multilevel path analysis will test the extent to which intentions and determinants of intention predict the use of each discrete CBT component. We also will test the extent to which theorized determinants of intention that include psychological, organizational, and contextual factors explain variation in intention and moderate the association between intentions and CBT use.

Discussion

Project ACTIVE will advance implementation theory, currently in its infancy, by testing the generalizability of a promising causal model of implementation. These results will inform the development of implementation strategies targeting modifiable factors that explain substantial variance in intention and implementation that can be applied broadly across EBPs.

Matched emotional supports in health care (MESH) framework: A stepped care model for health care workers

Frontline health care workers (HCWs) are experiencing a range of emotional responses to the COVID-19 pandemic, including anxiety, traumatic stress, and burnout. As many as half of all HCWs will exhibit clinically significant distress. This distress may endure for years, and health care institutions must respond to these emotional needs. We propose the Matched Emotional Supports in Health Care (MESH) Framework to guide institutions in implementing a tiered, or "stepped care" model for deploying sustainable emotional support programs for HCWs for COVID-19 and beyond. Recognizing the variability in HCWs' response to stress, MESH outlines a continuum of services, including universal (e.g., self-help), selected (e.g., support from trained volunteers), and indicated (e.g., professional therapy, psychotropic medication management) interventions matched to individual need. We provide a targeted review of evidence-based resources available at each level of care and potential processes for determining when higher levels of care are needed. Finally, we delineate key implementation factors for institutions to consider in developing, implementing, and sustaining services for HCWs. Employing the MESH Framework may also facilitate health care institutions' ability to meet the fourth aim of the Quadruple Aims of Health Care to ensure a healthy workforce for this and future crises. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Clinical Supervision in Community Mental Health: Characterizing Supervision as Usual and Exploring Predictors of Supervision Content and Process

Clinical supervision can be leveraged to support implementation of evidence-based practices in community mental health settings, though it has been understudied. This study focuses on 32 supervisors at 23 mental health organizations in Philadelphia. We describe characteristics of supervisors and organizations and explore predictors of supervision content and process. Results highlight a low focus on evidence-based content and low use of active supervision processes. They underscore the need for further attention to the community mental health context when designing supervision-targeted implementation strategies. Future work should assess whether supervision models specific to community mental health are needed.

Treating Social Anxiety in an Era of Social Distancing: Adapting Exposure Therapy for Youth During COVID-19

The novel coronavirus (COVID-19) pandemic has caused widespread disruption to our traditional way of life and mental health therapy has not been spared. A combination of increased anxiety, diminished social opportunities, and the shift to telehealth service provision presents particular challenges for the treatment of social anxiety in youth, which relies heavily on exposures to social situations with peers, adults, or other feared social stimuli. The objective of this commentary is to provide guidance to clinicians working with youth with social anxiety on how to maintain ethical, evidence-informed provision of exposure therapy in light of these unusual circumstances. We first present an overview of how COVID-19 may uniquely impact youth with social anxiety and highlight the importance of continuing to provide exposure-based treatments during this time. We then discuss guiding principles for delivering exposure therapy during COVID-19. We focus on providing practical examples of how common social anxiety exposures can be adapted and delivered successfully through telehealth while abiding by COVID-19 social distancing guidelines. Finally, we discuss key recommendations to assist clinicians in moving treatment forward while considering changing safety guidelines pertaining to COVID-19.

The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development

Background

Developing pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Scale-Self-Reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY), a pragmatic, clinician-report instrument to measure cognitive behavioral therapy (CBT) delivery.

Approach

We describe a five-step community-partnered development process. Initial goals were to create a self-report instrument that paralleled an existing direct observation measure of clinician delivery of CBT use to facilitate later assessment of measure performance. Cognitive interviews with community clinicians (n = 6) and consultation with CBT experts (n = 6) were used to enhance interpretability and usability as part of an iterative refinement process. The instrument was administered to 247 community clinicians along with an established self-reported measure of clinician delivery of CBT and other treatments to assess preliminary psychometric performance. Preliminary psychometrics were promising.

Conclusion

Our community-partnered development process showed promising success and can guide future development of pragmatic implementation measures both to facilitate measurement of ongoing implementation efforts and future research aimed at building learning mental health systems.

Plain language summary

Developing brief, user-friendly, and accurate tools to measure how therapists deliver cognitive behavioral therapy (CBT) in routine practice is important for advancing the reach of CBT into community settings. To date, developing such "pragmatic" measures has been difficult. There is little known about how researchers can best develop these types of assessment tools so that they (1) are easy for clinicians in practice to use and (2) provide valid and useful information about implementation outcomes. As a result, there are few well-validated measures in existence that measure therapist use of CBT that are feasible for use in community practice. This paper contributes to the literature by describing our community-partnered process for developing a measure of therapist use of CBT (Therapy Process Observation Coding Scale -Self-Reported Therapist Intervention Fidelity for Youth; TPOCS-SeRTIFY). This descriptive case study outlines the community-partnered approach we took to develop this measure. This case study will contribute to future research by serving as a guide to others aiming to develop pragmatic implementation measures. In addition, the TPOCS-SeRTIFY is a pragmatic measure of clinician use of CBT that holds promise for its use by both researchers and clinicians to measure the success of CBT implementation efforts.

Training with tarantulas: A randomized feasibility and acceptability study using experiential learning to enhance exposure therapy training

BACKGROUND

Although exposure is a key evidence-based intervention for anxiety, it is infrequently used in clinical settings. This study employed a novel training strategy, experiential learning, to improve exposure implementation. This study aimed to assess the feasibility and acceptability of experiential training and preliminary training effectiveness.

METHODS

Participants were 28 therapists who were randomized to (a) training-as-usual or (b) experiential training (training-as-usual plus a one-session treatment for fear of spiders). Workshops lasted one day and were followed by three months of weekly consultation.

RESULTS

Experiential training was viewed as feasible and acceptable. Participants, including those who were fearful of spiders, had a positive response to the training and reported it to be useful. There was a significant increase in the number of exposures used by therapists receiving experiential training compared to training-as-usual at 1-month follow-up.

CONCLUSIONS

A one-day training resulted in significant improvements in knowledge, attitudes toward exposure, and self-efficacy in using exposure. Preliminary findings suggest that experiential training resulted in greater use of exposure post-training compared to training-as-usual. Results provide evidence for the feasibility and acceptability of experiential training as a strategy to increase the use of evidence-based interventions.

Identifying the organizational innovation-specific capacity needed for exposure therapy

BACKGROUND

Current approaches to increasing the rates of clinician use of exposure therapy for anxiety disorders in community settings are limited. Research underscores the importance of addressing contextual variables to facilitate clinician use of evidence-based practices; however, no studies have identified the innovation-specific organizational capacity necessary to implement exposure therapy. Such work is critical to ensure that treatment-seeking individuals with anxiety receive effective care.

METHODS

We used a two-step process to identify the innovation-specific organizational capacity necessary to deliver exposure. First, 24 leaders of specialty anxiety clinics in the United States (50% female, mean [M]age = 47.7 years) completed a survey about the organizational innovation-specific capacity (e.g., policies and procedures) they employ to support their providers in delivering exposure therapy. Second, 19 community clinicians (79% female, M age = 42.9 years) reported on the extent to which these characteristics were present in their settings.

RESULTS

In Step 1, specialty clinic leaders unanimously endorsed six organizational characteristics as essential and five as important within the areas of organizational policies, supervisory support, and peer clinician support. These characteristics were present in more than 90% of specialty clinics. In Step 2, therapists in community clinics reported these characteristics were minimally present in their organizations.

CONCLUSIONS

Specialty clinic leaders exhibited consensus on the innovation-specific organizational capacity necessary to implement exposure therapy. Identified characteristics were largely absent from community clinics. Developing fiscal, policy, or organizational strategies that enhance the organizational capacity within community settings may improve the patients' access to effective treatment for anxiety disorders.

Implementing nudges for suicide prevention in real-world environments: project INSPIRE study protocol

Background

Suicide is a global health issue. There are a number of evidence-based practices for suicide screening, assessment, and intervention that are not routinely deployed in usual care settings. The goal of this study is to develop and test implementation strategies to facilitate evidence-based suicide screening, assessment, and intervention in two settings where individuals at risk for suicide are especially likely to present: primary care and specialty mental health care. We will leverage methods from behavioral economics, which involves understanding the many factors that influence human decision making, to inform strategy development.

Methods

We will identify key mechanisms that limit implementation of evidence-based suicide screening, assessment, and intervention practices in primary care and specialty mental health through contextual inquiry involving behavioral health and primary care clinicians. Second, we will use contextual inquiry results to systematically design a menu of behavioral economics-informed implementation strategies that cut across settings, in collaboration with an advisory board composed of key stakeholders (i.e., behavioral economists, clinicians, implementation scientists, and suicide prevention experts). Finally, we will conduct rapid-cycle trials to test and refine the menu of implementation strategies. Primary outcomes include clinician-reported feasibility and acceptability of the implementation strategies.

Discussion

Findings will elucidate ways to address common and unique barriers to evidence-based suicide screening, assessment, and intervention practices in primary care and specialty mental health care. Results will yield refined, pragmatically tested strategies that can inform larger confirmatory trials to combat the growing public health crisis of suicide.

Therapist Financial Strain and Turnover: Interactions with System-Level Implementation of Evidence-Based Practices

Therapist turnover is a major problem in community mental health. Financial strain, which is composed of cognitive, emotional, and behavioral responses to the experience of economic hardship, is an understudied antecedent of therapist turnover given the tumultuous financial environment in community mental health. We prospectively examined the relationship between therapist financial strain and turnover in 247 therapists in 28 community mental health agencies. We expected greater therapist financial strain to predict higher turnover and participation in a system-funded evidence-based practice (EBP) training initiative to alleviate this effect. Controlling for covariates, financial strain predicted therapist turnover (OR 1.12, p = .045), but not for therapists who participated in an EBP training initiative. Reducing financial strain and/or promoting EBP implementation may be levers to reduce turnover.

Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes

A lack of effective therapist training is a major barrier to evidence-based intervention (EBI) delivery in the community. Systematic reviews published nearly a decade ago suggested that traditional EBI training leads to higher knowledge but not more EBI use, indicating that more work is needed to optimize EBI training and implementation. This systematic review synthesizes the training literature published since 2010 to evaluate how different training models (workshop, workshop with consultation, online training, train-the-trainer, and intensive training) affect therapists' knowledge, beliefs, and behaviors. Results and limitations for each approach are discussed. Findings show that training has advanced beyond provision of manuals and brief workshops; more intensive training models show promise for changing therapist behavior. However, methodological issues persist, limiting conclusions and pointing to important areas for future research.

Testing a theory of strategic implementation leadership, implementation climate, and clinicians' use of evidence-based practice: a 5-year panel analysis

BACKGROUND

Implementation theory suggests that first-level leaders, sometimes referred to as middle managers, can increase clinicians' use of evidence-based practice (EBP) in healthcare settings by enacting specific leadership behaviors (i.e., proactive, knowledgeable, supportive, perseverant with regard to implementation) that develop an EBP implementation climate within the organization; however, longitudinal and quasi-experimental studies are needed to test this hypothesis.

METHODS

Using data collected at three waves over a 5-year period from a panel of 30 outpatient children's mental health clinics employing 496 clinicians, we conducted a quasi-experimental difference-in-differences study to test whether within-organization change in implementation leadership predicted within-organization change in EBP implementation climate, and whether change in EBP implementation climate predicted within-organization change in clinicians' use of EBP. At each wave, clinicians reported on their first-level leaders' implementation leadership, their organization's EBP implementation climate, and their use of both EBP and non-EBP psychotherapy techniques for childhood psychiatric disorders. Hypotheses were tested using econometric two-way fixed effects regression models at the organization level which controlled for all stable organizational characteristics, population trends in the outcomes over time, and time-varying covariates.

RESULTS

Organizations that improved from low to high levels of implementation leadership experienced significantly greater increases in their level of EBP implementation climate (d = .92, p = .017) and within-organization increases in implementation leadership accounted for 11% of the variance in improvement in EBP implementation climate beyond all other covariates. In turn, organizations that improved from low to high levels of EBP implementation climate experienced significantly greater increases in their clinicians' average EBP use (d = .55, p = .007) and within-organization improvement in EBP implementation climate accounted for 14% of the variance in increased clinician EBP use. Mediation analyses indicated that improvement in implementation leadership had a significant indirect effect on clinicians' EBP use via improvement in EBP implementation climate (d = .26, 95% CI [.02 to .59]).

CONCLUSIONS

When first-level leaders increase their frequency of implementation leadership behaviors, organizational EBP implementation climate improves, which in turn contributes to increased EBP use by clinicians. Trials are needed to test strategies that target this implementation leadership-EBP implementation climate mechanism.

Evidence Base Update for Brief, Free, and Accessible Youth Mental Health Measures

Evidence-based assessment (EBA) is foundational to high-quality mental health care for youth and is a critical component of evidence-based practice delivery, yet is underused in the community. Administration time and measure cost are barriers to use; thus, identifying and disseminating brief, free, and accessible measures are critical. This Evidence Base Update evaluates the empirical literature for brief, free, and accessible measures with psychometric support to inform research and practice with youth. A systematic review using PubMed and PsycINFO identified measures in the following domains: overall mental health, anxiety, depression, disruptive behavior, traumatic stress, disordered eating, suicidality, bipolar/mania, psychosis, and substance use. To be eligible for inclusion, measures needed to be brief (50 items or less), free, accessible, and have psychometric support for their use with youth. Eligible measures were evaluated using adapted criteria established by De Los Reyes and Langer (2018) and were classified as having excellent, good, or adequate psychometric properties. A total of 672 measures were identified; 95 (14%) met inclusion criteria. Of those, 21 (22%) were "excellent," 34 (36%) were "good," and 40 (42%) were "adequate." Few measures had support for their use to routinely monitor progress in therapy. Few measures with excellent psychometric support were identified for disordered eating, suicidality, psychosis, and substance use. Future research should evaluate existing measures for use with routine progress monitoring and ease of implementation in community settings. Measure development is needed for disordered eating, suicidality, psychosis, and substance use to increase availability of brief, free, accessible, and validated measures.

Variability in clinician intentions to implement specific cognitive-behavioral therapy components

BACKGROUND

CBT comprises many discrete components that vary in complexity, but implementation and training efforts often approach CBT as a single entity. We examined variability in clinician intentions to use different structural and interventional components of CBT for three different clinical groups: clients receiving CBT, clients with depression, and clients with anxiety.

METHODS

Clinicians (n = 107) trained in CBT completed a one-time electronic survey. Clinicians' intentions were measured using established item stems from social psychology adapted to examine intentions to use six specific CBT components: exposure therapy, cognitive restructuring, behavioral activation, planning homework, reviewing homework, and agenda-setting.

RESULTS

Intentions were weakest, on average, for exposure. They were strongest, on average, for reviewing homework. A series of ANOVAs with Tukey's post-hoc tests revealed that participants intended to use exposure with clients receiving CBT (p = .015) and clients with anxiety (p < .001) significantly more than for clients with depression. Participants intended to use behavioral activation with clients with depression (p = .01) significantly more than for clients with anxiety. No other intentions to use CBT components differed among these three clinical populations.

CONCLUSIONS

When studying determinants of CBT use and designing interventions to increase use, implementers should consider that different CBT components may require different implementation strategies.

TRIAL REGISTRATION

Not applicable.

Money makes the world go 'round: A qualitative examination of the role funding plays in large-scale implementation and sustainment of youth evidence-based practice

Funding is posited to affect evidence-based practice implementation, yet the complex interplay between financial matters and successful implementation is understudied. This study examined stakeholder perspectives on the impact of funding in evidence-based practice implementation. All participants were key stakeholders (e.g., clinicians, case managers, agency leaders; N = 41) involved in a trauma-focused cognitive-behavioral therapy implementation effort using a community-based learning collaborative model within the community's child welfare system. Semistructured interviews were conducted and qualitatively analyzed as part of a program evaluation of the implementation effort. Funding emerged as a key theme influencing implementation within this program evaluation from the perspective of all stakeholders. Thirty-four participants (83%) independently raised funding as an important factor affecting implementation outcomes across seven specific themes: (a) the impact of privatization, (b) turfism, (c) money as a primary implementation facilitator, (d) implementation costs impacting participation, (e) burden associated with funding evaluation efforts, (f) need for reimbursement practices to align with the use of trauma-informed treatment, and (g) a sense of shared mission to serve clients above money. Recommendations for addressing these challenges are provided. Future research should examine funding qualitatively and quantitatively across diverse communities and funding systems to improve understanding of the impact of funding on implementation and, ultimately, care provided to clients. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Characterizing the heterogeneity of clinician practice use in community mental health using latent profile analysis

BACKGROUND

The behavioral health service provider population is highly heterogeneous. However, it is rarely treated as such within evidence-based practice implementation efforts. This study aimed to evaluate, as a proof of concept, the utility of latent profile analysis to identify distinct profiles of clinician practices in a large sample of youth-serving community mental health clinicians. This study also aimed to identify predictors of profile membership to inform implementation efforts.

METHODS

Participants were 484 practicing clinicians (79.4% female, 45.7% White, M age = 37.1 years). As part of a larger survey, clinicians reported on their use of cognitive, behavioral, family, and psychodynamic treatment techniques with a representative client on their caseload. Latent profile analysis was used to determine the presence of clinician practice profiles. Multilevel multinomial logistic regressions examined predictors of profile membership.

RESULTS

Latent profile analysis indicated a 4-profile solution best fit the data, with clinicians who: 1) used generally low levels of all examined techniques and preferred cognitive techniques (Low Eclectics, 16%), 2) delivered moderate levels of all techniques (Moderate Eclectics, 53%), 3) demonstrated preference for use of family techniques (Family Preferred, 11%), and 4) used high levels of all techniques (Super Users, 20%). Clinician discipline (e.g., social work), education, and years of experience predicted profile membership.

CONCLUSIONS

Findings from this proof of concept study underscore the utility of latent profile analysis to characterize the complex and heterogeneous makeup of community mental health. Results extend prior work highlighting the eclectic nature of community mental health practice. Predictor analyses underscore the important influence of clinician background characteristics on practice use.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia

Background

Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static.

Method

Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia’s public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised.

Results

Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048).

Conclusions

System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

Individualized Progress Measures Are More Acceptable to Clinicians Than Standardized Measures: Results of a National Survey

Despite research supporting measurement-based care grounded in standardized progress measures, such measures are underutilized by clinicians. Individualized measures of client-specific targets present an alternative, but little is known about their acceptability or use. We compared attitudes toward and use of standardized and individualized progress measures in a national sample of 504 clinicians. Clinicians reported neutral to positive attitudes toward both types of measures, but strongly preferred and were more likely to use individualized measures. Clinician attitudes, theoretical orientation, and work setting predicted assessment preferences and practices. Implications for dissemination and implementation of measurement-based care are discussed.

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate

Organizational culture and climate are important determinants of behavioral health service delivery for youth. The Organizational Social Context measure is a well validated assessment of organizational culture and climate that has been developed and extensively used in public sector behavioral health service settings. The degree of concordance between administrators and clinicians in their reports of organizational culture and climate may have implications for research design, inferences, and organizational intervention. However, the extent to which administrators' and clinicians' reports demonstrate concordance is just beginning to garner attention in public behavioral health settings in the United States. We investigated the concordance between 73 administrators (i.e., supervisors, clinical directors, and executive directors) and 247 clinicians in 28 child-serving programs in a public behavioral health system. Findings suggest that administrators, compared to clinicians, reported more positive cultures and climates. Organizational size moderated this relationship such that administrators in small programs (<466 youth clients served annually) provided more congruent reports of culture and climate in contrast to administrators in large programs (≥466 youth clients served annually) who reported more positive cultures and climates than clinicians. We propose a research agenda that examines the effect of concordance between administrators and clinicians on organizational outcomes in public behavioral health service settings.

Profiles and Correlates of Parent-Child Agreement on Social Anxiety Symptoms in Youth with Autism Spectrum Disorder

This study characterized patterns and correlates of parent-youth agreement on social anxiety in youth with and without autism spectrum disorder (ASD). Participants (279 verbally-fluent youth aged 8-16 years, N_ASD = 144, N_TD = 135) completed the SASC-R. Youth with ASD exhibited higher social anxiety across informants. While TD youth endorsed higher anxiety than did parents, self- and parent-reports did not differ in youth with ASD. For children with ASD, higher parent-youth agreement was associated with lower lifetime ASD symptoms and higher adaptive skills. For TD youth, agreement on high anxiety was associated with lowest adaptive skills. Demographic factors (age, verbal IQ, gender) did not relate to agreement for either group. In ASD, parent-child agreement on youth anxiety, either high or low, was associated with better outcomes.

Transforming Mental Health Delivery Through Behavioral Economics and Implementation Science: Protocol for Three Exploratory Projects

BACKGROUND

Efficacious psychiatric treatments are not consistently deployed in community practice, and clinical outcomes are attenuated compared with those achieved in clinical trials. A major focus for mental health services research is to develop effective and cost-effective strategies that increase the use of evidence-based assessment, prevention, and treatment approaches in community settings.

OBJECTIVE

The goal of this program of research is to apply insights from behavioral economics and participatory design to advance the science and practice of implementing evidence-based practice (EBP) for individuals with psychiatric disorders across the life span.

METHODS

Project 1 (Assisting Depressed Adults in Primary care Treatment [ADAPT]) is patient-focused and leverages decision-making heuristics to compare ways to incentivize adherence to antidepressant medications in the first 6 weeks of treatment among adults newly diagnosed with depression. Project 2 (App for Strengthening Services In Specialized Therapeutic Support [ASSISTS]) is provider-focused and utilizes normative pressure and social status to increase data collection among community mental health workers treating children with autism. Project 3 (Motivating Outpatient Therapists to Implement: Valuing a Team Effort [MOTIVATE]) explores how participatory design can be used to design organizational-level implementation strategies to increase clinician use of EBPs. The projects are supported by a Methods Core that provides expertise in implementation science, behavioral economics, participatory design, measurement, and associated statistical approaches.

RESULTS

Enrollment for project ADAPT started in 2018; results are expected in 2020. Enrollment for project ASSISTS will begin in 2019; results are expected in 2021. Enrollment for project MOTIVATE started in 2018; results are expected in 2019. Data collection had begun for ADAPT and MOTIVATE when this protocol was submitted.

CONCLUSIONS

This research will advance the science of implementation through efforts to improve implementation strategy design, measurement, and statistical methods. First, we will test and refine approaches to collaboratively design implementation strategies with stakeholders (eg, discrete choice experiments and innovation tournaments). Second, we will refine the measurement of mechanisms related to heuristics used in decision making. Third, we will develop new ways to test mechanisms in multilevel implementation trials. This trifecta, coupled with findings from our 3 exploratory projects, will lead to improvements in our knowledge of what causes successful implementation, what variables moderate and mediate the effects of those causal factors, and how best to leverage this knowledge to increase the quality of care for people with psychiatric disorders.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03441399; https://www.clinicaltrials.gov/ct2/show/NCT03441399 (Archived by WebCite at http://www.webcitation.org/74dRbonBD).

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/12121.