A Tale of Two Taxes: Implementation of Earmarked Taxes for Behavioral Health Services in California and Washington State

OBJECTIVE

The authors sought to characterize perceptions of the impacts, attributes, and support for taxes earmarked for behavioral health services and to compare perceptions of the taxes among professionals in California and Washington, two states differing in earmarked tax designs.

METHODS

Surveys were completed by 155 public agency and community organization professionals involved in tax implementation in California (N=87) and Washington State (N=68) during 2022-2023 (29% response rate). Respondents indicated their perceptions of the taxes' impacts, attributes, and support. Responses were summed as aggregate scores and were also analyzed as individual items. Bivariate analyses were used to compare responses of professionals in California versus Washington State.

RESULTS

Earmarked taxes were generally regarded positively. Of the respondents, >80% strongly agreed that the taxes increased funding for services and were helpful, and only 10% strongly agreed that the taxes decreased behavioral health funding from other sources. Substantially more respondents in California than in Washington State strongly agreed that taxes' reporting requirements were complicated (45% vs. 5%, p<0.001) and that the taxes increased unjustified scrutiny of services or systems (33% vs. 2%, p<0.001). However, more respondents in California than in Washington State also strongly agreed that the taxes increased public awareness about behavioral health (56% vs. 15%, p<0.001) and decreased behavioral health stigma (47% vs. 14%, p<0.001).

CONCLUSIONS

Perceptions of the strengths and weaknesses of taxes earmarked for behavioral health services may vary by design features of the tax. Such features include stigma-reduction initiatives and tax spending and reporting requirements.

Acceptability and feasibility of policy implementation strategies for taxes earmarked for behavioral health services

Background

This study's aims are to: (1) Compare the acceptability and feasibility of five types of implementation strategies that could be deployed to increase the reach of evidence-based practices (EBPs) with revenue from policies that earmark taxes for behavioral health services, and (2) Illustrate how definitions of implementation strategies and measures of acceptability and feasibility can be used in policy-focused implementation science research.

Methods

Web-based surveys of public agency and community organization professionals involved with earmarked tax policy implementation were completed in 2022-2023 (N = 211, response rate = 24.9%). Respondents rated the acceptability and feasibility of five types of implementation strategies (dissemination, implementation process, integration, capacity-building, and scale-up). Aggregate acceptability and feasibility scores were calculated for each type of strategy (scoring range 4-20). Analyses of variance compared scores across strategies and between organizational actor types.

Findings

For acceptability, capacity-building strategies had the highest rating (M = 16.3, SD = 3.0), significantly higher than each of the four other strategies, p ≤ . 004), and scale-up strategies had the lowest rating (M = 15.6). For feasibility, dissemination strategies had the highest rating (M = 15.3, significantly higher than three of the other strategies, p ≤ .002) and scale-up strategies had the lowest rating (M = 14.4).

Conclusions

Capacity-building and dissemination strategies may be well-received and readily deployed by policy implementers to support EBPs implementation with revenue from taxes earmarked for behavioral health services. Adapting definitions of implementation strategies for policy-focused topics, and applying established measures of acceptability and feasibility to these strategies, demonstrates utility as an approach to advance research on policy-focused implementation strategies.

Variability in the Integration of Peers in a Multi-site Digital Mental Health Innovation Project

Peer support specialists ("peers") who have the lived experience of, and are in recovery from, mental health challenges are increasingly being integrated into mental health care as a reimbursable service across the US. This study describes the ways peers were integrated into Help@Hand, a multi-site innovation project that engaged peers throughout efforts to develop and offer digital mental health interventions across counties/cities ("sites") in California. Using a mixed methods design, we collected quantitative data via quarterly online surveys, and qualitative data via semi-annual semi-structured phone interviews with key informants from Help@Hand sites. Quantitative data were summarized as descriptive findings and qualitative data from interviews were analyzed using rapid qualitative analysis methods. In the final analytic phase, interview quotes were used to illustrate the complex realities underlying quantitative responses. 117 quarterly surveys and 46 semi-annual interviews were completed by key informants from 14 sites between September 2020 and January 2023. Peers were integrated across diverse activities for support and implementation of digital mental health interventions, including development of training and educational materials (78.6% of sites), community outreach (64.3%), technology testing (85.7%), technology piloting (90.9%), digital literacy training (71.4%), device distribution (63.6%), technical assistance (72.7%), and cross-site collaboration (66.7%). Peer-engaged activities shifted over time, reflecting project phases. Peer-provided digital literacy training and technology-related support were key ingredients for project implementations. This study indicates the wide range of ways peers can be integrated into digital mental health intervention implementations. Considering contextual readiness for peer integration may enhance their engagement into programmatic activities.

Adaptation of the brainwriting premortem technique to inform the co-creation of COVID-19 testing strategies in underserved communities in South San Diego

INTRODUCTION

Meaningful engagement of partners in co-creating and refining health-related programs can increase the initial uptake, sustained implementation, broad reach, and effectiveness of these programs. This is especially important for underserved communities where resources are limited and need to be prioritized. Brainwriting premortem is a novel qualitative approach to partner engagement that combines the strengths of individual idea generation with the concept of premortem exercise that addresses failure points prior to the implementation of new programs.

METHODS

An adapted form of brainwriting premortem was used to inform iterative refinements to a COVID-19 testing program at a Federally Qualified Health Center (FQHC) in San Diego. Patients and providers from the FQHC participated in interviews at two time points (early- and mid-implementation of the program). Interview data were transcribed, translated, and analyzed using a rapid qualitative approach. Key themes and sub-themes were identified and used to inform refinements to the program.

RESULTS

A total of 11 patients (7 Spanish- and 4 English-speaking) and 8 providers participated in the brainwriting premortem interviews. Key themes related to possible reasons for COVID-19 testing program failure: advertising/sharing information; access to testing; handling of test results; staff and patient safety; patient beliefs and views regarding the SARS-CoV-2 virus; and COVID-19 testing options offered. Proposed solutions were offered for the key failures except for patient beliefs and views regarding the SARS-CoV-2 virus. Additional solutions offered were related to education, physical operations, and recruitment strategies. Real-time changes to the program flow and components were made in response to 7 suggestions from patients and 11 from providers. Changes related to the process of returning results were the most common, and included sending results via email with distinct workflows based on the test result.

CONCLUSION

The implementation of the adapted brainwriting premortem technique allowed us to incorporate the perspective of key partners in the delivery and iterative refinement of the COVID-19 testing program. This was an effective tool in the context of an FQHC and can be a promising and approach to incorporate iterative input from patients and providers to ensure successful program implementation. Future studies, particularly those requiring rapid response to public health emergencies, should consider the use of this technique.

Real-World Adoption of Mental Health Support Among Adolescents: Cross-Sectional Analysis of the California Health Interview Survey

OBJECTIVE

We aim to examine: (a) the extent to which patterns of adoption of counseling services and digital mental health interventions (DMHIs) shifted in recent years (2019-2021); (b) the impact of distress on adoption of mental health support; and (c) reasons related to adolescents' low adoption of DMHIs when experiencing distress.

METHODS

Data were from three cohorts of adolescents aged 12-17 years (n = 847 in 2019; n = 1,365 in 2020; n = 1,169 in 2021) recruited as part of the California Health Interview Survey. We estimated logistic regression models to examine the likelihood of using mental health support as a function of psychological distress, sociodemographic characteristics, and cohorts. We also analyzed adolescents' self-reported reasons for not trying DMHIs as a function of distress.

RESULTS

The proportion of adolescents reporting elevated psychological distress (∼50%) was higher than those adopting counseling services (<20%) or DMHIs (<10%). A higher level of distress was associated with a greater likelihood of receiving counseling (OR = 1.15), and using DMHIs to connect with a professional (Odds ratio (OR) = 1.11) and for self-help (OR = 1.17). Among those experiencing high distress, adolescents' top reason for not adopting an online tool was a lack of perceived need (19.2%).

CONCLUSION

Adolescents' main barriers to DMHI adoption included a lack of perceived need, which may be explained by a lack of mental health literacy. Thoughtful marketing and dissemination efforts are needed to increase mental health awareness and normalize adoption of counseling services and DMHIs.

Community-engaged optimization of COVID-19 rapid evaluation and testing experiences: roll-out implementation optimization trial

BACKGROUND

There continues to be a need for COVID-19 testing that is pragmatic, community-centered, and sustainable. This study will refine and test implementation strategies prioritized by community partners: (1) walk-up no-cost testing, (2) community health worker (promotores)-facilitated testing and preventive care counseling, (3) vending machines that dispense no-cost, self-testing kits.

METHODS

A co-designed Theory of Change from an earlier study phase and the Practical, Robust Implementation and Sustainment Model (PRISM) will guide the study design, measures selection, and evaluation. The first aim is to refine and operationalize a multi-component implementation strategy bundle and outcome measures for COVID-19 testing. A Community and Scientific Advisory Board (CSAB) will be established and include community members, clinical providers/staff from the partnering Federally Qualified Health Center (FQHC), public health researchers, policymakers, and a county health department ambassador. Engagement of CSAB members will be assessed through structured ethnography and a survey about the quality and quantity of engagement practices. The second aim is to implement and evaluate the impact of the implementation strategy bundle to optimize COVID-19 testing in communities using a roll-out implementation optimization (ROIO) design. Seven thousand and five hundred community members will be enrolled across four FQHC clinics over 18 months. Participants will be invited to complete an electronic survey about their demographics, health, and COVID-19 testing results and experiences. CSAB members and clinic partners will participate in PRISM fit and determinant assessments prior to each clinic rollout and post-trial. Interviews will be conducted with 60 community participants and 12 providers/staff following a 3-month rollout period at each clinic, inquiring about their experiences with the implementation strategies. Quantitative data will be analyzed using hierarchical multilevel models to determine the impact of implementation strategies. Qualitative data will be analyzed using rapid qualitative approaches to summarize implementation experiences and identify necessary changes prior to subsequent rollouts. A matrix approach will be used to triangulate data from quantitative and qualitative sources based on PRISM domains.

DISCUSSION

This is one of the first pragmatic implementation trials to use a ROIO design and aims to co-create a sustainable and equitable COVID-19 testing program. Findings are likely to generalize to other public health prevention efforts.

TRIAL REGISTRATION

NCT05894655 March 2, 2023.

Process evaluation of an academic dissemination and implementation science capacity building program

The UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center (DISC) launched in 2020 to provide dissemination and implementation science (DIS) training, technical assistance, community engagement, and research advancement. DISC developed a program-wide logic model to inform a process evaluation of member engagement and impact related to DISC services. The DISC Logic Model (DLM) served as the framework for a process evaluation capturing quantitative and qualitative information about scientific activities, outputs, and outcomes. The evaluation involved a multimethod approach with surveys, attendance tracking, feedback forms, documentation of grant outcomes, and promotions metrics (e.g., Twitter engagement). There were 540 DISC Members at the end of year 2 of the DISC. Engagement in the DISC was high with nearly all members endorsing at least one scientific activity. Technical assistance offerings such as DISC Journal Club and consultation were most frequently used. The most common scientific outputs were grant submission (65, 39%), formal mentoring for career award (40, 24%), and paper submission (34, 21%). The DLM facilitated a comprehensive process evaluation of our center. Actionable steps include prioritizing technical assistance, strengthening networking opportunities, identifying streamlined approaches to facilitate DIS grant writing through writing workshops, as well as "office hours" or organized writing leagues.

Factors associated with COVID-19 vaccine uptake in a US/Mexico border community: demographics, previous influenza vaccination, and trusted sources of health information

Background

COVID-19 vaccine uptake has been uneven, particularly across racial/ethnic and age groups. This study seeks to understand factors associated with COVID-19 vaccine uptake in a large cross-sectional sample of predominantly Latinos/Latinas individuals living near the US/Mexico border.

Methods

Data are extracted from a 176-item survey conducted as part of a parent study focused on the co-creation of a COVID-19 testing program for underserved communities developed through a partnership between an academic institution and a Federally Qualified Health Center. The following participant variables were examined: health history, COVID-19 symptoms, COVID-19 testing and vaccine experiences, and perceptions of sources of health information. Participant characteristics were compared using chi-square tests. Multivariate logistic regressions were used for the final statistical model.

Results

From 1 May 2021 to 30 April 2022, 4,964 adults, 66% of whom were identified as women, completed the survey. Approximately 80% of participants reported having received at least one COVID-19 vaccine. Female sex, older age, Hispanic/Latino(a) ethnicity, previous influenza vaccination, advanced education, and perceived elevated risk of COVID-19 were significantly (p < 0.05) associated with having received a COVID-19 vaccine. Regarding sources of health information, individuals who indicated they trust their doctor, healthcare provider, or the US government "a great deal" were more likely to have received a COVID-19 vaccine compared to individuals who indicated that they trusted these sources "not at all." In contrast, those who reported having "a great deal" of trust in their faith leader or their social media contacts were significantly less likely to have received a COVID-19 vaccine than those who reported that they trusted these sources "not at all."

Conclusion

Sex, education, past influenza vaccination, perceived risk of COVID-19 infection, and trust in specific sources of information were correlated with the uptake of COVID-19 vaccination. Additional research is needed to better understand why this confluence of factors, particularly the unique findings about trusted sources of information, are associated with vaccine uptake. Understanding these associations, specifically within underserved, Latino/Hispanic communities, is an important first step to inform efforts aimed at increasing and sustaining COVID-19 vaccine uptake and adoption of other public health interventions.

Scaling and sustaining COVID-19 vaccination through meaningful community engagement and care coordination for underserved communities: hybrid type 3 effectiveness-implementation sequential multiple assignment randomized trial

BACKGROUND

COVID-19 inequities are abundant in low-income communities of color. Addressing COVID-19 vaccine hesitancy to promote equitable and sustained vaccination for underserved communities requires a multi-level, scalable, and sustainable approach. It is also essential that efforts acknowledge the broader healthcare needs of these communities including engagement in preventive services.

METHODS

This is a hybrid type 3 effectiveness-implementation study that will include a multi-level, longitudinal, mixed-methods data collection approach designed to assess the sustained impact of a co-created multicomponent strategy relying on bidirectional learning, shared decision-making, and expertise by all team members. The study capitalizes on a combination of implementation strategies including mHealth outreach with culturally appropriate messaging, care coordination to increase engagement in high priority preventive services, and the co-design of these strategies using community advisory boards led by Community Weavers. Community Weavers are individuals with lived experience as members of an underserved community serving as cultural brokers between communities, public health systems, and researchers to co-create community-driven, culturally sensitive public health solutions. The study will use an adaptive implementation approach operationalized in a sequential multiple assignment randomized trial design of 300 participants from three sites in a Federally Qualified Health Center in Southern California. This design will allow examining the impact of various implementation strategy components and deliver more intensive support to those who benefit from it most. The primary effectiveness outcomes are COVID-19 vaccine completion, engagement in preventive services, and vaccine confidence. The primary implementation outcomes are reach, adoption, implementation, and maintenance of the multicomponent strategy over a 12-month follow-up period. Mixed-effects logistic regression models will be used to examine program impacts and will be triangulated with qualitative data from participants and implementers.

DISCUSSION

This study capitalizes on community engagement, implementation science, health equity and communication, infectious disease, and public health perspectives to co-create a multicomponent strategy to promote the uptake of COVID-19 vaccination and preventive services for underserved communities in San Diego. The study design emphasizes broad engagement of our community and clinic partners leading to culturally sensitive and acceptable strategies to produce lasting and sustainable increases in vaccine equity and preventive services engagement.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT05841810 May 3, 2023.

Recommendations for post-implementation adaptations to optimize family navigation in pediatric primary care: a qualitative study with parents and navigators

BACKGROUND

Family Navigation (FN) is an evidence-based care management intervention designed to reduce disparities in access to care by providing families with individually tailored support and care coordination. Early data suggest FN is effective, but effectiveness is significantly influenced by both contextual (e.g. setting) and individual (e.g., ethnicity) variables. To better understand how FN could be tailored to address this variability in effectiveness, we set forth to explore proposed adaptations to FN by both navigators and families who received FN.

METHODS

This study was a nested qualitative study set within a larger randomized clinical trial of FN to improve access to autism diagnostic services in urban pediatric primary care practices in Massachusetts, Pennsylvania, and Connecticut serving low-income, racial and ethnic minority families. Following FN implementation, key informant interviews were conducted based on the Framework for Reporting Adaptations and Modifications-Expanded (FRAME) with a purposeful sample of parents of children who received FN (n = 21) and navigators (n = 7). Interviews were transcribed verbatim and were coded using framework-guided rapid analysis to categorize proposed adaptations to FN.

RESULTS

Parents and navigators proposed 38 adaptations in four domains: 1) content of the intervention (n = 18), 2) context of the intervention (n = 10), 3) training and evaluation (n = 6), and 4) implementation and scale-up (n = 4). The most frequently endorsed adaptation recommendations focused on content (e.g., lengthening FN, providing parents with additional education on autism and parenting children with autism) and implementation (e.g., increasing access to navigation). Although probes targeted critical feedback, parents and navigators were overwhelmingly positive about FN.

CONCLUSIONS

This study builds upon prior FN effectiveness and implementation research by providing concrete areas for adaptation and refinement of the intervention. Recommendations by parents and navigators have the potential to inform improvement of existing navigation programs and development of new programs in similarly underserved populations. These findings are critical as adaptation (cultural and otherwise) is an important principle in the field of health equity. Ultimately, adaptations will need to be tested to determine clinical and implementation effectiveness.

TRIAL REGISTRATION

ClinicalTrials.gov, registration number NCT02359084, February 9, 2015.

Facilitators of and Barriers to Integrating Digital Mental Health Into County Mental Health Services: Qualitative Interview Analyses

BACKGROUND

Digital mental health interventions (DMHIs) represent a promising solution to address the growing unmet mental health needs and increase access to care. Integrating DMHIs into clinical and community settings is challenging and complex. Frameworks that explore a wide range of factors, such as the Exploration, Preparation, Implementation, Sustainment (EPIS) framework, can be useful for examining multilevel factors related to DMHI implementation efforts.

OBJECTIVE

This paper aimed to identify the barriers to, facilitators of, and best practice recommendations for implementing DMHIs across similar organizational settings, according to the EPIS domains of inner context, outer context, innovation factors, and bridging factors.

METHODS

This study stems from a large state-funded project in which 6 county behavioral health departments in California explored the use of DMHIs as part of county mental health services. Our team conducted interviews with clinical staff, peer support specialists, county leaders, project leaders, and clinic leaders using a semistructured interview guide. The development of the semistructured interview guide was informed by expert input regarding relevant inner context, outer context, innovation factors, and bridging factors in the exploration, preparation, and implementation phases of the EPIS framework. We followed a recursive 6-step process to conduct qualitative analyses using inductive and deductive components guided by the EPIS framework.

RESULTS

On the basis of 69 interviews, we identified 3 main themes that aligned with the EPIS framework: readiness of individuals, readiness of innovations, and readiness of organizations and systems. Individual-level readiness referred to the extent to which clients had the necessary technological tools (eg, smartphones) and knowledge (digital literacy) to support the DMHI. Innovation-level readiness pertained to the accessibility, usefulness, safety, and fit of the DMHI. Organization- and system-level readiness concerned the extent to which providers and leadership collectively held positive views about DMHIs as well as the extent to which infrastructure (eg, staffing and payment model) was appropriate.

CONCLUSIONS

The successful implementation of DMHIs requires readiness at the individual, innovation, and organization and system levels. To improve individual-level readiness, we recommend equitable device distribution and digital literacy training. To improve innovation readiness, we recommend making DMHIs easier to use and introduce, clinically useful, and safe and adapting them to fit into the existing client needs and clinical workflow. To improve organization- and system-level readiness, we recommend supporting providers and local behavioral health departments with adequate technology and training and exploring potential system transformations (eg, integrated care model). Conceptualizing DMHIs as services allows the consideration of both the innovation characteristics of DMHIs (eg, efficacy, safety, and clinical usefulness) and the ecosystem around DMHIs, such as individual and organizational characteristics (inner context), purveyors and intermediaries (bridging factor), client characteristics (outer context), as well as the fit between the innovation and implementation settings (innovation factor).

Implementation of KIDMATCH: A Clinical Decision Support Tool for Diagnosing Pediatric Patients with Multisystem Inflammatory Syndrome and Kawasaki Disease

Multisystem inflammatory syndrome in children (MIS-C) is a novel disease identified during the COVID-19 pandemic that may lead to cardiac dysfunction or death in pediatric patients. Early detection of MIS-C remains a challenge given the lack of a diagnostic test and its clinical similarities to Kawasaki disease (KD) and other acute childhood illnesses. We developed and validated the KawasakI Disease vs Multisystem InflAmmaTory syndrome in CHildren (KIDMATCH) clinical decision support tool for screening patients for MIS-C, KD, or other febrile illnesses. Here we describe the implementation and iterative refinement of KIDMATCH with provider feedback as a web calculator in the clinical workflow within Rady Children's Hospital. Our findings demonstrate KIDMATCH and its underlying artificial intelligence model have clinical utility in aiding clinicians at the time of initial evaluation within the hospital setting to distinguish patients who have MIS-C, KD, or other febrile illnesses.

Insights from Immigrant and Refugee Communities Regarding COVID-19 Needs and Opportunities: A Mixed Methods Study

Introduction

: This mixed methods study describes processes to actively engage underserved, immigrant, and refugee communities in COVID-19 vaccine efforts to co-create culturally relevant resources and dissemination.

Methods

: A survey on health care characteristics and COVID-19 attitudes was deployed between March-November 2021 followed by listening sessions conducted in December 2021. All participants were recruited through the project's Community Advisory Board. The survey, completed by 77 adults, was available in English, Spanish, Burmese, Kizigua, and Karen. Listening sessions were led by trained, multilingual, and multicultural interviewers.

Results

: Doctors/healthcare providers were rated as the most trusted messengers of COVID-19 information, however, trusted sources varied across communities. Data from three listening sessions (with a total of 14 participants) expanded survey findings with a focus on trusted sources of COVID-19 communication.

Conclusions

: This study confirmed the importance of healthcare professionals as trusted messengers for COVID-19 information among underserved communities. Qualitative data highlighted the importance of schools, ethnically-based community organizations, and friends/family with health and English literacy skills for immigrant and refugee communities. Findings suggest opportunities for collaboration with specific trusted sources for future public health dissemination efforts.

Trial registration

: not applicable.

Psychometric Assessment of the Eyberg Child Behavior Inventory in Children with Autism in Community Settings

The Eyberg Child Behavior Inventory (ECBI) is a frequently used measure to assess interfering behaviors in children and psychometric properties have recently been examined in children with autism spectrum disorder (ASD). There is a need to confirm the identified factors and examine the factor structure in a racially/ethnically diverse, community-based sample. The current study conducts a psychometric analysis of the ECBI in a sample of children with ASD receiving publicly-funded mental health services. Data were collected from 201 children with ASD ages 5-13 years (60% Hispanic/Latinx) participating in a community effectiveness trial. Confirmatory factor analysis indicated poor model fit using previously identified factors and a new four-factor solution was identified. Clinical and research implications of these findings are discussed.

Caregiver Insights and Improvement Strategies for Youth with Autism Undergoing Gastrointestinal Endoscopy

Limited guidance is available for families of youth with ASD (YASD) to prepare for invasive medical procedures. This study examined caregiver perspectives regarding YASD's gastrointestinal endoscopy (GE) experience to improve the endoscopy experience for YASD. Thirty-four caregivers of YASD, (M = 9.85 years, SD = 4.6) who underwent GE at Rady Children's Hospital, San Diego between May 2018 and July 2019 (identified via electronic health record) participated in a structured phone interview. Caregivers reported a positive experience due to the procedural team's responsiveness to the needs of YASD and appropriately answering/addressing questions/concerns. Caregivers reported a need for ASD-specific information on how to prepare for GE. Specific recommendations are discussed. Study findings offer strategies to improve the care experience of YASD undergoing GE.

A policy implementation study of earmarked taxes for mental health services: study protocol

BACKGROUND

Insufficient funding is frequently identified as a critical barrier to the implementation and sustainment of evidence-based practices (EBPs). Thus, increasing access to funding is recognized as an implementation strategy. Policies that create earmarked taxes-defined as taxes for which revenue can only be spent on specific activities-are an increasingly common mental health financing strategy that could improve the reach of EBPs. This project's specific aims are to (1) identify all jurisdictions in the USA that have implemented earmarked taxes for mental health and catalogue information about tax design; (2) characterize experiences implementing earmarked taxes among local (e.g., county, city) mental health agency leaders and other government and community organization officials and assess their perceptions of the acceptability and feasibility of different types of policy implementation strategies; and (3) develop a framework to guide effect earmarked tax designs, inform the selection of implementation strategies, and disseminate the framework to policy audiences.

METHODS

The project uses the Exploration, Preparation, Implementation, Sustainment (EPIS) framework to inform data collection about the determinants and processes of tax implementation and Leeman's typology of implementation strategies to examine the acceptability and feasibility strategies which could support earmarked tax policy implementation. A legal mapping will be conducted to achieve aim 1. To achieve aim 2, a survey will be conducted of 300 local mental health agency leaders and other government and community organization officials involved with the implementation of earmarked taxes for mental health. The survey will be followed by approximately 50 interviews with these officials. To achieve aim 3, quantitative and qualitative data will be integrated through a systematic framework development and dissemination process.

DISCUSSION

This exploratory policy implementation process study will build the evidence base for outer-context implementation determinants and strategies by focusing on policies that earmarked taxes for mental health services.

A systematic review of dissemination and implementation science capacity building programs around the globe

BACKGROUND

Research centers and programs focused on dissemination and implementation science (DIS) training, mentorship, and capacity building have proliferated in recent years. There has yet to be a comprehensive inventory of DIS capacity building program (CBP) cataloging information about activities, infrastructure, and priorities as well as opportunities for shared resources, collaboration, and growth. The purpose of this systematic review is to provide the first inventory of DIS CBPs and describe their key features and offerings.

METHODS

We defined DIS CBPs as organizations or groups with an explicit focus on building practical knowledge and skills to conduct DIS for health promotion. CBPs were included if they had at least one capacity building activity other than educational coursework or training alone. A multi-method strategy was used to identify DIS CBPs. Data about the characteristics of DIS CBPs were abstracted from each program's website. In addition, a survey instrument was developed and fielded to gather in-depth information about the structure, activities, and resources of each CBP.

RESULTS

In total, 165 DIS CBPs met our inclusion criteria and were included in the final CBP inventory. Of these, 68% are affiliated with a United States (US) institution and 32% are internationally based. There was one CBP identified in a low- and middle-income country (LMIC). Of the US-affiliated CBPs, 55% are embedded within a Clinical and Translational Science Award program. Eighty-seven CBPs (53%) responded to a follow-up survey. Of those who completed a survey, the majority used multiple DIS capacity building activities with the most popular being Training and Education (n=69, 79%) followed by Mentorship (n=58, 67%), provision of DIS Resources and Tools (n=57, 66%), Consultation (n=58, 67%), Professional Networking (n=54, 62%), Technical Assistance (n=46, 52%), and Grant Development Support (n=45, 52%).

CONCLUSIONS

To our knowledge, this is the first study to catalog DIS programs and synthesize learnings into a set of priorities and sustainment strategies to support DIS capacity building efforts. There is a need for formal certification, accessible options for learners in LMICs, opportunities for practitioners, and opportunities for mid/later stage researchers. Similarly, harmonized measures of reporting and evaluation would facilitate targeted cross-program comparison and collaboration.

Common Elements Approaches to Implementation Research and Practice: Methods and Integration with Intervention Science

We propose that common elements approaches can advance implementation research and practice and facilitate pragmatic use of intervention and implementation evidence. Common elements are practices or processes frequently shared by interventions or implementations. Traditional common elements methodologies use synthesis, distillation, and statistics to describe and evaluate the merit of common ingredients in effective interventions. Recent developments include identifying and testing common configurations of elements, processes, and context variables across the literature of effective interventions and implementations. While common elements thinking has grown popular in intervention science, it has rarely been utilized in implementation science, and specifically, combined with the intervention literature. The goals of this conceptual methodology paper are to (1) provide an overview of the common elements concept and how it may advance implementation research and usability for practice, (2) give a step-by-step guide to systematic common elements reviews that synthesizes and distills the intervention and implementation literature together, and (3) offer recommendations for advancing element-level evidence in implementation science. A narrative review of the common elements literature was conducted with attention to applications to implementation research. A six-step guide to using an advanced common elements methodology was provided. Examples of potential results are presented, along with a review of the implications for implementation research and practice. Finally, we reviewed methodological limitations in current common elements approaches, and identified steps towards realizing their potential. Common elements methodologies can (a) synthesize and distill the implementation science literature into practical applications, (b) generate evidence-informed hypotheses about key elements and determinants in implementation and intervention processes and mechanisms, and (c) promote evidence-informed precision tailoring of intervention and implementation to context. To realize this potential, common elements approaches need improved reporting of details from both successful and unsuccessful intervention and implementation research, more data availability, and more testing and investigation of causal processes and mechanisms of change from diverse theories.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-023-00077-4.

Developing Relational Coordination: A Qualitative Study of Outpatient Mental Health Teams

Previous studies have shown Relational Coordination improves team functioning in healthcare settings. The aim of this study was to examine the relational factors needed to support team functioning in outpatient mental health care teams with low staffing ratios. We interviewed interdisciplinary mental health teams that had achieved high team functioning despite low staffing ratios in U.S. Department of Veterans Affairs medical centers. We conducted qualitative interviews with 21 interdisciplinary team members across three teams within two medical centers. We used directed content analysis to code the transcripts with a priori codes based on the Relational Coordination dimensions, while also being attentive to emergent themes. We found that all seven dimensions of Relational Coordination were relevant to improved team functioning: frequent communication, timely communication, accurate communication, problem-solving communication, shared goals, shared knowledge, and mutual respect. Participants also described these dimensions as reciprocal processes that influenced each other. In conclusion, relational Coordination dimensions can play pivotal roles in improving team functioning both individually and in combination. Communication dimensions were a catalyst for developing relationship dimensions; once relationships were developed, there was a mutually reinforcing cycle between communication and relationship dimensions. Our results suggest that establishing high-functioning mental health care teams, even in low-staffed settings, requires encouraging frequent communication within teams. Moreover, attention should be given to ensuring appropriate representation of disciplines among leadership and defining roles of team members when teams are formed.

Using ethnographic approaches to document, evaluate, and facilitate virtual community-engaged implementation research

BACKGROUND

Community Advisory Boards (CABs) have been frequently used to engage diverse partners to inform research projects. Yet, evaluating the quality of engagement has not been routine. We describe a multi-method ethnographic approach documenting and assessing partner engagement in two "virtual" CABs, for which we conducted all meetings remotely.

METHODS

Two research projects for increasing equitable COVID-19 testing, vaccination, and clinical trial participation for underserved communities involved remote CAB meetings. Thirty-three partners representing 17 community groups participated in 15 sessions across the two CABs facilitated by a social change organization. We developed ethnographic documentation forms to assess multiple aspects of CAB member engagement (e.g., time spent speaking, modality used, types of interactions). Documenters were trained to observe CAB sub-groups via virtual sessions. Debriefing with the documentation team after CAB meetings supported quality assurance and process refinement. CAB members completed a brief validated survey after each meeting to assess the quality and frequency of engagement. Content and rapid thematic analysis were used to analyze documentation data. Quantitative data were summarized as frequencies and means. Qualitative and quantitative findings were triangulated.

RESULTS

A total of 4,540 interactions were identified across 15 meetings. The most frequent interaction was providing information (44%), followed by responding (37-38%). The quality and frequency of stakeholder engagement were rated favorably (average 4.7 of 5). Most CAB members (96%) reported good/excellent engagement. Specific comments included appreciation for the diversity of perspectives represented by the CAB members and suggestions for improved live interpretation. Debriefing sessions led to several methodological refinements for the documentation process and forms.

CONCLUSION

We highlight key strategies for documenting and assessing community engagement. Our methods allowed for rich ethnographic data collection that refined our work with community partners. We recommend ongoing trainings, including debriefing sessions and routinely reviewed assessment of data to strengthen meaningful community engagement.

Enhancing review criteria for dissemination and implementation science grants

BACKGROUND

The existing grant review criteria do not consider unique methods and priorities of Dissemination and Implementation Science (DIS). The ImplemeNtation and Improvement Science Proposals Evaluation CriTeria (INSPECT) scoring system includes 10 criteria based on Proctor et al.'s "ten key ingredients" and was developed to support the assessment of DIS research proposals. We describe how we adapted INSPECT and used it in combination with the NIH scoring system to evaluate pilot DIS study proposals through our DIS Center.

METHODS

We adapted INSPECT to broaden considerations for diverse DIS settings and concepts (e.g., explicitly including dissemination and implementation methods). Five PhD-level researchers with intermediate to advanced DIS knowledge were trained to conduct reviews of seven grant applications using both the INSPECT and NIH criteria. The INSPECT overall scores range from 0 to 30 (higher scores are better), and the NIH overall scores range from 1 to 9 (lower scores are better). Each grant was independently reviewed by two reviewers, then discussed in a group meeting to compare the experiences using both criteria to evaluate the proposal and to finalize scoring decisions. A follow-up survey was sent to grant reviewers to solicit further reflections on each scoring criterion.

RESULTS

Averaged across reviewers, the INSPECT overall scores ranged from 13 to 24, while the NIH overall scores ranged from 2 to 5. Reviewer reflections highlighted the unique value and utility for each scoring criterion. The NIH criteria had a broad scientific purview and were better suited to evaluate more effectiveness-focused and pre-implementation proposals not testing implementation strategies. The INSPECT criteria were easier to rate in terms of the quality of integrating DIS considerations into the proposal and to assess the potential for generalizability, real-world feasibility, and impact. Overall, reviewers noted that INSPECT was a helpful tool to guide DIS research proposal writing.

CONCLUSIONS

We confirmed complementarity in using both scoring criteria in our pilot study grant proposal review and highlighted the utility of INSPECT as a potential DIS resource for training and capacity building. Possible refinements to INSPECT include more explicit reviewer guidance on assessing pre-implementation proposals, providing reviewers with the opportunity to submit written commentary with each numerical rating, and greater clarity on rating criteria with overlapping descriptions.

Optimizing ATTAIN implementation in a federally qualified health center guided by the FRAME-IS

Implementation strategies are methods or techniques used to adopt, implement, and sustain evidence-based practices (EBPs). Implementation strategies are dynamic and may require adaptation to fit implementation contexts, especially in low-resource settings, which are most likely to serve racially and ethnically diverse patients. The framework for reporting adaptations and modifications to evidence-based implementation strategies (FRAME-IS) was used to document adaptations to implementation strategies to inform an optimization pilot of Access to Tailored Autism Integrated Care (ATTAIN; an integrated care model for children with autism and co-occurring mental health needs) in a federally qualified health center (FQHC) near the United States/Mexico border. Quantitative and qualitative data were collected from 36 primary care providers who participated in the initial ATTAIN feasibility pilot to inform adaptations. Adaptations were mapped to the FRAME-IS through an iterative template analysis to inform an optimization pilot at a FQHC 1-year from the start of the COVID-19 pandemic. Four implementation strategies (training and workflow reminders, provider/clinic champions, periodic reflections, and technical assistance) were employed during the feasibility pilot and were adapted for the optimization pilot to fit the needs of the FQHC and service delivery changes prompted by the pandemic. Findings demonstrate the utility of using the FRAME-IS to systematically inform EBP optimization in a FQHC providing care to underserved communities. Findings will inform future research studies implementing integrated mental health models in low-resourced primary care settings. Implementation outcomes and provider perceptions of ATTAIN at the FQHC are also reported. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Advancing research on teams and team effectiveness in implementation science: An application of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework

Background

Effective teams are essential to high-quality healthcare. However, teams, team-level constructs, and team effectiveness strategies are poorly delineated in implementation science theories, models, and frameworks (TMFs), hindering our understanding of how teams may influence implementation. The Exploration, Preparation, Implementation, Sustainment (EPIS) framework is a flexible and accommodating framework that can facilitate the application of team effectiveness approaches in implementation science.

Main Text

We define teams and provide an overview of key constructs in team effectiveness research. We describe ways to conceptualize different types of teams and team constructs relevant to implementation within the EPIS framework. Three case examples illustrate the application of EPIS to implementation studies involving teams. Within each study, we describe the structure of the team and how team constructs influenced implementation processes and outcomes.

Conclusions

Integrating teams and team constructs into the EPIS framework demonstrates how TMFs can be applied to advance our understanding of teams and implementation. Implementation strategies that target team effectiveness may improve implementation outcomes in team-based settings. Incorporation of teams into implementation TMFs is necessary to facilitate application of team effectiveness research in implementation science.

Understanding the Role of Support in Digital Mental Health Programs With Older Adults: Users' Perspective and Mixed Methods Study

BACKGROUND

Digital mental health interventions have the potential to increase mental health support among isolated older adults. However, the older adult population can experience several barriers to accessing and using digital health resources and may need extra support to experience its benefits.

OBJECTIVE

This paper aimed to understand what older adults experience as an important aspect of support during engagement in a digital mental health program. The program entailed 3 months of staff support to participate in digital literacy training and engage with the digital mental health platform myStrength, which offers support for a range of mental health challenges, including depression and anxiety.

METHODS

A total of 30 older adults participated in surveys and interviews to assess their experience of participating in a digital mental health program provided by county mental health services. As part of the program, participants attended 4 classes of digital literacy training, had access to the digital mental health platform myStrength for 2 months with staff support (and 10 months after the program without support), and received support from program staff during the entire 3-month program. Survey data were analyzed using descriptive statistics, and interview data were analyzed using thematic analysis.

RESULTS

A thematic analysis of the interview data revealed that participants valued ongoing support in 3 main areas: technical support to assist them in using technology, guided support to remind them to use myStrength and practice skills they had learned, and social support to enable them to connect with others through the program. Furthermore, participants reported that social connections was the most important aspect of the program and that they were mainly motivated to participate in the program because it was recommended to them by trusted others such as a community partner or because they believed it could potentially help others.

CONCLUSIONS

Our findings can be used to inform the design of future digital mental health programs for older adults who may have unique support needs in terms of dedicated technical support and ongoing guided support to use technology and social support to increase social connectedness.

Where is "policy" in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example

BACKGROUND

Implementation science aims to accelerate the public health impact of evidence-based interventions. However, implementation science has had too little focus on the role of health policy - and its inseparable politics, polity structures, and policymakers - in the implementation and sustainment of evidence-based healthcare. Policies can serve as determinants, implementation strategies, the evidence-based "thing" to be implemented, or another variable in the causal pathway to healthcare access, quality, and patient outcomes. Research describing the roles of policy in dissemination and implementation (D&I) efforts is needed to resolve persistent knowledge gaps about policymakers' evidence use, how evidence-based policies are implemented and sustained, and methods to de-implement policies that are ineffective or cause harm. Few D&I theories, models, or frameworks (TMF) explicitly guide researchers in conceptualizing where, how, and when policy should be empirically investigated. We conducted and reflected on the results of a scoping review to identify gaps of existing Exploration, Preparation, Implementation, and Sustainment (EPIS) framework-guided policy D&I studies. We argue that rather than creating new TMF, researchers should optimize existing TMF to examine policy's role in D&I. We describe six recommendations to help researchers optimize existing D&I TMF. Recommendations are applied to EPIS, as one example for advancing TMF for policy D&I.

RECOMMENDATIONS

(1) Specify dimensions of a policy's function (policy goals, type, contexts, capital exchanged). (2) Specify dimensions of a policy's form (origin, structure, dynamism, outcomes). (3) Identify and define the nonlinear phases of policy D&I across outer and inner contexts. (4) Describe the temporal roles that stakeholders play in policy D&I over time. (5) Consider policy-relevant outer and inner context adaptations. (6) Identify and describe bridging factors necessary for policy D&I success.

CONCLUSION

Researchers should use TMF to meaningfully conceptualize policy's role in D&I efforts to accelerate the public health impact of evidence-based policies or practices and de-implement ineffective and harmful policies. Applying these six recommendations to existing D&I TMF advances existing theoretical knowledge, especially EPIS application, rather than introducing new models. Using these recommendations will sensitize researchers to help them investigate the multifaceted roles policy can play within a causal pathway leading to D&I success.

Understanding implementation research collaborations from a co-creation lens: Recommendations for a path forward

Increasing calls within the field of implementation science (IS) research seek to promote active engagement of diverse and often disenfranchised stakeholder voices to increase buy-in, fidelity, outcome relevance, and sustainment of evidence-based practices (EBPs). Including such voices requires cultural humility and the integration of multiple perspectives and values among organizations, groups, and individuals. However, the IS field lacks guidance for researchers on structuring collaborative approaches to promote a co-created process (i.e., synergistic approach to goal attainment). We contend that improved operationalization of co-created implementation collaborations is critical to sparking synergy and addressing differentials based on power, privilege, knowledge, and access to resources among stakeholders. These differentials can undermine future implementation and sustainment efforts if not addressed early in the research effort. An insufficient understanding of the guiding principles of co-created implementation collaborations may limit the scientific value of evaluation processes, and researchers' ability to replicate outcomes. We propose a perspective foregrounded in the concept of co-creation to guide the structuring of implementation collaboratives through five principles. We offer three case examples informed by the Exploration, Preparation, Implementation, Sustainment (EPIS) Framework to illustrate the application of these co-creation principles. Lastly, we offer recommendations for promoting co-creation in IS research moving forward.

Implementation outcomes from a pilot of "Access to Tailored Autism Integrated Care" for children with autism and mental health needs

LAY ABSTRACT

Children with autism frequently experience co-occurring mental health needs. The "Access to Tailored Autism Integrated Care (ATTAIN)" model was co-created with caregivers, pediatric providers, and health care leaders to identify mental health needs and link to mental health care for autistic children. This article describes outcomes from a pilot study of Access to Tailored Autism Integrated Care with 36 pediatric primary care providers from seven clinics within three healthcare systems. Providers participated in an initial Access to Tailored Autism Integrated Care training and received ongoing online support over 4 months with autistic patients ages 4-16 years old. Survey and interview assessments measured provider perceptions of feasibility, acceptability, and intentions to continue using Access to Tailored Autism Integrated Care after the pilot. Providers reported that Access to Tailored Autism Integrated Care was feasible, acceptable, that the initial training was helpful in their implementation but that more specific and tailored implementation support was needed. Results show that Access to Tailored Autism Integrated Care is a promising model to support mental health screening and linkage for children with autism in primary care. Findings provide information on specific areas of the Access to Tailored Autism Integrated Care model that could be benefit from additional refinement to support more widespread use in primary care settings.

Proyecto Compadre: Using Implementation Science to Tailor Peer Navigation for Latino Men in the US-Mexico Border Region

BACKGROUND

Latino men who have sex with men (MSM) in San Diego have poor HIV testing and prevention outcomes compared with non-Latino White men. Peer navigation (PN) is a promising evidence-based intervention to reduce disparities but needs tailoring for Latino MSM.

SETTINGS

Health centers near the US-Mexico border.

METHODS

Using the Exploration, Preparation, Implementation, Sustainment Framework, we conducted mixed-methods implementation science study. In phase I, we conducted interviews with Latino men (n = 15), focus groups with staff (n = 7), and surveys with all to understand the Exploration, Preparation, Implementation, Sustainment factors associated with HIV testing and care linkage. In phase II, we conducted 31 web-based surveys with Latino men and staff to rank intervention and implementation strategies from phase I. Quantitative data were analyzed descriptively, integrated with qualitative data, and reviewed by our community-academic partnership to develop an implementation model.

RESULTS

Latino men (N = 15) were 94% Spanish speaking, 67% gay identified, 27% US born, and their suggestions were to have navigators use peer referral to address barriers such as stigma; use the Latino social network to expand reach, leverage social media for peer-led intervention, and disseminate HIV information. Staff (N = 26) were 77% Spanish speaking, 35% gay-identified, 96% trained in cultural competency, and suggested including culturally appropriate HIV educational materials in Spanish, status and identity neutral programs, administrative/supervisorial/training structure for PNs, and PN compensation and team integration. Overall, results emphasized a need for a formalized PN model centered on referrals and using existing Latino community social networks.

CONCLUSIONS

Findings can be packaged for future implementation of PN programs for Latino MSM.

Engaging Underserved Communities in COVID-19 Health Equity Implementation Research: An Analysis of Community Engagement Resource Needs and Costs

Background:

Meaningful community engagement is instrumental to effective implementation and sustainment of equitable public health interventions. Significant resources are necessary to ensure that community engagement takes place in culturally sensitive, trusted ways that optimize positive public health outcomes. However, the types and costs of resources best suited to enable meaningful community engagement in implementation research are not well-documented. This study’s objectives are (1) to describe a pragmatic method for systematically tracking and documenting resources utilized for community engagement activities, (2) report resources across phases of implementation research, and (3) provide recommendations for planning and budgeting for community engagement in health equity implementation research.

Methods:

Community engagement partners completed a tracking log of their person-hours for community engagement activities across three phases of community engagement (startup, early, maintenance) in two implementation research projects to promote equity in COVID-19 testing and vaccination for underserved communities. Both projects completed a six-session Theory of Change (i.e., a facilitated group discussion about current and desired conditions that culminated with a set of priorities for strategic change making) over 4 months with respective Community Advisory Boards (CAB) that included community organizers, promotores, federally qualified health center providers and administrators, and public health researchers. The reported person-hours that facilitated community member engagement were documented and summarized within and across project phases.

Results:

For both projects, the startup phase required the highest number of person-hours (M = 60), followed by the maintenance (M = 53) and early phase (M = 47). Within the startup phase, a total of 5 community engagement activities occurred with identifying and inviting CAB members incurring the greatest number of person-hours (M = 19). Within the early phase, a total of 11 community engagement activities occurred with coordinating and leading live interpretation (Spanish) during CAB sessions incurring the greatest number of person-hours (M = 10). The maintenance phase included 11 community engagement activities with time dedicated to written translation of CAB materials into Spanish incurring the greatest number of person-hours (M = 10).

Conclusions:

Study findings indicate that the most significant investment of resources is required in the startup period. Needed resources decreased, albeit with a greater diversity of activities, in later phases of community engagement with Spanish language translation requiring most in the later stage of the study. This study contributes to the community engagement and implementation science literature by providing a pragmatic tracking and measurement approach and recommendations for planning for and assessing costs to facilitate meaningful community engagement in public health implementation research.

Co-creating a Theory of Change to advance COVID-19 testing and vaccine uptake in underserved communities

OBJECTIVES

To describe the use of a Theory of Change to meaningfully engage community members from or support underserved communities in two National Institutes of Health-funded implementation science projects aimed at promoting equitable access to COVID-19 testing and vaccination for underserved communities.

STUDY SETTING

Both projects focused on Latino, Black, and immigrant and refugee communities in South/Central San Diego and/or individuals accessing care at a federally qualified health center near the US/Mexico border during December 2020-April 2021.

STUDY DESIGN

By using a participatory action research design, Community Advisory Boards (CABs) were established for each project with 11 and 22 members. CAB members included community organizers, promotores de salud (community health workers), clinic providers and administrators, and public health researchers. The CABs were guided through a seven-session Theory of Change process, focused on identifying necessary conditions that must exist to eliminate COVID-19 disparities along with specified actions to create those conditions and a blueprint for assessing the impact of those actions.

DATA COLLECTION

Each session lasted 2 h hosted virtually and was augmented by interactive web-based activities. There was a live interpreter who facilitated the participation of Spanish-speaking CAB members. A Theory of Change for each project was completed in approximately 4 months.

PRINCIPAL FINDINGS

Nine necessary conditions were identified related to (1) accessible and available services; (2) culturally and linguistically competent programming; (3) investment in trusted community and faith leaders; (4) social safety nets to provide ancillary services. Corresponding actions to create these conditions and measures to indicate success in creating these conditions were operationalized by the CAB.

CONCLUSIONS

While resource-intensive, a CAB-led Theory of Change process yielded a rich opportunity to engage diverse groups that typically are not invited to inform these processes.

Mental health screening in pediatric primary care for children with autism

LAY ABSTRACT

Children with autism experience high rates of co-occurring mental health conditions like challenging behaviors and anxiety. However, these co-occurring mental health needs are often not identified when they first become problematic. Pediatricians and their care staff are in a good position to identify mental health needs early and support families to connect to needed services. This study describes a project focused on mental health screening for children with autism in pediatric primary care clinics. Over half of eligible patients were screened using the Pediatric Symptom Checklist-17. Many children with autism had clinically elevated scores, suggesting the need for mental health assessment or services. In particular, children with positive screens had clinical elevations on the challenging behavior and attention subscales of the Pediatric Symptom Checklist-17. This finding is consistent with typical trends in co-occurring challenging behavior presentations in children with autism. Mental health screening in primary care is feasible and offers a promising opportunity to identify co-occurring mental health needs for children with autism early. Screening rates varied between clinics, suggesting tailored to improve routine screening in pediatric primary care for children with autism.

Pre-Implementation Organizational Environment Associated with Pediatric Integrated Care Readiness in Primary Care

Objective

To characterize pre-implementation organizational factors associated with Access To Tailored Autism Integrated Care (ATTAIN), an integrated care model for children with autism and identified or suspected co-occurring mental health needs.

Methods

Pediatric primary care providers (n=36) completed surveys as part of a pilot study testing ATTAIN feasibility. Measures assessed: background characteristics; implementation climate; organizational readiness; evidence-based practice (EBP) attitudes; knowledge, confidence and comfort caring for children with autism.

Results

Compared to providers from a network of primary care practices and an integrated healthcare system, providers from a Federally Qualified Health Center (FQHC) reported less positive perceptions of climate supportive of EBP implementation and, specifically, selection for openness and rewards for EBPs. Implementation climate was associated with autism knowledge, comfort with mental health referral and follow up, and organizational change efficacy.

Conclusions

Findings advance understanding of pre-implementation organizational context factors important to assess for EBP implementation in diverse pediatric care settings.

STOP COVID-19 CA: Community engagement to address the disparate impacts of the COVID-19 pandemic in California

Objective

To describe the early activities and lessons of the Share, Trust, Organize, Partner COVID-19 California Alliance (STOP COVID-19 CA), the California awardee of the NIH-funded multi-state Community Engagement Alliance (CEAL) against COVID-19. The Alliance was established to ensure equity in Coronavirus-19 disease (COVID-19) research, clinical practice, and public health for communities most impacted by the COVID-19 pandemic.

Study setting

The STOP COVID-19 CA Alliance network of 11 universities and affiliated partner community-based organizations (CBOs) across California.

Study design

Mixed methods evaluation consisting of an analysis of activity (August 2020 to December 2021) detailed in reports submitted by community-academic teams and a survey (August 2021) of academic investigators and affiliated community-based organization (CBO) partners.

Data collection

We summarized activities from the 11 community-academic teams' progress reports and described results from an online survey of academic investigators and CBO partners in the California Alliance.

Principal findings

A review of progress reports (n = 256) showed that teams fielded surveys to 11,000 Californians, conducted 133 focus groups, partnered with 29 vaccine/therapeutics clinical trials, and led more than 300 town halls and vaccine events that reached Californians from communities disproportionately impacted by COVID-19. Survey responses from academic investigators and CBO partners emphasized the importance of learning from the successes and challenges of the California Alliance teams' COVID-19 initiatives. Both academic and CBO respondents highlighted the need for streamlined federal and institutional administrative policies, and fiscal practices to promote more effective and timely operations of teams in their efforts to address the numerous underlying health and social disparities that predispose their communities to higher rates of, and poor outcomes from, COVID-19.

Conclusions

STOP COVID-19 CA represents a new and potentially sustainable statewide community engagement model for addressing health disparities in multiethnic/multicultural and geographically dispersed communities.

Using a participatory method to test a strategy supporting the implementation of a state policy on screening children for adverse childhood experiences (ACEs) in a Federally Qualified Health Center system: a stepped-wedge cluster randomized trial

BACKGROUND

Adverse childhood experiences (ACEs) are potentially traumatic events occurring before age 18, such as maltreatment or exposure to violence. ACE screening is increasingly recommended to prevent and address physical and mental health conditions associated with ACEs. To promote ACE screening uptake, the state of California issued the "ACEs Aware" policy that provides Medicaid reimbursement for ACE screening annually for child primary care visits. However, policy directives alone often do not translate into effective screening efforts and greater access to care. Few rigorous studies have developed and tested implementation strategies for ACE pediatric screening policies. This study will fill this gap by testing a multifaceted implementation strategy in partnership with a Federally Qualified Health Center (FQHC) system serving low-income families in Southern California to support the ACE Aware policy.

METHODS

We will use Implementation Mapping, with study process and consideration of determinants and mechanisms guided by the EPIS framework, to co-create and refine an implementation strategy. The proposed strategy is comprised of online training videos, a customized algorithm and use of technology to improve workflow efficiency, implementation training to internal FQHC personnel, clinic support and coaching, and written implementation protocols. A hybrid type 2, stepped-wedge cluster randomized trial design with five primary care clinics will test whether a multifaceted implementation strategy improves (a) fidelity to the ACE screening protocol, (b) reach defined as the proportion of eligible children screened for ACEs, and (c) the impact of the ACE policy on child-level mental health referrals and symptom outcomes. The study will use mixed methods with data to include electronic health records, surveys, and interviews with clinic personnel and caregivers.

DISCUSSION

This study is designed to increase the capacity of FQHCs' inner context to successfully implement an outer context-initiated ACE policy designed to benefit pediatric patients. It capitalizes on a rare opportunity to use a co-creation approach to develop, adapt, refine, and pilot test an implementation strategy to maximize the impact of a new state-wide policy intended to improve ACE assessment and subsequent care to improve child health, particularly those from underserved communities.

TRIAL REGISTRATION

Trial # NCT04916587  registered at ClinicalTrials.gov on June 4, 2021.

Moving beyond Aim Three: a need for a transdisciplinary approach to build capacity for economic evaluations in implementation science

BACKGROUND

Understanding the costs and economic benefits of implementation has been identified by policymakers and researchers as critical to increase the uptake and sustainment of evidence-based practices, but this topic remains relatively understudied. Conducting team science with health economists has been proposed as a solution to increase economic evaluation in implementation science; however, these recommendations ignore the differences in goals and perspectives in these two fields. Our recent qualitative research identified that implementation researchers predominantly approach health economists to examine costs, whereas the majority of health economists expressed limited interest in conducting economic evaluations and a desire to be more integrated within implementation science initiatives. These interviews pointed to challenges in establishing fruitful partnerships when health economists are relegated to the "Third Aim" (i.e., lowest-priority research objective) in implementation science projects by their research partners.

DISCUSSION

In this debate paper, we argue that implementation researchers and health economists need to focus on team science research principles to expand capacity to address pressing research questions that cut across the two fields. Specifically, we use the four-phase model of transdisciplinary research to outline the goals and processes needed to build capacity in this area (Hall et al., Transl Behav Med 2:415-30, 2012). The first phase focuses on the development of transdisciplinary research teams, including identifying appropriate partners (e.g., considering policy or public health researchers in addition to health economists) and building trust. The conceptual phase focuses on strategies to consider when developing joint research questions and methodology across fields. In the implementation phase, we outline the effective processes for conducting research projects, such as team learning. Finally, in the translation phase, we highlight how a transdisciplinary approach between health economists and implementation researchers can impact real-world practice and policy. The importance of investigating the economic impact of evidence-based practice implementation is widely recognized, but efforts have been limited due to the challenges in conducting team science across disciplines. Training in team science can help advance transdisciplinary efforts, which has the potential to increase the rigor and impact of economic evaluations in implementation science while expanding the roles taken by health economists.

Curating the Digital Mental Health Landscape With a Guide to Behavioral Health Apps: A County-Driven Resource

With more than 10,000 mental health apps available, consumers and clinicians who want to adopt such tools can be overwhelmed by the multitude of options and lack of clear evaluative standards. Despite the increasing prevalence of curated lists, or app guides, challenges remain. Organizations providing mental health services to consumers have an opportunity to address these challenges by producing guides that meet relevant standards of quality and are tailored to local needs. This column summarizes an example of the collaborative process of app guide development in a publicly funded mental health service context and highlights opportunities and barriers identified through the process.

Mental Health Services for Autistic Individuals Across the Lifespan: Recent Advances and Current Gaps

PURPOSE OF REVIEW

This synthesis of recent mental health services research with autistic individuals presents significant advances, current gaps, and recommendations for improving mental healthcare for this population.

RECENT FINDINGS

Recent advances include improved understanding of co-occurring mental health conditions among autistic individuals, a growing evidence base for interventions to address them, the development and implementation of new service models to support mental health for this population, and a substantial increase in mental health services and implementation research focused on autism. Ongoing challenges include a lack of mental health interventions designed for community implementation with autistic individuals, limited workforce capacity, complex and disconnected service systems, and racial, ethnic, and socioeconomic disparities in accessibility and quality of mental health services. Despite the advances in our understanding of mental health needs and mental health services for autistic individuals, several critical gaps remain. We encourage future efforts to develop and test interventions that can be used in community settings, train and incentivize the workforce to provide them, realign policies and funding with best practice, and embrace an equity-focused approach to autism research and care.

Advancing discussion of ethics in mixed methods health services research

BACKGROUND

To describe the ethical issues and experiences of scientists conducting mixed methods health services research and to advance empirical and conceptual discussion on ethical integrity in mixed methods health research.

METHODS

The study was conducted with 64 scholars, faculty and consultants from the NIH-funded Mixed Methods Research Training Program (MMRTP) for the Health Sciences. This was a cross-sectional study. Survey results were analyzed using descriptive statistics to characterize responses and open coding to summarize strategies about eight ethical mixed methods research issues. Respondents completed an online survey to elicit experiences related to eight ethical issues (informed consent, confidentiality, data management, burden, safety, equitable recruitment, communication, and dissemination) and strategies for addressing them.

RESULTS

Only about one-third of respondents thought their research ethics training helped them plan, conduct, or report mixed methods research. The most frequently occurring ethical issues were participant burden, dissemination and equitable recruitment (> 70% endorsement). Despite occurring frequently, < 50% of respondents rated each ethical issue as challenging. The most challenging ethical issues were related to managing participant burden, communication, and dissemination. Strategies reported to address ethical issues were largely not specific or unique to mixed methods with the exception of strategies to mitigate participant burden and, to a lesser degree, to facilitate equitable recruitment and promote dissemination of project results.

CONCLUSIONS

Mixed methods health researchers reported encountering ethical issues often yet varying levels of difficulty and effectiveness in the strategies used to mitigate ethical issues. This study highlights some of the unique challenges faced by mixed methods researchers to plan for and appropriately respond to arising ethical issues such as managing participant burden and confidentiality across data sources and utilizing effective communication and dissemination strategies particularly when working with a multidisciplinary research team. As one of the first empirical studies to examine mixed methods research ethics, our findings highlight the need for greater attention to ethics in health services mixed methods research and training.

Rise in Use of Digital Mental Health Tools and Technologies in the United States During the COVID-19 Pandemic: Survey Study

BACKGROUND

Accompanying the rising rates of reported mental distress during the COVID-19 pandemic has been a reported increase in the use of digital technologies to manage health generally, and mental health more specifically.

OBJECTIVE

The objective of this study was to systematically examine whether there was a COVID-19 pandemic-related increase in the self-reported use of digital mental health tools and other technologies to manage mental health.

METHODS

We analyzed results from a survey of 5907 individuals in the United States using Amazon Mechanical Turk (MTurk); the survey was administered during 4 week-long periods in 2020 and survey respondents were from all 50 states and Washington DC. The first set of analyses employed two different logistic regression models to estimate the likelihood of having symptoms indicative of clinical depression and anxiety, respectively, as a function of the rate of COVID-19 cases per 10 people and survey time point. The second set employed seven different logistic regression models to estimate the likelihood of using seven different types of digital mental health tools and other technologies to manage one's mental health, as a function of symptoms indicative of clinical depression and anxiety, rate of COVID-19 cases per 10 people, and survey time point. These models also examined potential interactions between symptoms of clinical depression and anxiety, respectively, and rate of COVID-19 cases. All models controlled for respondent sociodemographic characteristics and state fixed effects.

RESULTS

Higher COVID-19 case rates were associated with a significantly greater likelihood of reporting symptoms of depression (odds ratio [OR] 2.06, 95% CI 1.27-3.35), but not anxiety (OR 1.21, 95% CI 0.77-1.88). Survey time point, a proxy for time, was associated with a greater likelihood of reporting clinically meaningful symptoms of depression and anxiety (OR 1.19, 95% CI 1.12-1.27 and OR 1.12, 95% CI 1.05-1.19, respectively). Reported symptoms of depression and anxiety were associated with a greater likelihood of using each type of technology. Higher COVID-19 case rates were associated with a significantly greater likelihood of using mental health forums, websites, or apps (OR 2.70, 95% CI 1.49-4.88), and other health forums, websites, or apps (OR 2.60, 95% CI 1.55-4.34). Time was associated with increased odds of reported use of mental health forums, websites, or apps (OR 1.20, 95% CI 1.11-1.30), phone-based or text-based crisis lines (OR 1.20, 95% CI 1.10-1.31), and online, computer, or console gaming/video gaming (OR 1.12, 95% CI 1.05-1.19). Interactions between COVID-19 case rate and mental health symptoms were not significantly associated with any of the technology types.

CONCLUSIONS

Findings suggested increased use of digital mental health tools and other technologies over time during the early stages of the COVID-19 pandemic. As such, additional effort is urgently needed to consider the quality of these products, either by ensuring users have access to evidence-based and evidence-informed technologies and/or by providing them with the skills to make informed decisions around their potential efficacy.

Forms and functions of bridging factors: specifying the dynamic links between outer and inner contexts during implementation and sustainment

BACKGROUND

Bridging factors are relational ties, formal arrangements, and processes that connect outer system and inner organizational contexts. They may be critical drivers of evidence-based practice (EBP) implementation and sustainment. Yet, the complex interplay between outer and inner contexts is often not considered. Bridging factors were recently defined in the updated Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Further identification and specification of this construct will advance implementation models, measures, and methods. Our goal is to advance bridging factor research by identifying relevant dimensions and exemplifying these dimensions through illustrative case studies.

METHODS

We used a multiple case study design. Each case (n = 10) represented different contexts, EBPs, and bridging factor types. Inclusion criteria were the presence of clearly distinguishable outer and inner contexts, identifiable bridging factor, sufficient information to describe how the bridging factor affected implementation, and variation from other cases. We used an iterative qualitative inquiry process to develop and refine a list of dimensions. Case data were entered into a matrix. Dimensions comprised the rows and case details comprised the columns. After a review of all cases, we collectively considered and independently coded each dimension as function or form.

RESULTS

We drew upon the concepts of functions and forms, a distinction originally proposed in the complex health intervention literature. Function dimensions help define the bridging factor and illustrate its purpose as it relates to EBP implementation. Form dimensions describe the specific structures and activities that illustrate why and how the bridging factor has been customized to a local implementation experience. Function dimensions can help researchers and practitioners identify the presence and purpose of bridging factors, whereas form dimensions can help us understand how the bridging factor may be designed or modified to support EBP implementation in a specific context. We propose five function and three form bridging factor dimensions.

CONCLUSIONS

Bridging factors are described in many implementation models and studies, but without explicit reference or investigation. Bridging factors are an understudied and critical construct that requires further attention to facilitate implementation research and practice. We present specific recommendations for a bridging factors research agenda.

Strengthening capacity for implementation of evidence-based practices for autism in schools: The roles of implementation climate, school leadership, and fidelity

Understanding the factors that influence the use of evidence-based practices (EBPs) for autism spectrum disorder (ASD) in schools is critical to the selection of effective implementation strategies to support their sustained use. The current study has 2 aims: (a) evaluate the association between school leadership profiles (undifferentiated and optimal) and fidelity of EBP implementation and (b) examine the role of implementation climate as a mediator in this association. Participants included 56 principals, 90 special education teachers, and 133 classroom staff from 66 elementary schools. Participants completed the Multifactor Leadership Questionnaire and Implementation Climate Scale. Teachers and staff reported on intensity (frequency) of EBP delivery, and the research team rated the accuracy (adherence) to the components of the EBP. Schools were required to implement at least 1 of 3 EBPs for ASD (discrete trial training, pivotal response training, or visual supports). Using structural equation modeling, we found that schools with optimal leadership had higher observed ratings of teacher and staff fidelity for pivotal response training accuracy (p < .05), but not for discrete trial training or visual supports. However, this association became nonsignificant with the introduction of implementation climate into the models. Optimal leadership profiles were linked to more positive teacher/staff-reported implementation climate, compared with undifferentiated profiles (p < .01), but found no association between implementation climate and fidelity. Overall, the results of this study indicate that the role of principal leadership in EBP implementation is complex, which has implications for fostering a conducive organizational implementation context in schools. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Implementation strategies for digital mental health interventions in health care settings

U.S. health care systems are tasked with alleviating the burden of mental health, but are frequently underprepared and lack workforce and resource capacity to deliver services to all in need. Digital mental health interventions (DMHIs) can increase access to evidence-based mental health care. However, DMHIs commonly do not fit into the day-to-day activities of the people who engage with them, resulting in a research-to-practice gap for DMHI implementation. For health care settings, differences between digital and traditional mental health services make alignment and integration challenging. Specialized attention is needed to improve the implementation of DMHIs in health care settings so that these services yield high uptake, engagement, and sustainment. The purpose of this article is to enhance efforts to integrate DMHIs in health care settings by proposing implementation strategies, selected and operationalized based on the discrete strategies established in the Expert Recommendations for Implementing Change project, that align to DMHI-specific barriers in these settings. Guidance is offered in how these strategies can be applied to DMHI implementation across four phases commonly distinguished in implementation science using the Exploration, Preparation, Implementation, Sustainment Framework. Next steps to advance research in this area and improve the research-to-practice gap for implementing DMHIs are recommended. Applying implementation strategies to DMHI implementation will enable psychologists to systematically evaluate this process, which can yield an enhanced understanding of the factors that facilitate implementation success and improve the translation of DMHIs from controlled trials to real-world settings. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Advancing the pragmatic measurement of sustainment: a narrative review of measures

BACKGROUND

Sustainment, an outcome indicating an intervention continues to be implemented over time, has been comparatively less studied than other phases of the implementation process. This may be because of methodological difficulties, funding cycles, and minimal attention to theories and measurement of sustainment. This review synthesizes the literature on sustainment measures, evaluates the qualities of each measure, and highlights the strengths and gaps in existing sustainment measures. Results of the review will inform recommendations for the development of a pragmatic, valid, and reliable measure of sustainment.

METHODS

A narrative review of published sustainment outcome and sustainability measures (i.e., factors that influence sustainment) was conducted, including appraising measures in the Society of Implementation Research Collaboration (SIRC) instrument review project (IRP) and the Dissemination and Implementation Grid-Enabled Measures database initiative (GEM-D&I). The narrative review used a snowballing strategy by searching the reference sections of literature reviews and definitions of sustainability and sustainment. Measures used frequently and judged to be comprehensive and/or validated by a team of implementation scientists were extracted for analysis.

RESULTS

Eleven measures were evaluated. Three of the included measures were found in the SIRC-IRP, three in the GEM-D&I database, (one measure was in both databases) and six were identified in our additional searches. Thirteen constructs relating to sustainment were coded from selected measures. Measures covered a range of determinants for sustainment (i.e., construct of sustainability) as well as constructs of sustainment as an outcome. Strengths of the measures included, development by expert panels knowledgeable about particular interventions, fields or contexts, and utility in specific scenarios. A number of limitations were found in the measures analyzed including inadequate assessment of psychometric characteristics, being overly intervention or context specific, being lengthy and/or complex, and focusing on outer context factors.

CONCLUSION

There is a lack of pragmatic and psychometrically sound measures of sustainment that can be completed by implementation stakeholders within inner context settings (e.g., frontline providers, supervisors).

Ten recommendations for using implementation frameworks in research and practice

Background

Recent reviews of the use and application of implementation frameworks in implementation efforts highlight the limited use of frameworks, despite the value in doing so. As such, this article aims to provide recommendations to enhance the application of implementation frameworks, for implementation researchers, intermediaries, and practitioners.

Discussion

Ideally, an implementation framework, or multiple frameworks should be used prior to and throughout an implementation effort. This includes both in implementation science research studies and in real-world implementation projects. To guide this application, outlined are ten recommendations for using implementation frameworks across the implementation process. The recommendations have been written in the rough chronological order of an implementation effort; however, we understand these may vary depending on the project or context: (1) select a suitable framework(s), (2) establish and maintain community stakeholder engagement and partnerships, (3) define issue and develop research or evaluation questions and hypotheses, (4) develop an implementation mechanistic process model or logic model, (5) select research and evaluation methods (6) determine implementation factors/determinants, (7) select and tailor, or develop, implementation strategy(s), (8) specify implementation outcomes and evaluate implementation, (9) use a framework(s) at micro level to conduct and tailor implementation, and (10) write the proposal and report. Ideally, a framework(s) would be applied to each of the recommendations. For this article, we begin by discussing each recommendation within the context of frameworks broadly, followed by specific examples using the Exploration, Preparation, Implementation, Sustainment (EPIS) framework.

Summary

The use of conceptual and theoretical frameworks provides a foundation from which generalizable implementation knowledge can be advanced. On the contrary, superficial use of frameworks hinders being able to use, learn from, and work sequentially to progress the field. Following the provided ten recommendations, we hope to assist researchers, intermediaries, and practitioners to improve the use of implementation science frameworks.

Defining and Predicting High Cost Utilization in Children's Outpatient Mental Health Services

Little is known about high-cost service users in the context of youth outpatient mental health, despite the fact that they account for a large proportion of overall mental healthcare expenditures. A nuanced understanding of these users is critical to develop and implement tailored services, as well as to inform relevant policies. This study aims to characterize high-cost service users by examining demographic factors, diagnoses, and service type use. Administrative service use data were extracted from a large County Department of Behavioral Health Services database. Latent profile analyses suggest a four-profile solution primarily distinguished by youth age and diagnostic complexity. Study findings have implications for defining high-cost service users and key targets for efforts aiming to improve outcomes for these youth.

Service use by youth with autism within a system-driven implementation of evidence-based practices in children's mental health services

Public mental health systems play an important role in caring for youth with autism spectrum disorder. Like other dually diagnosed populations, youth with autism spectrum disorder may receive services in the context of evidence-based practice implementation efforts within public mental health systems. Little is known about service use patterns within the context of system-driven implementations efforts for this population. This case-control study examined mental health service patterns of 2537 youth with autism spectrum disorder compared to 2537 matched peers receiving care in the Los Angeles County Department of Mental Health, the largest public mental health department in the United States, within the context of a system-driven implementation of multiple evidence-based practices. Although not the primary target of this implementation effort, youth with autism spectrum disorder were served when they met criteria for the services based on their presenting mental health symptoms. Comparative analyses using administrative claims data were conducted to examine differences in mental health utilization patterns and clinical characteristics. Findings revealed significant differences in the volume and duration of mental health services as well as differences in the service type and evidence-based practice delivered between youth with and without autism spectrum disorder. Results provide direction targeting implementation efforts for youth with autism spectrum disorder within a public mental health system care reform.