Sustained Implementation Support Scale: Validation of a Measure of Program Characteristics and Workplace Functioning for Sustained Program Implementation

Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C): development and psychometric evaluation of a measure of sustainability determinants in the early childhood education and care setting

BACKGROUND

There is a need for valid and reliable measures of determinants of sustainability of public health interventions in early childhood education and care (ECEC) settings. This study aimed to develop and evaluate the psychometric and pragmatic properties of such a measure - the Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C).

METHODS

We undertook a two-phase process guided by the COnsensus-based Standards for the selection of health status Measurement INstruments checklist (COSMIN) and Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Phase 1 involved measure development; i.e., determining items and scales through an iterative process and assessment of face and content validity. Phase 2 involved the evaluation of psychometric and pragmatic properties. The 29-item measure completed by service executives (directors and nominated supervisors) was embedded in a larger survey from a national sample of Australian ECEC services assessing their implementation of nutrition and physical activity programs. Structural validity, concurrent validity, known groups validity, internal consistency, floor and ceiling effects, norms, and pragmatic qualities of the measure were assessed according to the PAPERS criteria.

RESULTS

The final measure contained 26 items, with respondents reporting how strongly they agreed or disagreed on a five-point Likert scale. Phase 1 assessments confirmed the relevance, and face and content validity of the scale. In Phase 2, we obtained 482 completed surveys, of which 84% (n = 405) completed the entire measure across 405 ECEC settings (one executive per service). Three of the four fit indices for the confirmatory factor analysis met the pre-specified criteria (SRMR = 0.056, CFI = 0.993, RMSEA = 0.067) indicating 'good' structural validity. The IMPRESS-C illustrated: 'good' internal consistency, with Cronbach's alpha values from 0.53 to 0.92; 'emerging' concurrent validity; 'poor' known groups validity; 'good' norms; and 'good' overall pragmatic qualities (cost, readability, length, and assessor burden).

CONCLUSIONS

The IMPRESS-C possesses strong psychometric and pragmatic qualities for assessing service executive-level perceptions of determinants influencing sustainment of public health interventions within ECEC settings. To achieve a full range of perspectives in this setting, future work should be directed to also develop and test measures of sustainability determinants at the implementer level (e.g., among individual educators and staff).

Predicting the Sustained Implementation of an Evidence-Based Parenting Program: A Structural Equation Modelling Approach

PURPOSE

Sustained implementation is required for evidence-based parenting programs to promote children and their families' wellbeing at the societal level. Previous literature has examined the role of a range of different factors in enhancing sustainability. However, the inter-relationship between, and the relative importance of different factors remain largely unknown. The overall aim of this study is to identify predictors of sustained program use, the relative importance of factors, and potential mediation pathways.

METHODS

We surveyed 1202 practitioners who were trained in at least one variant of the Triple P-Positive Parenting Program, at least one and half years before data collection. The present data were linked with data collected during professional training. We first examined the independent effect of each factor on sustained program use, then, developed and evaluated a structural equation model of sustained program use.

RESULTS

The structural equation model explained a considerable amount of variance in sustained program use, with seven positive predictors and one negative predictor identified. Organisational support was identified as a key facilitator, which was not only positively linked with other facilitators but also had an independent positive effect. Perceived usefulness of the program was the most important practitioner-level facilitator, which might be contributed by both research-based evidence and practice-based evidence. Practitioners' self-regulation in program delivery impacted sustained use by influencing other factors such as perceived usefulness of the program.

CONCLUSION

The findings provided insight into factors influencing the sustainability of evidence-based parenting programs which could be used to inform future implementation practice.

Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review

BACKGROUND

Sustainability is concerned with the long-term delivery and subsequent benefits of evidence-based interventions. To further this field, we require a strong understanding and thus measurement of sustainability and what impacts sustainability (i.e., sustainability determinants). This systematic review aimed to evaluate the quality and empirical application of measures of sustainability and sustainability determinants for use in clinical, public health, and community settings.

METHODS

Seven electronic databases, reference lists of relevant reviews, online repositories of implementation measures, and the grey literature were searched. Publications were included if they reported on the development, psychometric evaluation, or empirical use of a multi-item, quantitative measure of sustainability, or sustainability determinants. Eligibility was not restricted by language or date. Eligibility screening and data extraction were conducted independently by two members of the research team. Content coverage of each measure was assessed by mapping measure items to relevant constructs of sustainability and sustainability determinants. The pragmatic and psychometric properties of included measures was assessed using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The empirical use of each measure was descriptively analyzed.

RESULTS

A total of 32,782 articles were screened from the database search, of which 37 were eligible. An additional 186 publications were identified from the grey literature search. The 223 included articles represented 28 individual measures, of which two assessed sustainability as an outcome, 25 covered sustainability determinants and one explicitly assessed both. The psychometric and pragmatic quality was variable, with PAPERS scores ranging from 14 to 35, out of a possible 56 points. The Provider Report of Sustainment Scale had the highest PAPERS score and measured sustainability as an outcome. The School-wide Universal Behaviour Sustainability Index-School Teams had the highest PAPERS score (score=29) of the measure of sustainability determinants.

CONCLUSIONS

This review can be used to guide selection of the most psychometrically robust, pragmatic, and relevant measure of sustainability and sustainability determinants. It also highlights that future research is needed to improve the psychometric and pragmatic quality of current measures in this field.

TRIAL REGISTRATION

This review was prospectively registered with Research Registry (reviewregistry1097), March 2021.

Understanding How Newly Implemented Rehabilitation Best Practices Are Sustained: A Realist Review

OBJECTIVE

We conducted a realist review to understand how (mechanism) and in what circumstances (context) evidence-based practices are sustained in rehabilitation (outcome).

DATA SOURCES

MEDLINE, Embase, reference lists, and targeted websites.

STUDY SELECTION

Two independent reviewers calibrated study selection; then 1 reviewer screened all titles and abstracts, while the second reviewer screened a random 20%. We repeated this process for full texts. We included 115 documents representing 61 implementation projects (8.9% of identified documents). Included documents described implementation projects in which physical therapists, occupational therapists, and/or speech-language pathologists were the target users of an evidence-based practice.

DATA EXTRACTION

Two reviewers repeated the independent process described in study selection to extract basic study and sustainability characteristics as well as context, mechanism, outcome, and strategy text.

DATA SYNTHESIS

Using basic numerical analyses, we found that only 54% of evidence-based practices in rehabilitation are sustained. Furthermore, while authors who reported sustainability planning sustained the practice 94% of the time, sustainability planning in rehabilitation is rare (only reported 26% of the time). Extracted text was synthesized using the realist technique of inductive and deductive retroduction in which context, mechanism, outcome, and strategy text are combined into narrative explanations of how sustainability works. To inform these explanations, we applied normalization process theory and the theory of planned behavior. Collectively, the 52 identified narratives provide evidence for 3 patterns: (1) implementation and sustainability phases are interconnected, (2) continued use of the evidence-based practice can be interpreted as the ultimate sustainability outcome, and (3) intermediate sustainability outcomes (ie, fit/alignment, financial support, benefits, expertise) can become contextual features influencing other sustainability outcomes.

CONCLUSIONS

Implementation teams can use the narrative explanations generated in this review to optimize sustainability planning. This can sustain practice changes and improve quality of care and patient outcomes. Future research should seek to iteratively refine the proposed narrative explanations.

Protocol for a hybrid type 3 cluster randomized trial of a technical assistance system supporting coalitions and evidence-based drug prevention programs

BACKGROUND

Over 5000 community anti-drug coalitions operating in the USA serve as a cornerstone of federal drug prevention. These coalitions, however, have demonstrated effectiveness in preventing substance use only when they use technical assistance (TA) and implement evidence-based programs (EBPs). The absence of TA and EBP implementation by coalitions is a key research-to-practice gap. The Coalition Check-Up TA system is designed to fill this gap by supporting community coalition implementation of EBPs. Existing TA models for evidence-based coalition approaches are resource intensive and coalition model specific. The Coalition Check-Up is a lower cost strategy that works with a variety of types of coalitions to support sustainable implementation of EBPs. This study protocol describes a hybrid type 3 effectiveness-implementation trial applying Wandersman's Interactive Systems Framework to test the effects of the Coalition Check-Up on coalition EBP implementation capacity and outcomes. The Interactive Systems Framework outlines how the prevention support system-especially TA-bolsters EBP dissemination and implementation.

METHODS

Using a cluster randomized controlled design, this trial will test the overall effectiveness of the Coalition Check-Up, including how it contributes to EBP implementation and prevention of youth substance use. The first aim is to estimate the impact of the Coalition Check-Up on coalitions' capacity to do their work. We will recruit 68 anti-drug coalitions for random assignment to the Coalition Check-Up or "TA as usual" condition. We will evaluate whether the Coalition Check-Up improves coalition capacity using measures of coalition member responses about team processes, coalition network composition, and collaborative structure. Our second aim is to estimate the impact of the Coalition Check-Up on implementation of EBPs, and our third aim is to estimate the impact of the Coalition Check-Up on youth substance use.

DISCUSSION

This project will clarify how the Coalition Check-Up, a scalable approach to TA due to its low cost, affects coalition capacity to support EBP implementation. Analyses also provide insight into causal pathways from the prevention support system to the prevention delivery system outlined by the Interactive Systems Framework. Results will build the evidence-base for how to support community coalitions' sustainable implementation of evidence-based prevention programs and policies.

TRIAL REGISTRATION

Clinicaltrials.gov registration number NCT04592120 . Registered on October 19, 2020.

Making change last? Exploring the value of sustainability approaches in healthcare: a scoping review

Background

Numerous models, tools and frameworks have been produced to improve the sustainability of evidence-based interventions. Due to the vast number available, choosing the most appropriate one is increasingly difficult for researchers and practitioners. To understand the value of such approaches, evidence warranting their use is needed. However, there is limited understanding of how sustainability approaches have been used and how they have impacted research or practice. This review aims to consolidate evidence on the application and impact of sustainability approaches in healthcare settings.

Methods

A systematic scoping review was designed to search for peer-reviewed publications detailing the use of sustainability approaches in practice. A 5-stage framework for scoping reviews directed the search strategy, and quality assessment was performed using the Mixed Method Appraisal Tool. Searches were performed through electronic citation tracking and snowballing of references. Articles were obtained through Web of Science, PubMed and Google Scholar. Six outcome variables for sustainability were explored to ascertain impact of approaches.

Results

This review includes 68 articles demonstrating the application of sustainability approaches in practice. Results show an increase in the use of sustainability approaches in peer-reviewed studies. Approaches have been applied across a range of healthcare settings, including primary, secondary, tertiary and community healthcare. Approaches are used for five main purposes, namely analysis, evaluation, guidance, assessment and planning. Results outline benefits (e.g. improved conceptualisation of sustainability constructs and improved ability to interpret sustainability data) and challenges (e.g. issues with approach constructs and difficulty in application) associated with using a sustainability approach in practice. Few articles (14/68) reported the sustainability outcome variables explored; therefore, the impact of approaches on sustainability remains unclear. Additional sustainability outcome variables reported in retrieved articles are discussed.

Conclusions

This review provides practitioners and researchers with a consolidated evidence base on sustainability approaches. Findings highlight the remaining gaps in the literature and emphasise the need for improved rigour and reporting of sustainability approaches in research studies. To guide future assessment and study of sustainability in healthcare settings an updated list of sustainability outcome variables is proposed.

Trial Registration

This review was registered on the PROSPERO database CRD 42016040081 in June 2016.

Measurement of sustainment of prevention programs and initiatives: the sustainment measurement system scale

Background

Enhancing the sustainability of evidence-based prevention programs for mental and behavioral health requires tools for measuring both sustainability determinants and sustainment outcomes. The aim of this study was to develop the Sustainment Measurement System Scale (SMSS) and to assess its reliability and construct validity for measuring both determinants and outcomes of efforts to sustain prevention programs and initiatives.

Methods

A 42-item scale comprised of items identified from qualitative data collected from 45 representatives of 10 programs and 8 SAMHSA program officers was administered to 186 representatives of 145 programs funded by 7 SAMHSA prevention grant initiatives. Cronbach’s alphas were used to determine inter-item reliability. Convergent validity was assessed by comparisons of a global measure of sustainment with current SAMHSA-funding status and continued operation in the same form. Discriminant validity was assessed by comparisons of sustainability determinants with whether or not the program had undergone adaptations.

Results

Confirmatory factor analysis provided support for a 35-item model fit to the data. Cronbach’s alpha was .84 for the sustainment outcome construct and ranged from .70 to .93 for the sustainability determinant constructs. All of the determinant constructs were significantly associated with sustainment outcome individual and global measures for the entire sample (p < 0.01 to 0.001) and for community-based programs and programs with a substance abuse focus (p < 0.05 to 0.001). Convergent validity was supported by significant associations between the global sustainment measure and current SAMHSA funding status and continued operation in the same form (p < 0.001). Four of the sustainability determinant constructs (responsive to community needs; coalitions, partnerships, and networks; organizational staff capability; and evaluation, feedback, and program outcomes) were also significantly associated with current SAMHSA funding status (p < 0.5 to 0.01). With the exception of organizational staff capability, all sustainability determinants were unrelated to program adaptation as predicted.

Conclusions

The SMSS demonstrated good reliability and convergent and discriminant validity in assessing likelihood of sustainment of SAMHSA funded prevention programs and initiatives. The measure demonstrates potential in identifying predictors of program sustainment and as a tool for enhancing the likelihood of successful sustainment through ongoing evaluation and feedback.

What influences evidence-based treatment sustainment after implementation support ends? A mixed method study of the adolescent-community reinforcement approach

BACKGROUND

Little is known about clinician perspectives regarding the factors that support or hinder the long-term delivery (i.e., sustainment) of evidence-based treatments in community-based treatment settings.

METHODS

Clinical staff from 82 community-based treatment organizations that received federal grant funding to support the delivery of the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based treatment for adolescent substance use, were asked to participate in interviews focused on understanding their perspectives about the sustainment of A-CRA. Qualitative themes were identified using inductive and deductive approaches. Then the themes were dichotomized (present/absent) so that quantitative comparisons could be made between staff from organizations that sustained and did not sustain delivery of A-CRA. Administrative data about each organization in relation to federal funding support and their primary focus was also examined to explore whether these characteristics were associated with A-CRA sustainment.

RESULTS

Staff (n = 134) representing 78 organizations participated in the interviews. Staff from organizations that had received multiple federal grants to support the delivery of A-CRA and whose primary focus was substance use rather than other conditions (mental health or primary care) were more likely to report sustaining A-CRA. Staff from sustaining organizations were more likely to report positive grant experiences and success with maintaining both organizational and external support in comparison to staff from non-sustaining organizations. Staff from non-sustaining organizations were more likely to report barriers to sustaining A-CRA, including more challenges with intervention delivery, and lack of internal support and external funding.

CONCLUSIONS

Our findings lend empirical support for implementation theories in that multiple factors appear to be associated with long-term delivery of an evidence-based treatment. Although A-CRA was generally perceived positively by staff from both organizations that sustained A-CRA and organizations that did not sustain A-CRA, inner setting factors (e.g., structural policies, leadership support and staff retention) along with outer setting factors (e.g., external funding support) were reported as key to A-CRA sustainment.

Conceptualizing and measuring sustainability of prevention programs, policies, and practices

A large knowledge gap exists regarding the measurement of sustainability of evidence-based prevention programs for mental and behavioral health. We interviewed 45 representatives of 10 grantees and 9 program officers within 4 Substance Abuse and Mental Health Services Administration prevention grant initiatives to identify experiences with implementation and sustainability barriers and facilitators; what "sustainability" means and what it will take to sustain their programs; and which Consolidated Framework for Implementation Research (CFIR) elements are important for sustainability. Lists of sustainability determinants and outcomes were then compiled from each data set and compared with one another. Analysis of themes from interviews and free lists revealed considerable overlap between sustainability determinants and outcomes. Four sustainability elements were identified by all three data sets (ongoing coalitions, collaborations, and networks and partnerships; infrastructure and capacity to support sustainability; community need for program; and ongoing evaluation of performance and outcomes), and 11 elements were identified by two of three data sets (availability of funding; consistency with organizational culture; evidence of positive outcomes; development of a plan for implementation and sustainment; presence of a champion; institutionalization and integration of program; institutional support and commitment; community buy-in and support; program continuity; supportive leadership; and opportunities for staff training). All but one of the CFIR domain elements (pressure from other states, tribes, or communities) were endorsed as important to sustainability by 50% or more of participants. It may be more important to implement a standardized process of eliciting determinants and outcomes of sustainability than to implement a single standardized instrument.

Improving the normalization of complex interventions: part 2 - validation of the NoMAD instrument for assessing implementation work based on normalization process theory (NPT)

INTRODUCTION

Successful implementation and embedding of new health care practices relies on co-ordinated, collective behaviour of individuals working within the constraints of health care settings. Normalization Process Theory (NPT) provides a theory of implementation that emphasises collective action in explaining, and shaping, the embedding of new practices. To extend the practical utility of NPT for improving implementation success, an instrument (NoMAD) was developed and validated.

METHODS

Descriptive analysis and psychometric testing of an instrument developed by the authors, through an iterative process that included item generation, consensus methods, item appraisal, and cognitive testing. A 46 item questionnaire was tested in 6 sites implementing health related interventions, using paper and online completion. Participants were staff directly involved in working with the interventions. Descriptive analysis and consensus methods were used to remove redundancy, reducing the final tool to 23 items. Data were subject to confirmatory factor analysis which sought to confirm the theoretical structure within the sample.

RESULTS

We obtained 831 completed questionnaires, an average response rate of 39% (range: 22-77%). Full completion of items was 50% (n = 413). The confirmatory factor analysis showed the model achieved acceptable fit (CFI = 0.95, TLI = 0.93, RMSEA = 0.08, SRMR = 0.03). Construct validity of the four theoretical constructs of NPT was supported, and internal consistency (Cronbach's alpha) were as follows: Coherence (4 items, α = 0.71); Collective Action (7 items, α = 0.78); Cognitive Participation (4 items, α = 0.81); Reflexive Monitoring (5 items, α = 0.65). The normalisation scale overall, was highly reliable (20 items, α = 0.89).

CONCLUSIONS

The NoMAD instrument has good face validity, construct validity and internal consistency, for assessing staff perceptions of factors relevant to embedding interventions that change their work practices. Uses in evaluating and guiding implementation are proposed.

Supporting the implementation of stroke quality-based procedures (QBPs): a mixed methods evaluation to identify knowledge translation activities, knowledge translation interventions, and determinants of implementation across Ontario

Background

In 2013, Health Quality Ontario introduced stroke quality-based procedures (QBPs) to promote use of evidence-based practices for patients with stroke in Ontario hospitals. The study purpose was to: (a) describe the knowledge translation (KT) interventions used to support stroke QBP implementation, (b) assess differences in the planned and reported KT interventions by region, and (c) explore determinants perceived to have affected outcomes.

Methods

A mixed methods approach was used to evaluate: activities, KT interventions, and determinants of stroke QBP implementation. In Phase 1, a document review of regional stroke network work plans was conducted to capture the types of KT activities planned at a regional level; these were mapped to the knowledge to action framework. In Phase 2, we surveyed Ontario hospital staff to identify the KT interventions used to support QBP implementation at an organizational level. Phase 3 involved qualitative interviews with staff to elucidate deeper understanding of survey findings.

Results

Of the 446 activities identified in the document review, the most common were ‘dissemination’ (24.2%; n = 108), ‘implementation’ (22.6%; n = 101), ‘implementation planning’ (15.0%; n = 67), and ‘knowledge tools’ (10.5%; n = 47). Based on survey data (n = 489), commonly reported KT interventions included: staff educational meetings (43.1%; n = 154), champions (41.5%; n = 148), and staff educational materials (40.6%; n = 145). Survey participants perceived stroke QBP implementation to be successful (median = 5/7; interquartile range = 4–6; range = 1–7; n = 335). Forty-four people (e.g., managers, senior leaders, regional stroke network representatives, and frontline staff) participated in interviews/focus groups. Perceived facilitators to QBP implementation included networks and collaborations with external organizations, leadership engagement, and hospital prioritization of stroke QBP. Perceived barriers included lack of funding, size of the hospital (i.e., too small), lack of resources (i.e., staff and time), and simultaneous implementation of other QBPs.

Conclusions

Information on the types of activities and KT interventions used to support stroke QBP implementation and the key determinants influencing uptake of stroke QBPs can be used to inform future activities including the development and evaluation of interventions to address barriers and leverage facilitators.

Electronic supplementary material

The online version of this article (10.1186/s12913-018-3220-9) contains supplementary material, which is available to authorized users.

The Sustainability of Evidence-Based Interventions and Practices in Public Health and Health Care

There is strong interest in implementation science to address the gap between research and practice in public health. Research on the sustainability of evidence-based interventions has been growing rapidly. Sustainability has been defined as the continued use of program components at sufficient intensity for the sustained achievement of desirable program goals and population outcomes. This understudied area has been identified as one of the most significant translational research problems. Adding to this challenge is uncertainty regarding the extent to which intervention adaptation and evolution are necessary to address the needs of populations that differ from those in which interventions were originally tested or implemented. This review critically examines and discusses conceptual and methodological issues in studying sustainability, summarizes the multilevel factors that have been found to influence the sustainability of interventions in a range of public health and health care settings, and highlights key areas for future research.

Measuring sustainment of an evidence based treatment for adolescent substance use

BACKGROUND

Measuring the extent a behavioral treatment is sustained in usual care practice settings after initial support for implementation ends is challenging. Oftentimes dichotomous self-report measures are used to measure health program sustainment, but these measures may not adequately capture the quality or extent of behavioral treatment delivery. Using data collected from community based organizations that received support to implement the Adolescent Community Reinforcement Approach (A-CRA), a measure of the extent of sustainment was derived.

METHODS

Based on implementation theories and implementation support protocols, a total of 10 core treatment elements were identified to measure the extent of sustainment using information collected from key clinical staff. Item response theory (IRT) and principal component analyses (PCA) were used to further refine the 10 elements into composite measures of sustainment. The association between the 10 elements and the relationships between the comprehensive measures to a dichotomous sustainment measure were also examined.

RESULTS

Results from PCA identified two components from the initial 10 elements for measuring the extent of A-CRA treatment sustainment. The two components described different aspects of organizational support for A-CRA treatment implementation, one representing the quality or extent of treatment staffing and delivery and the other representing the quality of or extent of clinical supervision.

CONCLUSIONS

Using IRT and PCA, we were able to derive components that could be used to measure the extent of EBT sustainment and also better capture the quality of treatment delivery than the use of simple dichotomous measure. The methodological contribution of our study is that we have demonstrated a general analytic approach that may be applicable for other psychosocial treatments.

Predicting evidence-based treatment sustainment: results from a longitudinal study of the Adolescent-Community Reinforcement Approach

BACKGROUND

Implementation support models are increasingly being used to enhance the delivery of evidence-based treatments (EBTs) in routine care settings. Little is known about the extent to which these models lead to continued EBT use after implementation support ends. Moreover, few empirical studies longitudinally examine the hypothesized factors associated with long-term psychosocial EBT use (i.e., sustainment). In an effort to address this gap, the current study examined sustainment of an EBT called the Adolescent-Community Reinforcement Approach (A-CRA) following the end of implementation support.

METHODS

Between 2006 and 2010, the Substance Abuse and Mental Health Services Administration awarded 3 years of A-CRA implementation support to 82 community-based organizations around the USA. The extent to which A-CRA was sustained following grant end and the hypothesized factors associated with EBT sustainment were collected using both retrospective and prospective data. We examined the extent to which 10 core treatment elements of A-CRA were sustained and the associations between the extent of A-CRA sustainment and hypothesized factors using a pattern-mixture longitudinal modeling approach.

RESULTS

Staff from 76 organizations participated in data collection for a 92.86% response rate. On average, about half of the 10 core treatment elements were sustained following the loss of implementation support. Factors that appeared most important to A-CRA sustainment included characteristics that were related to the outer setting (communication, funding, and partnerships), inner setting (political support, organizational capacity, and supervisor turnover rate), implementation support period (number of clinicians and supervisors certified and employed at support end and number of youth served), and staff perceptions of the intervention (implementation difficulty, relative advantage, and perceived success).

CONCLUSIONS

Even with multiple years of implementation support, community-based organizations face challenges in sustaining EBT delivery over time. Consistent with implementation theories, multiple factors appear related with EBT sustainment, including the degree of implementation during the initial support period, as well as adequate funding, infrastructure support, and staff support following the end of funding.

Sustained Implementation of Evidence-based Programs in Disadvantaged Communities: A Conceptual Framework of Supporting Factors

This paper presents a review of the empirical literature for studies evaluating factors that facilitate and create barriers to sustained program implementation in disadvantaged communities. It outlines study methodology and sustainment outcomes and proposes a conceptual model that involves implementation sustainment support for providers delivering evidence-based health and family services in disadvantaged communities. Sustained program implementation in the community setting is a significant issue as only 43% of studies reported successfully sustained programs. The review identified 18 factors that facilitate success and create barriers to program sustainment. The factors are synthesized into three themes; program characteristics, workplace capacity, and process and interaction factors. The majority of factors map onto commonly cited sustainability influences in implementation science. However, there was an additional focus for studies included in this review on the importance of factors such as program burden, program familiarity and perceived competence in program skills, workplace support for the program, staff mobility and turnover, supervision and peer support, and ongoing technical assistance. The need to use a conceptual framework and develop measures to guide and evaluate capacity building in EBP implementation and sustainment in low-resource community settings is highlighted.