Advancing understanding and identifying strategies for sustaining evidence-based practices: a review of reviews

A systematic review of experimentally tested implementation strategies across health and human service settings: evidence from 2010-2022

BACKGROUND

Studies of implementation strategies range in rigor, design, and evaluated outcomes, presenting interpretation challenges for practitioners and researchers. This systematic review aimed to describe the body of research evidence testing implementation strategies across diverse settings and domains, using the Expert Recommendations for Implementing Change (ERIC) taxonomy to classify strategies and the Reach Effectiveness Adoption Implementation and Maintenance (RE-AIM) framework to classify outcomes.

METHODS

We conducted a systematic review of studies examining implementation strategies from 2010-2022 and registered with PROSPERO (CRD42021235592). We searched databases using terms "implementation strategy", "intervention", "bundle", "support", and their variants. We also solicited study recommendations from implementation science experts and mined existing systematic reviews. We included studies that quantitatively assessed the impact of at least one implementation strategy to improve health or health care using an outcome that could be mapped to the five evaluation dimensions of RE-AIM. Only studies meeting prespecified methodologic standards were included. We described the characteristics of studies and frequency of implementation strategy use across study arms. We also examined common strategy pairings and cooccurrence with significant outcomes.

FINDINGS

Our search resulted in 16,605 studies; 129 met inclusion criteria. Studies tested an average of 6.73 strategies (0-20 range). The most assessed outcomes were Effectiveness (n=82; 64%) and Implementation (n=73; 56%). The implementation strategies most frequently occurring in the experimental arm were Distribute Educational Materials (n=99), Conduct Educational Meetings (n=96), Audit and Provide Feedback (n=76), and External Facilitation (n=59). These strategies were often used in combination. Nineteen implementation strategies were frequently tested and associated with significantly improved outcomes. However, many strategies were not tested sufficiently to draw conclusions.

CONCLUSION

This review of 129 methodologically rigorous studies built upon prior implementation science data syntheses to identify implementation strategies that had been experimentally tested and summarized their impact on outcomes across diverse outcomes and clinical settings. We present recommendations for improving future similar efforts.

Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C): development and psychometric evaluation of a measure of sustainability determinants in the early childhood education and care setting

BACKGROUND

There is a need for valid and reliable measures of determinants of sustainability of public health interventions in early childhood education and care (ECEC) settings. This study aimed to develop and evaluate the psychometric and pragmatic properties of such a measure - the Integrated Measure of PRogram Element SuStainability in Childcare Settings (IMPRESS-C).

METHODS

We undertook a two-phase process guided by the COnsensus-based Standards for the selection of health status Measurement INstruments checklist (COSMIN) and Psychometric and Pragmatic Evidence Rating Scale (PAPERS). Phase 1 involved measure development; i.e., determining items and scales through an iterative process and assessment of face and content validity. Phase 2 involved the evaluation of psychometric and pragmatic properties. The 29-item measure completed by service executives (directors and nominated supervisors) was embedded in a larger survey from a national sample of Australian ECEC services assessing their implementation of nutrition and physical activity programs. Structural validity, concurrent validity, known groups validity, internal consistency, floor and ceiling effects, norms, and pragmatic qualities of the measure were assessed according to the PAPERS criteria.

RESULTS

The final measure contained 26 items, with respondents reporting how strongly they agreed or disagreed on a five-point Likert scale. Phase 1 assessments confirmed the relevance, and face and content validity of the scale. In Phase 2, we obtained 482 completed surveys, of which 84% (n = 405) completed the entire measure across 405 ECEC settings (one executive per service). Three of the four fit indices for the confirmatory factor analysis met the pre-specified criteria (SRMR = 0.056, CFI = 0.993, RMSEA = 0.067) indicating 'good' structural validity. The IMPRESS-C illustrated: 'good' internal consistency, with Cronbach's alpha values from 0.53 to 0.92; 'emerging' concurrent validity; 'poor' known groups validity; 'good' norms; and 'good' overall pragmatic qualities (cost, readability, length, and assessor burden).

CONCLUSIONS

The IMPRESS-C possesses strong psychometric and pragmatic qualities for assessing service executive-level perceptions of determinants influencing sustainment of public health interventions within ECEC settings. To achieve a full range of perspectives in this setting, future work should be directed to also develop and test measures of sustainability determinants at the implementer level (e.g., among individual educators and staff).

Role of coproduction in the sustainability of innovations in applied health and social care research: a scoping review

BACKGROUND

Innovations such as toolkits and frameworks are developed through applied health and social care research, to address identified gaps in quality or safety of care. The intention is to subsequently implement these innovations into practice to bring about improvements. Challenges can arise from poor choice of implementation strategies or lack of alignment to local contexts. Research has identified the importance of involving and engaging patients, health professionals and other stakeholders in the design and delivery of the underpinning research, and in informing subsequent implementation. However, how and why such coproduction influences the sustainability of innovations in health and social care is unclear.

OBJECTIVE

The objective of this scoping review is to identify and present the available evidence regarding the role of coproduction in the sustainability of innovations in applied health and social care research.

INCLUSION CRITERIA

This scoping review includes papers related to the role of coproduction in the sustainability of innovations in applied health and social care research published in peer-reviewed journals. The review is limited to articles reporting applied health and social care research conducted in the United Kingdom.

METHODS

Scopus, Web of Science, CINAHL and MEDLINE were searched for studies. Titles and abstracts were screened by two independent reviewers for assessment against the inclusion criteria, followed by a full-text review and data extraction. Data were extracted using a data extraction form developed by the reviewers. The completed forms were imported into NVivo and analysed using basic qualitative content analysis.

RESULTS

Our review provides insight into the role of coproduction in the sustainability of innovations in applied health and social care research. Our findings highlight that sustainability is a dynamic process, supported by coproduction activities such as ongoing collaborative partnerships; these can be planned for in both the research design and implementation phases of a project.

Defining the transition from new to normal: a qualitative investigation of the clinical change process

Background

Understanding how and when a new evidence-based clinical intervention becomes standard practice is crucial to ensure that healthcare is delivered in alignment with the most up-to-date knowledge. However, rigorous methods are needed to determine when a new clinical practice becomes normalized to the standard of care. To address this gap, this study qualitatively explores how, when, and why a clinical practice change becomes normalized within healthcare organizations.

Methods

We used purposive sampling to recruit clinical leaders who worked in implementation science across diverse health contexts. Enrolled participants completed semi-structured interviews. Qualitative data analysis was guided by a modified version of the Normalization Process Theory (NPT) framework to identify salient themes. Identified normalization strategies were mapped to the Expert Recommendations for Implementation Change (ERIC) project.

Results

A total of 17 individuals were interviewed. Participants described four key signals for identifying when a novel clinical practice becomes the new normal: 1) integration into existing workflows; 2) scaling across the entire organizational unit; 3) staff buy-in and ownership; and 4) sustainment without ongoing monitoring. Participants identified salient strategies to normalize new clinical interventions: 1) taking a patient approach; 2) gaining staff buy-in and ownership; and 3) conducting ongoing measurement of progress towards normalization.

Conclusions

The results offer valuable insight into the indicators that signify when a novel clinical practice becomes normalized, and the strategies employed to facilitate this transition. These findings can inform future research to develop instruments that implementation leaders can use to systematically measure the clinical change process.

Innovation in the Delivery of Behavioral Health Services

Several factors motivate the need for innovation to improve the delivery of behavioral health services, including increased rates of mental health and substance use disorders, limited access to services, inconsistent use of evidence-based practices, and persistent racial and ethnic disparities. This narrative review identifies promising innovations that address these challenges, assesses empirical evidence for the effectiveness of these innovations and the extent to which they have been adopted and implemented, and suggests next steps for research. We review five categories of innovations: organizational models, including a range of novel locations for providing services and new ways of organizing services within and across sites; information and communication technologies; workforce; treatment technologies; and policy and regulatory changes. We conclude by discussing the need to strengthen and accelerate the contributions of implementation science to close the gap between the launch of innovative behavioral health services and their widespread use.

Implementation considerations for risk-tailored cancer screening in the population: A scoping review

BACKGROUND

Risk-tailored screening has emerged as a promising approach to optimise the balance of benefits and harms of existing population cancer screening programs. It tailors screening (e.g., eligibility, frequency, interval, test type) to individual risk rather than the current one-size-fits-all approach of most organised population screening programs. However, the implementation of risk-tailored cancer screening in the population is challenging as it requires a change of practice at multiple levels i.e., individual, provider, health system levels. This scoping review aims to synthesise current implementation considerations for risk-tailored cancer screening in the population, identifying barriers, facilitators, and associated implementation outcomes.

METHODS

Relevant studies were identified via database searches up to February 2023. Results were synthesised using Tierney et al. (2020) guidance for evidence synthesis of implementation outcomes and a multilevel framework.

RESULTS

Of 4138 titles identified, 74 studies met the inclusion criteria. Most studies in this review focused on the implementation outcomes of acceptability, feasibility, and appropriateness, reflecting the pre-implementation stage of most research to date. Only six studies included an implementation framework. The review identified consistent evidence that risk-tailored screening is largely acceptable across population groups, however reluctance to accept a reduction in screening frequency for low-risk informed by cultural norms, presents a major barrier. Limited studies were identified for cancer types other than breast cancer.

CONCLUSIONS

Implementation strategies will need to address alternate models of delivery, education of health professionals, communication with the public, screening options for people at low risk of cancer, and inequity in outcomes across cancer types.

An initial typology of approaches used by policy and practice agencies to achieve sustained implementation of interventions to improve health

BACKGROUND

Scientific investigation of how to sustain the implementation of evidence-based interventions (EBI) is emerging. Sustaining the implementation of EBIs helps ensure their effects on improving health endure. External policy or practice agencies, such as government health departments, are often tasked with supporting individual organisations with sustaining their delivery of EBIs, for example, through financing, training or the provision of other supports. However, to our knowledge, the approaches taken by policy and practice agencies to support the sustainment of EBIs have not been consolidated, categorised and described as a typology.

MAIN BODY

To improve conceptual clarity and support both research and practice, we developed an initial working typology of the practical approaches to sustain implementation of EBIs (i.e. sustainment) in order to improve long term health from the perspective of these agencies. The working typology includes three broad approaches. The first, termed 'Self-Sustainment', is when implementation of the EBI by an organisation (e.g. hospital, clinic, school) is expected to continue (sustain) in the absence of external (agency) support. The second, termed 'Static Sustainment Support', involves the provision of pre-defined external (agency) support to assist organisations to continue implementation of an EBI. The final approach is termed 'Dynamic Sustainment Support', whereby support provided by an external agency is dynamic (continues to be adapted) overtime to assist organisations continue implementation of an intervention which may itself also evolve.

CONCLUSIONS

We describe the contexts and circumstances where each may be most appropriate in achieving sustained implementation and discuss their research and practice implications.

Connecting Clinical Capacity and Intervention Sustainability in Resource-Variable Pediatric Oncology Centers in Latin America

Clinical capacity for sustainability, or the clinical resources needed to sustain an evidence-based practice, represent proximal determinants that contribute to intervention sustainment. We examine the relationship between clinical capacity for sustainability and sustainment of PEWS, an evidence-based intervention to improve outcomes for pediatric oncology patients in resource-variable hospitals. We conducted a cross-sectional survey among Latin American pediatric oncology centers participating in Proyecto Escala de Valoración de Alerta Temprana (EVAT), an improvement collaborative to implement Pediatric Early Warning Systems (PEWS). Hospitals were eligible if they had completed PEWS implementation. Clinicians were eligible to participate if they were involved in PEWS implementation or used PEWS in clinical work. The Spanish language survey consisted of 56 close and open-ended questions about the respondent, hospital, participants' assessment of clinical capacity to sustain PEWS using the clinical sustainability assessment tool (CSAT), and perceptions about PEWS and its use as an intervention. Results were analyzed using a multi-level modeling approach to examine the relationship between individual, hospital, intervention, and clinical capacity determinants to PEWS sustainment. A total of 797 responses from 37 centers in 13 countries were included in the analysis. Eighty-seven percent of participants reported PEWS sustainment. After controlling for individual, hospital, and intervention factors, clinical capacity was significantly associated with PEWS sustainment (OR 3.27, p < .01). Marginal effects from the final model indicate that an increasing capacity score has a positive influence (11% for every additional CSAT point) of predicting PEWS sustainment. PEWS is a sustainable intervention and clinical capacity to sustain PEWS contributes meaningfully to PEWS sustainment.

Sustainability determinants of an intervention to identify clinical deterioration and improve childhood cancer survival in Latin American hospitals: the INSPIRE study protocol

BACKGROUND

More than 90% of children with cancer live in low-resourced settings, where survival is only 20%. Sustainable evidence-based (EB) interventions yielding ongoing beneficial patient outcomes are critical to improve childhood cancer survival. A better understanding of factors promoting intervention sustainability in these settings is urgently needed. The aim of this study is to provide an empirical understanding of how clinical capacity for sustainability, or the resources needed to sustain an intervention, impacts the sustainment of Pediatric Early Warning System (PEWS), an EB intervention that improves pediatric oncology outcomes in low-resource hospitals by detecting clinical deterioration and preventing the need for more intense treatment.

METHODS

We will conduct a prospective, longitudinal study of approximately 100 resource-variable hospitals implementing and sustaining PEWS participating in Proyecto EVAT, a quality improvement collaborative of Latin American pediatric oncology centers. Aim 1: We will evaluate how clinical capacity for sustainability changes over time through 5 to 9 prospective measurements of capacity via survey of clinical staff using PEWS (approximately n = 13 per center) during the phases of PEWS adoption, implementation, and sustainability using the Clinical Sustainability Assessment Tool (CSAT). Aim 2: We will determine the relationship between capacity and a) PEWS sustainment and b) clinical deterioration mortality among pediatric oncology patients at centers sustaining PEWS for 2 to 10 years using chart review and an existing patient outcomes registry. Aim 3: We will develop novel strategies to promote sustainability by gaining a deeper understanding of perceived challenges to building capacity and PEWS sustainment. In combination with quantitative outcomes, we will conduct 24 focus groups with staff (doctors, nurses, and administrators) from hospitals with both high (n = 4) and low capacity (n = 4). We will then use implementation mapping to generate theoretically driven, empirically-supported sustainability strategies.

DISCUSSION

This study will advance implementation science by providing a theoretically driven, foundational understanding of factors that predict sustainability among a large, diverse cohort of hospitals. We will then use this knowledge to develop sustainability evidence-informed strategies that optimize capacity and promote long-term sustainment of PEWS and improvements in patient outcomes, thus promoting equity in childhood cancer care globally.

External context in individual placement and support implementation: a scoping review with abductive thematic analysis

BACKGROUND

Context including the external context may considerably affect the adoption, implementation, sustainment, and scale-up of evidence-based practices. We investigated external contextual features by conducting a scoping review of empirical research regarding the implementation of an evidence-based psychiatric or mental health vocational rehabilitation service called Individual Placement and Support (IPS).

METHODS

The protocol for the scoping review was registered with the Open Science Framework. We used the methodology by Joanna Briggs Institute for conducting the scoping review and reported it according to the PRISMA-ScR checklist. We searched 12 databases for research regarding 'Individual Placement and Support' or 'Evidence-Based Supported Employment'. We retained peer-reviewed empirical studies investigating external contextual factors and their impact on IPS implementation outcomes. We extracted data from the eligible articles and conducted descriptive and thematic analyses.

RESULTS

Fifty-nine original research papers met our eligibility requirements and were retained after reviewing 1124 titles and abstracts and 119 full texts. The analysis generated two main themes: (1) external contextual determinants of service delivery and (2) external systems influencing the evidence-to-practice process. The first main theme encompassed policies and laws, financing, and administratively instituted support resources, and organizational arrangements associated with external stakeholders that may facilitate or hinder the local implementation. The second main theme comprised strategies and actions used by different stakeholders to facilitate implementation locally or scale-up efforts at a system level.

DISCUSSION

Our scoping review illustrates the important role that external contextual factors play and how they may facilitate or hinder the implementation and scale-up of the IPS model across mental health services in different countries. Consideration of these factors by decision-makers in mental health and welfare services, planners, providers, and practitioners is likely to facilitate the development of effective strategies for bridging the evidence-practice gap in implementing the EBPs. Finally, the scoping review identified gaps in knowledge and offered suggestions for future research.

TRIAL REGISTRATION

Open Science Framework.

Sustainability, spread, and scale in trials using audit and feedback: a theory-informed, secondary analysis of a systematic review

BACKGROUND

Audit and feedback (A&F) is a widely used implementation strategy to influence health professionals' behavior that is often tested in implementation trials. This study examines how A&F trials describe sustainability, spread, and scale.

METHODS

This is a theory-informed, descriptive, secondary analysis of an update of the Cochrane systematic review of A&F trials, including all trials published since 2011. Keyword searches related to sustainability, spread, and scale were conducted. Trials with at least one keyword, and those identified from a forward citation search, were extracted to examine how they described sustainability, spread, and scale. Results were qualitatively analyzed using the Integrated Sustainability Framework (ISF) and the Framework for Going to Full Scale (FGFS).

RESULTS

From the larger review, n = 161 studies met eligibility criteria. Seventy-eight percent (n = 126) of trials included at least one keyword on sustainability, and 49% (n = 62) of those studies (39% overall) frequently mentioned sustainability based on inclusion of relevant text in multiple sections of the paper. For spread/scale, 62% (n = 100) of trials included at least one relevant keyword and 51% (n = 51) of those studies (31% overall) frequently mentioned spread/scale. A total of n = 38 studies from the forward citation search were included in the qualitative analysis. Although many studies mentioned the need to consider sustainability, there was limited detail on how this was planned, implemented, or assessed. The most frequent sustainability period duration was 12 months. Qualitative results mapped to the ISF, but not all determinants were represented. Strong alignment was found with the FGFS for phases of scale-up and support systems (infrastructure), but not for adoption mechanisms. New spread/scale themes included (1) aligning affordability and scalability; (2) balancing fidelity and scalability; and (3) balancing effect size and scalability.

CONCLUSION

A&F trials should plan for sustainability, spread, and scale so that if the trial is effective, the benefits can continue. A deeper empirical understanding of the factors impacting A&F sustainability is needed. Scalability planning should go beyond cost and infrastructure to consider other adoption mechanisms, such as leadership, policy, and communication, that may support further scalability.

TRIAL REGISTRATION

Registered with Prospero in May 2022. CRD42022332606.

A cluster randomised controlled trial to assess the effectiveness of a multi-strategy sustainability intervention on teachers' sustained implementation of classroom physical activity breaks (energisers): study protocol

BACKGROUND

Governments internationally have invested hugely in the implementation and scale-up of school-based physical activity interventions, but have little evidence of how to best sustain these interventions once active implementation support ceases. This study will assess the effectiveness of a multi-strategy sustainability intervention on classroom teachers' sustainment of energisers (short 3-5 min physical activity breaks during class-time) scheduled across the school day from baseline to 12 and 24-month follow-up.

METHODS

A cluster randomised controlled trial will be conducted in 50 primary schools within the Hunter New England, Illawarra Shoalhaven, Murrumbidgee and Northern New South Wales (NSW) Local Health Districts of NSW Australia. Schools will be randomly allocated to receive either usual support or the multi-strategy sustainability intervention that includes: centralised technical assistance from a trained project officer; formal commitment and mandated change obtained from school principals; training in-school champions; reminders for teachers; educational materials provided to teachers; capturing and sharing local knowledge; and engagement of parents, carers and the wider school community. The primary trial outcome will be measured via a teacher logbook to determine the between-group difference in the change in mean minutes of energisers scheduled across the school day at 12 and 24-month follow-up compared to baseline. Analyses will be performed using an intention to treat framework. Linear mixed models will be used to assess intervention effects on the primary outcome at both follow-up periods.

DISCUSSION

This study will be one of the first randomised controlled trials to examine the impact of a multi-strategy sustainability intervention to support schools' sustainment of a physical activity intervention. The proposed research will generate new evidence needed for the partnering organisations to protect their considerable investments to date in physical activity promotion in this setting and will provide seminal evidence for the field globally.

TRIAL REGISTRATION

ACTRN12620000372987 version 1 registered 17th March 2020. Version 3 (current version) updated 4th August 2023.

Supporting Sustained STAR-VA Implementation: Evaluation of a Sustainment Coaching Program

OBJECTIVES

The Veterans Health Administration (VA) STAR-VA program is a person-centered, interdisciplinary intervention for managing distress behaviors in dementia in VA nursing homes, called Community Living Centers (CLCs). Teams often struggle to sustain STAR-VA after site-lead partner turnover. The STAR-VA Sustainment Coaching Program was developed to support a new local STAR-VA behavioral coordinator and/or nurse champion (ie, site-leads) and engage the team to implement sustainment strategies. The aims of this evaluation were to document sustainment needs, barriers and facilitators, and effective strategies used during a sustainment intervention.

DESIGN

We describe qualitative and quantitative findings of a program evaluation of the STAR-VA Sustainment Coaching pilot program.

SETTING AND PARTICIPANTS

Eight CLC teams participated in the pilot program, 5 training a new nurse champion, 2 a new behavioral coordinator, and 1 both new site-leads.

METHODS

CLC teams completed a needs assessment, developed sustainment goals and plans, tracked sustainment interventions implemented, and reported case outcomes. Outcome assessment and tracking reports were summarized and themes identified using a qualitative inductive approach.

RESULTS

Common sustainment needs were to train a new nurse champion, promote staff understanding of dementia and distress behaviors, and promote a culture of person-centered dementia care. Most sites selected sustainment goals of implementing training procedures and behavioral rounds. Sustainment barriers included limited staff time, staff turnover, lack of supportive routines and tools, and limited awareness of STAR-VA. Facilitators included leadership support, staff interest, and training resources. Most sites reported successful implementation of STAR-VA behavior rounds, whereas strategies to achieve training-related goals had variable success.

CONCLUSIONS AND IMPLICATIONS

The STAR-VA Sustainment Coaching Program helped CLC teams identify and implement strategies to support sustained STAR-VA implementation. The results of this project are informing ongoing efforts to integrate STAR-VA components into usual care processes for CLC residents with dementia.

Factors That Support Sustainability of Health Systems Change to Increase Colorectal Cancer Screening in Primary Care Clinics: A Longitudinal Qualitative Study

BACKGROUND

From 2015 to 2020, the Centers for Disease Control and Prevention's Colorectal Cancer Control Program (CRCCP) supported 30 awardees in partnering with primary care clinics to implement evidence-based interventions (EBIs) and supporting activities (SAs) to increase colorectal cancer (CRC) screening. This study identified factors that facilitated early implementation and sustainability within partner clinics.

METHODS

We conducted longitudinal qualitative case studies of four CRCCP awardees and four of their partner clinics. We used the Consolidated Framework for Implementation Research (CFIR) to frame understanding of factors related to implementation and sustainability. A total of 41 semi-structured interviews were conducted with key staff and stakeholders exploring implementation practices and facilitators to sustainability. Qualitative thematic analysis of interview transcripts identified emerging themes across awardees and clinics.

RESULTS

Qualitative themes related to six CFIR inner setting constructs-structural characteristics, readiness for implementation, networks and communication, culture, and implementation climate-were identified. Themes related to early implementation included conducting readiness assessments to tailor implementation, providing moderate funding to clinics, identifying clinic champions, and coordinating EBIs and SAs with existing clinic practices. Themes related to sustainability included the importance of ongoing electronic health record (EHR) support, clinic leadership support, team-based care, and EBI and SA integration with clinic policies, workflows, and procedures.

IMPLICATIONS

Findings help to inform future scale-up of and decision-making within CRC screening programs and other chronic disease prevention programs implementing EBIs and SAs within primary care clinics and also highlight factors that maximize sustainability within these programs.

Associations between depression, fear of COVID-19 infection and students' self-care measures used during the first wave of the pandemic

BACKGROUND

COVID-19 lockdown resulted in the closure of schools with associated problems. The aim of this study was to determine the associations between depression, fear of contracting COVID-19 infection and the use of self-care measures by college students during the first wave of the COVID-19 pandemic.

METHODS

This was a cross-sectional study that collected data from undergraduate and postgraduate college students 18 years and older from 152 countries between June and December 2020. Study participants were recruited through crowdsourcing using various social media platforms including Facebook, Twitter, and Instagram, WhatsApp groups and emails to participants in the collaborators' networks. The dependent variables were fear of contracting COVID-19 and depression while the independent variable was students' self-care measures. Multivariable logistic regression models were conducted to assess the associations between the dependent and independent variables.

RESULTS

Of the 2840 respondents, 1305 (46.0%) had fears of contracting COVID-19 and 599 (21.1%) reported depression. The most common self-care measures were phone calls with friends/family (60.1%) and video chat (52.8%). Learning a new skill was significantly associated with higher odds of fear of contracting COVID-19 (AOR = 1.669) and lower odds of having depression (AOR = 0.684). Talking to friends/family through video chat (AOR = 0.809) was significantly associated with lower odds of feeling depressed while spending time with pets (AOR = 1.470) and taking breaks from the news/social media (AOR = 1.242) were significantly associated with higher odds of feeling depressed. Students from lower middle-income countries (AOR = 0.330) had significantly lower odds of feeling depressed than students from low-income countries.

CONCLUSION

Self-care strategies involving social interactions were associated with less depression. Coping strategies with more cognitive demands may significantly reduce the risk of fear of COVID-19. Special attention needs to be given to students in low-income countries who have higher odds of depression during the pandemic than students from other countries.

Translation and cross-cultural adaptation of the National Health Service Sustainability Model to the Chinese healthcare context

BACKGROUND

International attention is being paid to the issue of making evidence sustainable after implementation. Developing an identification model is essential to promote and monitor the sustainability of evidence implementation. However, this model is not available in Chinese. This study aims to translate the National Health Service Sustainability Model into Chinese and to verify whether the model is adapted to the Chinese healthcare environment.

METHODS

This study follows the translation and validation guidelines developed by Sousa and Rojjanasrirat. The translations include forward and backward translations and their comparison. Expert reviews were used to validate the content validity of the Chinese version of the National Health Service sustainability model. Cognitive interviews were used to assess the validity of the language in the Chinese setting.

RESULTS

The translation was conducted by a bilingual research team and took 12 months. Expert reviews were undertaken with eight experts, and cognitive interviews with six participants. The content validity of the model is excellent, but at least 20% of the experts still felt that items one, three, five and nine needed refinements. In the cognitive interviews, most items, instructions and response options were well understood by the participants responsible for the evidence-based practice project. However, some language issues were still identified in items one, three, four, five, seven, nine, and ten. Participants reported that the sustainability results of the model assessment were consistent with their previous judgments of the items. Based on the expert review and interview results, items one, three, four, five, seven, nine and ten require further refinement. In summary, seven of the ten items have been amended.

CONCLUSIONS

This study provides insight into how the National Health Service sustainability model can be used in the Chinese healthcare setting and paves the way for future large-scale psychometric testing.

Developing a sustainability readiness strategy for health systems: Toolkit, interactive tools, and virtual support system

While the literature strongly supports the need for sustainability of evidence-based interventions (EBIs), we present a review of the literature that indicates only three articles discuss a health-focused sustainability strategy. The aims of our sustainability readiness strategy (SRS) are to increase infrastructure capacity and EBI advocacy to impact the level of sustainability readiness. In this article, we describe the development of an evidence-informed promising practice sustainability readiness strategy (SRS) with three evidence-based components. This strategy: 1) is based on an adaptation of the Getting To Outcomes® (GTO) evidence-based implementation process, 2) includes a logic model with documented evidence of the connection between targeted readiness factors and sustainability outcomes, and 3) describes resources considered necessary to support implementation of the readiness strategy, namely a step-by-step Toolkit, Excel™ Tools, webinar coaching and evaluation guides, and a coaching and evaluation training guide. The national SRS survey results are presented. Lessons learned and future dissemination and implementation plans are described.

A Pre-Implementation Enhancement Strategy to Increase the Yield of Training and Consultation for School-Based Behavioral Preventive Practices: a Triple-Blind Randomized Controlled Trial

As the most common setting where youth access behavioral healthcare, the education sector frequently employs training and follow-up consultation as cornerstone implementation strategies to promote the implementation of evidence-based practices (EBPs). However, these strategies alone are not sufficient to promote desirable implementation (e.g., intervention fidelity) and youth behavioral outcomes (e.g., mitigated externalizing behaviors). Theory-informed pragmatic pre-implementation enhancement strategies (PIES) are needed to prevent the lackluster outcomes of training and consultation. Specifically, social cognitive theory explicates principles that inform the design of PIES content and specify mechanisms of behavior change (e.g., "intentions to implement" (ITI)) to target increasing providers' responsiveness to training and consultation. This triple-blind parallel randomized controlled trial preliminarily examined the efficacy of a pragmatic PIES based on social cognitive theories (SC-PIES) to improve implementation and youth behavioral outcomes from universal preventive EBPs in the education sector. Teachers from a diverse urban district were recruited and randomly assigned to the treatment (SC-PIES; ntreatment = 22) or active control condition (administrative meeting; ncontrol = 21). Based on the condition assigned, teachers received the SC-PIES or met with administrators before their EBP training. We assessed teachers' ITI, intervention fidelity, and youth behavioral outcome (academic engagement as an incompatible behavior to externalizing disorders) at baseline, immediately after training, and 6 weeks afterward. A series of ANCOVAs detected sizeable effects of SC-PIES, where teachers who received SC-PIES demonstrated significantly larger improvement in their ITI, intervention fidelity, and youth behaviors as compared to the control. Conditional analyses indicated that teachers' ITI partially mediated the effect of SC-PIES on intervention fidelity, which in turn led to improved youth behaviors. Findings suggest that theory-informed pragmatic PIES targeting providers' ITI can boost their responsiveness to implementation strategies, as reflected in improved implementation behaviors and youth behavioral outcomes. The results have implications for targeting motivational mechanisms of behavior change and situating preventive implementation strategies at the intersection between the preparation and active implementation stages of an implementation process. Limitations and implications for research and practice are discussed. Clinicaltrials.gov: NCT05240222. Registered on: 2/14/2022. Retrospectively registered.  https://clinicaltrials.gov/show/NCT05240222.

Implementing a Financial Decision-Making Scale in APS Financial Exploitation Investigations: Use of the PARIHS Conceptual Framework

BACKGROUND AND OBJECTIVES

Adult protective services (APS) agencies investigate cases of financial exploitation, and a critical aspect of such investigations is often the assessment of decision-making abilities. This study examined APS workers' implementation of a 10-item financial decision-making screening tool, the Financial Decision Tracker (FDT), across a 34-month period: pre-COVID-19, throughout COVID-19 restrictions, and for 1 year following the restrictions.

RESEARCH DESIGN AND METHODS

Using the Promoting Action in Research Implementation in the Health Services implementation science conceptual framework, we examined aspects of context, facilitation, and evidence to determine how well APS workers were trained, certified, and skilled in using the FDT. Using individual and group interviews, we assessed factors often related to successful implementation (context and facilitation) and measured the number of scales used, the types of decisions under investigation, and how the tool's scoring system aligned with the APS workers' final ratings (evidence).

RESULTS

Overall, implementation was sustained throughout the 34-month period. However, during COVID-19 restrictions, usage dropped to a rate 58% below that prior to and following the restrictions. A total of 839 scales were administered. Individuals with no decision-making deficits were slightly older (78.7 vs 77.1 years; t(837) = 2.54, p = .01) and had completed high school at a significantly higher rate (69% vs 59%, χ 2(1) = 5.20, p = .023) than those who had decision-making deficits.

DISCUSSION AND IMPLICATIONS

Overall, the implementation trial can be considered a success. The FDT was used so often and effectively that it is now an APS best practices tool, meaning that in audits, the correct use of the FDT will be assessed.

Identifying Factors Affecting the Sustainability of the STAR-VA Program in the Veterans Health Administration

BACKGROUND AND OBJECTIVES

Sustained implementation of new programs in complex care systems like nursing homes is challenging. This prospective qualitative evaluation examined factors affecting the sustainability of the Staff Training in Assisted Living Residences in Veterans Health Administration (STAR-VA) program in Veterans Health Administration (VA) Community Living Centers (CLC, i.e., nursing homes). STAR-VA is an evidence-based interdisciplinary, resident-centered, behavioral approach for managing distress behaviors in dementia.

EVALUATION DESIGN AND METHODS

In 2019, we conducted 39 semistructured phone interviews with STAR-VA key informants across 20 CLCs. We identified a priori themes based on the Organizational Memory Framework, which includes 7 Knowledge Reservoirs (KRs): people, routines, artifacts, relationships, organizational information space, culture, and structure. We conducted content-directed analysis of transcripts to identify factors to program sustainment.

RESULTS

We identified 9 sustainment facilitators across KRs: engaged site leaders and champions, regular meetings and trainings, written documentation and resources, regular and open communication, available educational tools (e.g., handouts and posters), adequate spaces, leadership support on many levels, staff buy-in across disciplines, and staff competencies and recognition. Ten barriers across KRs included: staffing concerns, inconsistent/inefficient routines, inconsistent documentation, lack of written policies, communication gaps, nonstandardized use of tools, constraints with meeting spaces and regulations on posting information, limited leadership support, division among staff, and missing performance expectations.

DISCUSSION AND IMPLICATIONS

Findings inform tailored strategies for optimizing STAR-VA program sustainment in CLCs, including the development of a sustained implementation guide, implementation resources, regional communities of practice, and STAR-VA integration into national CLC quality improvement routines for team communication and problem-solving.

"It's all about the money": an interpretive description of embedding physical therapy-led falls prevention group exercise in long-term care

BACKGROUND

Falls prevention interventions are effective for community dwelling older adults however, the same cannot be said for older adults living in long-term care (LTC). The Staying UpRight (SUp) randomized controlled trial was designed to test the effectiveness of a progressive strength and balance group exercise program delivered to LTC residents. This paper explores the factors impacting LTC providers' decisions to continue the program on completion of the funded trial period.

METHODS

A qualitative study using an Interpretive Description approach. Semi-structured interviews and focus groups were conducted with 15 LTC staff involved in the randomized controlled trial. Data were analysed using conventional content analysis.

RESULTS

Practice change occurred following participation in the trial with some facilities starting exercise groups, some increasing the number of exercise groups offered and physical therapists selecting elements of the program to adopt into their practice. Decisions about continuing with SUp as designed were constrained by organizational decisions regarding funding and resources. Three factors were identified which informed decision-making: business models and philosophies, requirements for evidence, and valuing physical therapy.

CONCLUSIONS

Managers and facilitators adapted SUp by selecting and delivering components of the program in response to the changes they had observed in participating residents. However, our findings highlight that while SUp was valued, the tight financial environment created by the current funding model in New Zealand did not support funding physical therapist delivered falls prevention exercise programs in LTC. This study may provide policy makers with important information on changes needed to support falls prevention service delivery in LTC.

TRIAL REGISTRATION

This study is a sub-study of a randomized controlled trial which was registered to the Australian New Zealand Clinical Trials Registry ACTRN12618001827224 on 09/11/2018. Universal trial number U1111-1217-7148.

The TRIBE model: How socioemotional processes fuel end-of-life treatment in the United States

Prior interventions have repeatedly failed to decrease the prescription and receipt of treatments and procedures that confer more harm than benefit at the End-of-Life (EoL); new approaches to intervention are needed. Ideally, future interventions would be informed by a social-ecological conceptual model that explains EoL healthcare utilization patterns, but current models ignore two facts: (1) healthcare is an inherently social activity, involving clinical teams and patients' social networks, and (2) emotions influence social activity. To address these omissions, we scaffolded Terror Management Theory and Socioemotional Selectivity Theory to create the Transtheoretical Model of Irrational Biomedical Exuberance (TRIBE). Based on Terror Management Theory, TRIBE suggests that the prospect of patient death motivates healthcare teams to conform to a biomedical norm of care, even when clinicians believe that biomedical interventions will likely be unhelpful. Based on Socioemotional Selectivity Theory, TRIBE suggests that the prospect of dwindling time motivates families to prioritize emotional goals, and leads patients to consent to disease-directed treatments they know will likely be unhelpful, as moral emotions motivate deference to the perceived emotional needs of their loved ones. TRIBE is unique among models of healthcare utilization in its acknowledgement that moral emotions and processes (e.g., shame, compassion, regret-avoidance) influence healthcare delivery, patients' interactions with family members, and patients' outcomes. TRIBE is especially relevant to potentially harmful EoL care in the United States, and it also offers insights into the epidemics of overtreatment in healthcare settings worldwide. By outlining the role of socioemotional processes in the care of persons with serious conditions, TRIBE underscores the critical need for psychological innovation in interventions, health policy and research on healthcare utilization.

Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review

BACKGROUND

Sustainability is concerned with the long-term delivery and subsequent benefits of evidence-based interventions. To further this field, we require a strong understanding and thus measurement of sustainability and what impacts sustainability (i.e., sustainability determinants). This systematic review aimed to evaluate the quality and empirical application of measures of sustainability and sustainability determinants for use in clinical, public health, and community settings.

METHODS

Seven electronic databases, reference lists of relevant reviews, online repositories of implementation measures, and the grey literature were searched. Publications were included if they reported on the development, psychometric evaluation, or empirical use of a multi-item, quantitative measure of sustainability, or sustainability determinants. Eligibility was not restricted by language or date. Eligibility screening and data extraction were conducted independently by two members of the research team. Content coverage of each measure was assessed by mapping measure items to relevant constructs of sustainability and sustainability determinants. The pragmatic and psychometric properties of included measures was assessed using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The empirical use of each measure was descriptively analyzed.

RESULTS

A total of 32,782 articles were screened from the database search, of which 37 were eligible. An additional 186 publications were identified from the grey literature search. The 223 included articles represented 28 individual measures, of which two assessed sustainability as an outcome, 25 covered sustainability determinants and one explicitly assessed both. The psychometric and pragmatic quality was variable, with PAPERS scores ranging from 14 to 35, out of a possible 56 points. The Provider Report of Sustainment Scale had the highest PAPERS score and measured sustainability as an outcome. The School-wide Universal Behaviour Sustainability Index-School Teams had the highest PAPERS score (score=29) of the measure of sustainability determinants.

CONCLUSIONS

This review can be used to guide selection of the most psychometrically robust, pragmatic, and relevant measure of sustainability and sustainability determinants. It also highlights that future research is needed to improve the psychometric and pragmatic quality of current measures in this field.

TRIAL REGISTRATION

This review was prospectively registered with Research Registry (reviewregistry1097), March 2021.

Do the Expert Recommendations for Implementing Change (ERIC) strategies adequately address sustainment?

Background

Sustainability science is an emerging area within implementation science. There is limited evidence regarding strategies to best support the continued delivery and sustained impact of evidence-based interventions (EBIs). To build such evidence, clear definitions, and ways to operationalize strategies specific and/or relevant to sustainment are required. Taxonomies and compilations such as the Expert Recommendations for Implementing Change (ERIC) were developed to describe and organize implementation strategies. This study aimed to adapt, refine, and extend the ERIC compilation to incorporate an explicit focus on sustainment. We also sought to classify the specific phase(s) of implementation when the ERIC strategies could be considered and applied.

Methods

We used a two-phase iterative approach to adapt the ERIC. This involved: (1) adapting through consensus (ERIC strategies were mapped against barriers to sustainment as identified via the literature to identify if existing implementation strategies were sufficient to address sustainment, needed wording changes, or if new strategies were required) and; (2) preliminary application of this sustainment-explicit ERIC glossary (strategies described in published sustainment interventions were coded against the glossary to identify if any further amendments were needed). All team members independently reviewed changes and provided feedback for subsequent iterations until consensus was reached. Following this, and utilizing the same consensus process, the Exploration, Preparation, Implementation and Sustainment (EPIS) Framework was applied to identify when each strategy may be best employed across phases.

Results

Surface level changes were made to the definitions of 41 of the 73 ERIC strategies to explicitly address sustainment. Four additional strategies received deeper changes in their definitions. One new strategy was identified: Communicate with stakeholders the continued impact of the evidence-based practice. Application of the EPIS identified that at least three-quarters of strategies should be considered during preparation and implementation phases as they are likely to impact sustainment.

Conclusion

A sustainment-explicit ERIC glossary is provided to help researchers and practitioners develop, test, or apply strategies to improve the sustainment of EBIs in real-world settings. Whilst most ERIC strategies only needed minor changes, their impact on sustainment needs to be tested empirically which may require significant refinement or additions in the future.

Review article: A primer for clinical researchers in the emergency department: Part XII. Sustainability of improvements in care: An introduction

Despite an increased focus on ways to improve implementation of evidence and de-implementation of practices with no known benefit, there is limited guidance on how to sustain these improvements. This review provides an introduction to sustainability of improvements in care and sustainability research, discussing how to support sustainability in practice and detailing a sustainability research agenda for the emergency medicine setting.

Implementation Science: Increasing the Public Health Impact of Audiology Research

PURPOSE

Implementation science aims to facilitate the use of evidence-based programs, practices, and policies in routine care settings. In audiology, as in other health disciplines, there is a persistent research-to-practice gap. Improving the adoption, reach, implementation, and sustainment of effective interventions in audiology would increase their public health impact, ensuring that all individuals needing hearing health care services could benefit from innovations and evidence-based best practices. This tutorial provides an introductory overview of implementation science relevant to the field of audiology, including Internet-based practices and interventions.

METHOD

Major concepts and themes of implementation science are presented, including implementation outcomes, implementation science frameworks, implementation strategies, current topics in implementation science, and study design considerations. Recent publications in audiology are highlighted to illustrate implementation science concepts and themes. The relevance of each topic to the use of evidence-based programs, practices, and policies in audiology is highlighted with reference to recent research in the field.

CONCLUSIONS

Challenges in the widespread delivery of evidence-based audiological practices and interventions limit their public health impact. The application of implementation science principles and methods in audiology research, as demonstrated in other areas of health research, can increase our focus on ensuring that effective practices are widely available, accessible, equitable, and sustainable to improve the lives of those who need them.

Sustainability of evidence-based practices in the management of infants with bronchiolitis in hospital settings - a PREDICT study protocol

Background

Understanding how and why de-implementation of low-value practices is sustained remains unclear. The Paediatric Research in Emergency Departments International CollaboraTive (PREDICT) Bronchiolitis Knowledge Translation (KT) Study was a cluster randomised controlled trial conducted in 26 Australian and New Zealand hospitals (May-November 2017). Results showed targeted, theory-informed interventions (clinical leads, stakeholder meetings, train-the-trainer workshop, targeted educational package, audit/feedback) were effective at reducing use of five low-value practices for bronchiolitis (salbutamol, glucocorticoids, antibiotics, adrenaline and chest x-ray) by 14.1% in acute care settings. The primary aim of this study is to determine the sustainability (continued receipt of benefits) of these outcomes at intervention hospitals two-years after the removal of study supports. Secondary aims are to determine sustainability at one-year after removal of study support at intervention hospitals; improvements one-and-two years at control hospitals; and explore factors that influence sustainability at intervention hospitals and contribute to improvements at control hospitals.

Methods

A mixed-methods study design. The quantitative component is a retrospective medical record audit of bronchiolitis management within 24 hours of emergency department (ED) presentations at 26 Australian (n = 20) and New Zealand (n = 6) hospitals, which participated in the PREDICT Bronchiolitis KT Study. Data for a total of 1800 infants from intervention and control sites (up to 150 per site) will be collected to determine if improvements (i.e., no use of all five low-value practices) were sustained two- years (2019) post-trial (primary outcome; composite score); and a further 1800 infants from intervention and control sites will be collected to determine sustained improvements one- year (2018) post-trial (secondary outcome). An a priori definition of sustainability will be used. The qualitative component will consist of semi-structured interviews with three to five key emergency department and paediatric inpatient medical and nursing staff per site (total n = 78-130). Factors that may have contributed to sustaining outcomes and/or interventions will be explored and mapped to an established sustainability framework.

Discussion

This study will improve our understanding of the sustainability of evidence-based bronchiolitis management in infants. Results will also advance implementation science research by informing future de-implementation strategies to reduce low-value practices and sustain practice change in paediatric acute care.

Trial registration

Australian and New Zealand Clinical Trials Registry No: ACTRN12621001287820.

Knowledge and attitudes of Implementation Support Practitioners-Findings from a systematic integrative review

BACKGROUND

It requires thoughtful planning and work to successfully apply and sustain research-supported interventions like healthcare treatments, social support, or preventive programs in practice. Implementation support practitioners (ISPs) such as facilitators, technical assistance providers, knowledge brokers, coaches or consultants may be involved to actively support the implementation process. This article presents knowledge and attitudes ISPs bring to their work.

METHODS

Building on a previously developed program logic, a systematic integrative review was conducted. Literature was sourced by searching nine electronic data bases, organizational websites, and by launching a call for publications among selected experts and social media. Article screening was performed independently by two researchers, and data from included studies were extracted by members of the research team and quality-assured by the lead researcher. The quality of included RCTs was assessed based on a framework by Hodder and colleagues. Thematic Analysis was used to capture information on knowledge and attitudes of ISPs across the included studies. Euler diagrams and heatmaps were used to present the results.

RESULTS

Results are based on 79 included studies. ISPs reportedly displayed knowledge about the clinical practice they work with, implementation / improvement practice, the local context, supporting change processes, and facilitating evidence-based practice in general. In particular, knowledge about the intervention to be implemented and its target population, specific improvement / implementation methods and approaches, organizational structures and sensitivities, training, and characteristics of (good) research was described in the literature. Seven themes describing ISPs' attitudes were identified: 1) professional, 2) motivated / motivating / encouraging / empowering, 3) empathetic / respectful / sensitive, 4) collaborative / inclusive, 5) authentic, 6) creative / flexible / innovative / adaptive, and 7) frank / direct / honest. Pertaining to a professional attitude, being responsive and focused were the most prevalent indicators across included publications.

CONCLUSION

The wide range and complexity of knowledge and attitudes found in the literature calls for a comprehensive and systematic approach to collaboratively develop a professional role for ISPs across disciplines. Embedding the ISP role in different health and social welfare settings will enhance implementation capacities considerably.

Influences of post-implementation factors on the sustainability, sustainment, and intra-organizational spread of complex interventions

Background

Complex interventions are increasingly applied to healthcare problems. Understanding of post-implementation sustainment, sustainability, and spread of interventions is limited. We examine these phenomena for a complex quality improvement initiative led by care aides in 7 care homes (long-term care homes) in Manitoba, Canada. We report on factors influencing these phenomena two years after implementation.

Methods

Data were collected in 2019 via small group interviews with unit- and care home-level managers (n = 11) from 6 of the 7 homes using the intervention. Interview participants discussed post-implementation factors that influenced continuing or abandoning core intervention elements (processes, behaviors) and key intervention benefits (outcomes, impact). Interviews were audio-recorded, transcribed verbatim, and analyzed with thematic analysis.

Results

Sustainment of core elements and sustainability of key benefits were observed in 5 of the 6 participating care homes. Intra-unit intervention spread occurred in 3 of 6 homes. Factors influencing sustainment, sustainability, and spread related to intervention teams, unit and care home, and the long-term care system.

Conclusions

Our findings contribute understanding on the importance of micro-, meso-, and macro-level factors to sustainability of key benefits and sustainment of some core processes. Inter-unit spread relates exclusively to meso-level factors of observability and practice change institutionalization. Interventions should be developed with post-implementation sustainability in mind and measures taken to protect against influences such as workforce instability and competing internal and external demands. Design should anticipate need to adapt interventions to strengthen post-implementation traction.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08026-x.

Structured approaches to implementation of clinical genomics: A scoping review

PURPOSE

This study aimed to assess the extent to which structured approaches to implementation of clinical genomics, proposed or adapted, are informed by evidence.

METHODS

A systematic approach was used to identify peer-reviewed articles and gray literature to report on 4 research questions: 1. What structured approaches have been proposed to support implementation? 2. To what extent are the structured approaches informed by evidence? 3. How have structured approaches been deployed in the genomic setting? 4. What are the intended outcomes of the structured approaches?

RESULTS

A total of 30 unique structured approaches to implementation were reported across 23 peer-reviewed publications and 11 gray literature articles. Most approaches were process models, applied in the preadoption implementation phase, focusing on a "service" outcome. Key findings included a lack of implementation science theory informing the development/implementation of newly designed structured approaches in the genomic setting and a lack of measures to assess implementation effectiveness.

CONCLUSION

This scoping review identified a significant number of structured approaches developed to inform the implementation of genomic medicine into clinical practice, with limited use of implementation science to support the process. We recommend the use of existing implementation science theory and the expertise of implementation scientists to inform the design of genomic programs being implemented into clinical care.

Clinician adoption of Parent–Child Interaction Therapy: A systematic review of implementation interventions

Background

Parent–Child Interaction Therapy (PCIT) is a parent training intervention for childhood conduct problems, distinctive in its use of live clinician coaching of the parent–child dyad via a one-way mirror and discrete earpiece. However, despite a compelling evidence base, uptake of evidence-based parent training programmes such as PCIT by clinicians in routine care settings remains poor. This systematic review aimed to identify and synthesise implementation interventions that have sought to increase clinician adoption of PCIT in usual care settings.

Methods

We searched MEDLINE (Ovid), Embase (Ovid), PsycInfo (Ovid), CINAHL (EBSCO), Science Citation Index and Social Sciences Citation Index, and Web of Science Core Collection from inception to October 2020. Articles were included if they tested (by way of randomised controlled trials, controlled clinical trials, interrupted time series and controlled before and after trials) implementation interventions across any and all of the patient, clinician, clinic, system or policy domains. Two independent reviewers screened and selected studies, assessed risk of bias and extracted data – summarising implementation intervention components according to items from the Template for Intervention Description and Replication (TIDieR) checklist ( Hoffmann et al., 2014 ).

Results

Of the 769 articles identified once duplicates were removed, 13 papers relating to three studies met the inclusion criteria – all were quantitative or mixed-methods examinations of the effectiveness of different PCIT clinician training or training-related consultation methods. A narrative description of interventions was provided, as quantitative synthesis was not possible.

Conclusions

Research attention has to date been focussed on the establishment of an evidence-base for PCIT's effectiveness, with relatively little attention to the dissemination, implementation and sustainment of this treatment. Those studies that do exist have focused on training methods and training-related expert consultation. Research attention could usefully turn to both adoption and sustainment of this effective treatment in usual care settings.

Plain language summary

In this review, we aimed to summarise what is already known about how to implement PCIT in community settings after clinicians have received training in the approach. While research relating to the implementation of other parent training programmes is interesting and informative, implementation efforts are most effective when tailored to a specific programme in a specific context. As such, it was important to review published studies relating to PCIT specifically. We identified three relevant studies, one of which is yet to publish its main implementation findings. The three studies have focused on how best to train clinicians in PCIT, including how best to provide post-training support from expert trainers. We concluded that a fruitful line for future research would be to focus on the post-training period, particularly how best to support clinicians to adopt and sustain PCIT in their practice.

Systematic review registration

The study was prospectively registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 01/10/2020 (CRD42020207118).

Iowa Implementation for Sustainability Framework

Background

An application-oriented implementation framework designed for clinicians and based on the Diffusion of Innovations theory included 81 implementation strategies with suggested timing for use within four implementation phases. The purpose of this research was to evaluate and strengthen the framework for clinician use and propose its usefulness in implementation research.

Methods

A multi-step, iterative approach guided framework revisions. Individuals requesting the use of the framework over the previous 7 years were sent an electronic questionnaire. Evaluation captured framework usability, generalizability, accuracy, and implementation phases for each strategy. Next, nurse leaders who use the framework pile sorted strategies for cultural domain analysis. Last, a panel of five EBP/implementation experts used these data and built consensus to strengthen the framework.

Results

Participants (n = 127/1578; 8% response) were predominately nurses (94%), highly educated (94% Master’s or higher), and from across healthcare (52% hospital/system, 31% academia, and 7% community) in the USA (84%). Most (96%) reported at least some experience using the framework and 88% would use the framework again. A 4-point scale (1 = not/disagree to 4 = very/agree) was used. The framework was deemed useful (92%, rating 3–4), easy to use (72%), intuitive (67%), generalizable (100%), flexible and adaptive (100%), with accurate phases (96%), and accurate targets (100%). Participants (n = 51) identified implementation strategy timing within four phases (Cochran’s Q); 54 of 81 strategies (66.7%, p < 0.05) were significantly linked to a specific phase; of these, 30 (55.6%) matched the original framework. Next, nurse leaders (n = 23) completed a pile sorting activity. Anthropac software was used to analyze the data and visualize it as a domain map and hierarchical clusters with 10 domains. Lastly, experts used these data and implementation science to refine and specify each of the 75 strategies, identifying phase, domain, actors, and function. Strategy usability, timing, and groupings were used to refine the framework.

Conclusion

The Iowa Implementation for Sustainability Framework offers a typology to guide implementation for evidence-based healthcare. This study specifies 75 implementation strategies within four phases and 10 domains and begins to validate the framework. Standard use of strategy names is foundational to compare and understand when implementation strategies are effective, in what dose, for which topics, by whom, and in what context.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01157-5.

Challenges to sustainability of pediatric early warning systems (PEWS) in low-resource hospitals in Latin America

Background

Sustainability, or continued use of evidence-based interventions for long-term patient benefit, is the least studied aspect of implementation science. In this study, we evaluate sustainability of a Pediatric Early Warning System (PEWS), an evidence-based intervention to improve early identification of clinical deterioration in hospitalized children, in low-resource settings using the Clinical Capacity for Sustainability Framework (CCS).

Methods

We conducted a secondary analysis of a qualitative study to identify barriers and enablers to PEWS implementation. Semi-structured interviews with PEWS implementation leaders and hospital directors at 5 Latin American pediatric oncology centers sustaining PEWS were conducted virtually in Spanish from June to August 2020. Interviews were recorded, professionally transcribed, and translated into English. Exploratory thematic content analysis yielded staff perceptions on PEWS sustainability. Coded segments were analyzed to identify participant perception about the current state and importance of sustaining PEWS, as well as sustainability successes and challenges. Identified sustainability determinants were mapped to the CCS to evaluate its applicability.

Results

We interviewed 71 staff including physicians (45%), nurses (45%), and administrators (10%). Participants emphasized the importance of sustaining PEWS for continued patient benefits. Identified sustainability determinants included supportive leadership encouraging ongoing interest in PEWS, beneficial patient outcomes enhancing perceived value of PEWS, integrating PEWS into the routine of patient care, ongoing staff turnover creating training challenges, adequate material resources to promote PEWS use, and the COVID-19 pandemic. While most identified factors mapped to the CCS, COVID-19 emerged as an additional external sustainability challenge. Together, these challenges resulted in multiple impacts on PEWS sustainment, ranging from a small reduction in PEWS quality to complete disruption of PEWS use and subsequent loss of benefits to patients. Participants described several innovative strategies to address identified challenges and promote PEWS sustainability.

Conclusion

This study describes clinician perspectives on sustainable implementation of evidence-based interventions in low-resource settings, including sustainability determinants and potential sustainability strategies. Identified factors mapped well to the CCS, however, external factors, such as the COVID pandemic, may additionally impact sustainability. This work highlights an urgent need for theoretically-driven, empirically-informed strategies to support sustainable implementation of evidence-based interventions in settings of all resource-levels.

The effectiveness of quality improvement collaboratives in improving stroke care and the facilitators and barriers to their implementation: a systematic review

Background

To successfully reduce the negative impacts of stroke, high-quality health and care practices are needed across the entire stroke care pathway. These practices are not always shared across organisations. Quality improvement collaboratives (QICs) offer a unique opportunity for key stakeholders from different organisations to share, learn and ‘take home’ best practice examples, to support local improvement efforts. This systematic review assessed the effectiveness of QICs in improving stroke care and explored the facilitators and barriers to implementing this approach.

Methods

Five electronic databases (MEDLINE, CINAHL, EMBASE, PsycINFO, and Cochrane Library) were searched up to June 2020, and reference lists of included studies and relevant reviews were screened. Studies conducted in an adult stroke care setting, which involved multi-professional stroke teams participating in a QIC, were included. Data was extracted by one reviewer and checked by a second. For overall effectiveness, a vote-counting method was used. Data regarding facilitators and barriers was extracted and mapped to the Consolidated Framework for Implementation Research (CFIR).

Results

Twenty papers describing twelve QICs used in stroke care were included. QICs varied in their setting, part of the stroke care pathway, and their improvement focus. QIC participation was associated with improvements in clinical processes, but improvements in patient and other outcomes were limited. Key facilitators were inter- and intra-organisational networking, feedback mechanisms, leadership engagement, and access to best practice examples. Key barriers were structural changes during the QIC’s active period, lack of organisational support or prioritisation of QIC activities, and insufficient time and resources to participate in QIC activities. Patient and carer involvement, and health inequalities, were rarely considered.

Conclusions

QICs are associated with improving clinical processes in stroke care; however, their short-term nature means uncertainty remains as to whether they benefit patient outcomes. Evidence around using a QIC to achieve system-level change in stroke is equivocal. QIC implementation can be influenced by individual and organisational level factors, and future efforts to improve stroke care using a QIC should be informed by the facilitators and barriers identified. Future research is needed to explore the sustainability of improvements when QIC support is withdrawn.

Trial registration

Protocol registered on PROSPERO (CRD42020193966).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01162-8.

Increasing Access to Contraception in the United States: Assessing Achievement and Sustainability

Background: During October 2016 through May 2018, a learning community was convened to focus on policies and programs to increase access to the full range of contraceptive options for women of reproductive age. The Increasing Access to Contraception (IAC) Learning Community included 27 jurisdictions, with teams from each jurisdiction consisting of state health department leaders, program staff, and provider champions. At the kick-off meeting, teams from each jurisdiction created action plans that outlined their goals. Methods: We contacted jurisdictions during May-June 2019, 1 year after the learning community ended, and invited them to complete a post-assessment of goal achievement and sustainment through semi-structured interviews over the telephone or via email. Results: Follow-up information was collected from 26 jurisdictions (96%) that participated in the learning community. The teams from these jurisdictions had created 79 total goals. At the time of the learning community closing meeting in May 2018, 35 goals (44%) had been achieved. Three jurisdictions achieved all their goals by the close of the learning community. At the time of the post-assessment 1 year later, jurisdictions were sustaining efforts for 69 (87%) of the total goals. In every jurisdiction, work on at least one goal that originated in the learning community was sustained. Conclusions: The jurisdictions that participated in the IAC Learning Community continued the work of their action plan goals 1 year after the formal closure of the learning community, indicating sustainability of the learning community activities, beyond what jurisdictions accomplished during formal participation.

Challenges in the Implementation of Chronic Obstructive Pulmonary Disease Guidelines in Low- and Middle-Income Countries: An Official American Thoracic Society Workshop Report

There is a substantial burden of chronic respiratory diseases, including chronic obstructive pulmonary disease (COPD), in low- and middle-income countries (LMICs). LMICs have particular challenges in delivering cost-effective prevention, diagnosis, and management of COPD. Optimal care can be supported by effective implementation of guidelines. This American Thoracic Society workshop considered challenges to implementation of COPD guidelines in LMICs. We make 10 specific recommendations: 1) relevant organizations should provide LMIC-specific COPD management guidance; 2) patient and professional organizations must persuade policy-makers of the importance of lung function testing programs in LMICs; 3) healthcare education and training should emphasize the early-life origins of COPD; 4) urgent action is required by governments to reduce airborne exposures, including exposures to tobacco smoke and indoor and outdoor air pollution; 5) guidance for COPD in LMICs should explicitly link across Essential Medicine Lists and the World Health Organization package of essential noncommunicable disease interventions for primary health care in low-resource settings and should consider availability, affordability, sustainability, and cost-effective use of medicines; 6) the pharmaceutical industry should work to make effective COPD and tobacco-dependence medicines globally accessible and affordable; 7) implementation of locally adapted, cost-effective pulmonary rehabilitation programs should be an international priority; 8) the World Health Organization Global Action Plan for the Prevention and Control of Noncommunicable Diseases should specify how improvements in respiratory health will be achieved; 9) research funders should increase the proportion of funding allocated to COPD in LMICs; and 10) the respiratory community should leverage the skills and enthusiasm of earlier-career clinicians and researchers to improve global respiratory health.

An equivalence analysis of provider education in youth mental health care

OBJECTIVE

To test for equivalence between providers with and without advanced degrees in multiple domains related to delivery of evidence-based treatment.

DATA SOURCE

Provider and client data from an effectiveness trial of Alternatives for Families: A Cognitive Behavioral Therapy (AF-CBT) in a major metropolitan area in the United States.

STUDY DESIGN

We tested for equivalence between providers (N = 182) with and without advanced degrees in treatment-related knowledge, practices, and attitudes; job demands and stress; and training engagement and trainer-rated competence in AF-CBT. We also conducted exploratory analyses to test for equivalence in family clinical outcomes.

DATA COLLECTION

Providers completed measures prior to randomization and at 6-month follow-up, after completion of training and consultation in AF-CBT. Children and caregivers completed assessments at 0, 6, 12, and 18 months.

PRINCIPAL FINDINGS

Providers without advanced degrees were largely non-inferior to those with advanced degrees in treatment-related knowledge, practices, and attitudes, while findings for job demands and stress were mixed. Providers without advanced degrees were non-inferior to providers with advanced degrees in consultation attendance (B = -1.42; confidence interval (CI) = -3.01-0.16; margin of equivalence (Δ) = 2), number of case presentations (B = 0.64; CI = -0.49-1.76; Δ = 2), total training hours (B = -4.57; CI = -10.52-1.37; Δ = 3), and trainer-rated competence in AF-CBT (B = -0.04; CI = -3.04-2.96; Δ = 4), and they were significantly more likely to complete training (odds ratio = 0.66; CI = 0.10-0.96; Δ = 30%). Results for clinical outcomes were largely inconclusive.

CONCLUSIONS

Provider-level outcomes for those with and without advanced degrees were generally comparable. Additional research is needed to examine equivalence in clinical outcomes. Expanding evidence-based treatment training to individuals without advanced degrees may help to reduce workforce shortages and improve reach of evidence-based treatments.