Patterns and correlates of mis-implementation in state chronic disease public health practice in the United States

Implementing group care in Dutch and Surinamese maternity and child care services: the vital importance of addressing outer context barriers

BACKGROUND

By addressing physical and psychosocial needs, group care (GC) improves health-related behaviours, peer support, parent-provider interactions and may improve birth outcomes. Hence, global implementation of GC is encouraged. Context analyses prior to implementation are vital to elucidate which local factors may support or hinder implementation.

METHODS

Contextual analyses conducted in the Netherlands and Suriname were compared to identify the factors relevant to the implementability of GC as perceived by healthcare professionals (HCPs). 32 semi-structured interviews were conducted with Dutch and Surinamese healthcare professionals. Audio recordings were transcribed verbatim and coded using the Framework approach. The Consolidated Framework for Implementation Research guided the development of the interview guide and of the coding tree.

RESULTS

Outer setting: Concerns regarding funding surfaced in both countries. Due to limited health insurance coverage, additional fees would limit accessibility in Suriname. In the Netherlands, midwives dreaded lower revenue due to reimbursement policies that favour one-on-one care. Inner setting: Appropriate space for GC was absent in one Dutch and three Surinamese facilities. Role division regarding GC implementation was clearer in the Netherlands than in Suriname.

INNOVATION

HCPs from both countries expected increased social support, health knowledge among women, and continuity of care(r). Individuals/innovation deliverers: Self-efficacy and motivation emerged as intertwined determinants to GC implementation in both countries. Individuals/innovation recipients: Competing demands can potentially lower acceptability of GC in both countries. While Dutch HCPs prioritised an open dialogue with mothers, Surinamese HCPs encouraged the inclusion of partners.

PROCESS

Campaigns to raise awareness of GC were proposed. Language barriers were a concern for Dutch but not for Surinamese HCPs.

CONCLUSIONS

While the most striking differences between both countries were found in the outer setting, they trickle down and affect all layers of context. Ultimately, at a later stage, the process evaluation will show if those outer setting barriers we identified prior to implementation actually hindered GC implementation. Changes to the health care systems would ensure sustained implementation in both countries, and this conclusion feeds into a more general discussion: how to proceed when contextual analyses reveal barriers that cannot be addressed with the time and resources available.

Barriers and Facilitators to Program Sustainability Among State Tobacco Control Programs

Public health programs, particularly tobacco control programs (TCPs) in state health departments, face numerous barriers and facilitators to sustainability, which affect delivery and, consequently, health outcomes achieved. We used the Program Sustainability Framework to review and analyze qualitative interview data from states that received training and technical assistance during the Plans, Actions, and Capacity to Sustain Tobacco Control (PACT) study to better understand the barriers and facilitators to sustainability capacity that these public health programs face at the state level. The PACT study was a multiyear, randomized controlled trial to assess the effectiveness of an action planning workshop and technical assistance in improving capacity for sustainability among 11 intervention and 12 control TCPs. Technical assistance calls focused on the progress and barriers of implementing the sustainability action plan created during the in-person workshops. Calls were audio recorded and professionally transcribed. Thematic analysis focused on the codes describing barriers and facilitators faced by TCPs in increasing their capacity for sustainability. Barriers were reported in the Organization Capacity, Environmental Support, Partnerships, Communication, and Funding Stability domains of the Program Sustainability Framework. Facilitators to action planning and building capacity for program sustainability were primarily in the Strategic Planning, Program Evaluation, Program Adaptation, and Partnership domains. Our study is the first to identify barriers and facilitators to increasing the capacity of program sustainability in TCPs. This work advances the understanding of program sustainability capacity and technical assistance for public health programs.

Understanding Misimplementation in U.S. State Health Departments: An Agent-Based Model

INTRODUCTION

The research goal of this study is to explore why misimplementation occurs in public health agencies and how it can be reduced. Misimplementation is ending effective activities prematurely or continuing ineffective ones, which contributes to wasted resources and suboptimal health outcomes.

METHODS

The study team created an agent-based model that represents how information flow, filtered through organizational structure, capacity, culture, and leadership priorities, shapes continuation decisions. This agent-based model used survey data and interviews with state health department personnel across the U.S. between 2014 and 2020; model design and analyses were conducted with substantial input from stakeholders between 2019 and 2021. The model was used experimentally to identify potential approaches for reducing misimplementation.

RESULTS

Simulations showed that increasing either organizational evidence-based decision-making capacity or information sharing could reduce misimplementation. Shifting leadership priorities to emphasize effectiveness resulted in the largest reduction, whereas organizational restructuring did not reduce misimplementation.

CONCLUSIONS

The model identifies for the first time a specific set of factors and dynamic pathways most likely driving misimplementation and suggests a number of actionable strategies for reducing it. Priorities for training the public health workforce include evidence-based decision making and effective communication. Organizations will also benefit from an intentional shift in leadership decision-making processes. On the basis of this initial, successful application of agent-based model to misimplementation, this work provides a framework for further analyses.

Barriers and Facilitators in the Implementation of Prevention Strategies for Chronic Disease Patients-Best Practice GuideLines and Policies' Systematic Review

Visits of chronically ill patients account for 80% of primary care consultations. Approximately 15-38% of patients have three or more chronic diseases, and 30% of hospitalisations result from the deteriorating clinical condition of these patients. The burden of chronic disease and multimorbidity is increasing in combination with the growing population of elderly people. However, many interventions found to be effective in health service studies fail to translate into meaningful patient care outcomes across multiple contexts. With the growing burden of chronic diseases, healthcare providers, health policymakers, and other healthcare system stakeholders are re-examining their strategies and opportunities for more effective prevention and clinical interventions. The study aimed to find the best practice guidelines and policies influencing effective intervention and making it possible to personalize prevention strategies. Apart from clinical treatment, it is essential to increase the effectiveness of non-clinical interventions that could empower chronic patients to increase their involvement in therapy. The review focuses on the best practice guidelines and policies in non-medical interventions and the barriers to and facilitators of their implementation into everyday practice. A systematic review of practice guidelines and policies was conducted to answer the research question. The authors screened databases and included 47 full-text recent studies in the qualitative synthesis.

Methodological approaches to study context in intervention implementation studies: an evidence gap map

BACKGROUND

Within implementation science studies, contextual analysis is increasingly recognized as foundational to interventions' successful and sustainable implementation. However, inconsistencies between methodological approaches currently limit progress in studying context and guidance to standardize the use of those approaches is scant. Therefore, this study's objective was to systematically review and map current methodological approaches to contextual analysis in intervention implementation studies. The results would help us both to systematize the process of contextual analysis and identify gaps in the current evidence.

METHODS

We conducted an evidence gap map (EGM) based on literature data via a stepwise approach. First, using an empirically developed search string, we randomly sampled 20% of all intervention implementation studies available from PubMed per year (2015-2020). Second, we assessed included studies that conducted a contextual analysis. Data extraction and evaluation followed the Basel Approach for CoNtextual ANAlysis (BANANA), using a color-coded rating scheme. Also based on BANANA and on the Context and Implementation of Complex Interventions (CICI) framework-an implementation framework that pays ample attention to context- we created visual maps of various approaches to contextual analysis.

RESULTS

Of 15, 286 identified intervention implementation studies and study protocols, 3017 were screened for inclusion. Of those, 110 warranted close examination, revealing 22% that reported on contextual analysis. Only one study explicitly applied a framework for contextual analysis. Data were most commonly collected via surveys (n = 15) and individual interviews (n = 13). Ten studies reported mixed-methods analyses. Twenty-two assessed meso-level contextual and setting factors, with socio-cultural aspects most commonly studied. Eighteen described the use of contextual information for subsequent project phases (e.g., intervention development/adaption, selecting implementation strategies). Nine reported contextual factors' influences on implementation and/or effectiveness outcomes.

CONCLUSIONS

This study describes current approaches to contextual analysis in implementation science and provides a novel framework for evaluating and mapping it. By synthesizing our findings graphically in figures, we provide an initial evidence base framework that can incorporate new findings as necessary. We strongly recommend further development of methodological approaches both to conduct contextual analysis and to systematize the reporting of it. These actions will increase the quality and consistency of implementation science research.

Leading the way: competencies of leadership to prevent mis-implementation of public health programs

Public health agencies are increasingly concerned with ensuring that they are maximizing limited resources by delivering effective programs to enhance population-level health outcomes. Preventing mis-implementation (ending effective activities prematurely or continuing ineffective ones) is necessary to sustain public health efforts and resources needed to improve health and well-being. The purpose of this paper is to identify the important qualities of leadership in preventing mis-implementation of public health programs. In 2019, 45 state health department chronic disease employees were interviewed via phone and audio-recorded, and the conversations were transcribed verbatim. Thematic analysis focused on items related to mis-implementation and the manners in which leadership were involved in continuing ineffective programs. Final themes were based on a Public Health Leadership Competency Framework. The following themes emerged from their interviews regarding the important leadership competencies to prevent mis-implementation: '(1) leadership and communication; (2) collaborative leadership (3) leadership to adapt programs; (4) leadership and organizational learning and development; and (5) political leadership'. This first of its kind study showed the close interrelationship between mis-implementation and leadership. Increased attention to public health leader competencies might help to reduce mis-implementation in public health practice and lead to more effective and efficient use of limited resources.

Program adaptation by health departments

Introduction

The dissemination of evidence-based interventions (i.e., programs, practices, and policies) is a core function of US state health departments (SHDs). However, interventions are originally designed and tested with a specific population and context. Hence, adapting the intervention to meet the real-world circumstances and population's needs can increase the likelihood of achieving the expected health outcomes for the target population from the implemented intervention. This study identified how SHD employees decide to adapt public health programs and what influences decisions on how to adapt them.

Materials and methods

SHD employees (n = 45) were interviewed using a qualitative semi-structured interview guide. Telephone interviews were audio-recorded and transcribed verbatim. The transcripts were consensus-coded and themes were identified using thematic analysis. Several themes aligned with the Model for Adaptation Design and Impact.

Results

Data, outcomes, and health department evaluations influenced decisions to adapt a program (pre-adaptation), and reasons to adapt a program included organizational and sociopolitical contextual factors. SHD middle-level managers, program managers and staff, and local agencies were involved in the decisions to adapt the programs. Finally, the goals for adapting a program included enhancing effectiveness/outcomes, reach and satisfaction with the program; funding; and partner engagement. After SHD employees decided to adapt a program, data and evidence guided the changes. Program staff and evaluators were engaged in the adaptation process. Program managers consulted partners to gather ideas on how best to adapt a program based on partners' experiences implementing the program and obtaining community input. Lastly, program managers also received input on adapting content and context from coalition meetings and periodic technical assistance calls.

Discussion

The findings related to decisions to adapt public health programs provide practitioners with considerations for adapting them. Findings reaffirm the importance of promoting public health competencies in program evaluation and adaptation, as well as systematically documenting and evaluating the adaptation processes. In addition, the themes could be studied in future research as mechanisms, mediators, and moderators to implementation outcomes.

"It's good to feel like you're doing something": a qualitative study examining state health department employees' views on why ineffective programs continue to be implemented in the USA

Background

Mis-implementation, the inappropriate continuation of programs or policies that are not evidence-based or the inappropriate termination of evidence-based programs and policies, can lead to the inefficient use of scarce resources in public health agencies and decrease the ability of these agencies to deliver effective programs and improve population health. Little is known about why mis-implementation occurs, which is needed to understand how to address it. This study sought to understand the state health department practitioners’ perspectives about what makes programs ineffective and the reasons why ineffective programs continue.

Methods

Eight state health departments (SHDs) were selected to participate in telephone-administered qualitative interviews about decision-making around ending or continuing programs. States were selected based on geographic representation and on their level of mis-implementation (low and high) categorized from our previous national survey. Forty-four SHD chronic disease staff participated in interviews, which were audio-recorded and transcribed verbatim. Transcripts were consensus coded, and themes were identified and summarized. This paper presents two sets of themes, related to (1) what makes a program ineffective and (2) why ineffective programs continue to be implemented according to SHD staff.

Results

Participants considered programs ineffective if they were not evidence-based or if they did not fit well within the population; could not be implemented well due to program restraints or a lack of staff time and resources; did not reach those who could most benefit from the program; or did not show the expected program outcomes through evaluation. Practitioners described several reasons why ineffective programs continued to be implemented, including concerns about damaging the relationships with partner organizations, the presence of program champions, agency capacity, and funding restrictions.

Conclusions

The continued implementation of ineffective programs occurs due to a number of interrelated organizational, relational, human resources, and economic factors. Efforts should focus on preventing mis-implementation since it limits public health agencies’ ability to conduct evidence-based public health, implement evidence-based programs effectively, and reduce the high burden of chronic diseases. The use of evidence-based decision-making in public health agencies and supporting adaptation of programs to improve their fit may prevent mis-implementation. Future work should identify effective strategies to reduce mis-implementation, which can optimize public health practice and improve population health.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-021-00252-4.

Approaches for Ending Ineffective Programs: Strategies From State Public Health Practitioners

Background: Public health agencies are increasingly concerned with ensuring they are maximizing limited resources by delivering evidence-based programs to enhance population-level chronic disease outcomes. Yet, there is little guidance on how to end ineffective programs that continue in communities. The purpose of this analysis is to identify what strategies public health practitioners perceive to be effective in de-implementing, or reducing the use of, ineffective programs. Methods: From March to July 2019, eight states were selected to participate in qualitative interviews from our previous national survey of US state health department (SHD) chronic disease practitioners on program decision making. This analysis examined responses to a question about "…advice for others who want to end an ineffective program." Forty-five SHD employees were interviewed via phone. Interviews were audio-recorded, and the conversations were transcribed verbatim. All transcripts were consensus coded, and themes were identified and summarized. Results: Participants were program managers or section directors who had on average worked 11 years at their agency and 15 years in public health. SHD employees provided several strategies they perceived as effective for de-implementation. The major themes were: (1) collect and rely on evaluation data; (2) consider if any of the programs can be saved; (3) transparently communicate and discuss program adjustments; (4) be tactful and respectful of partner relationships; (5) communicate in a way that is meaningful to your audience. Conclusions: This analysis provides insight into how experienced SHD practitioners recommend ending ineffective programs which may be useful for others working at public health agencies. As de-implementation research is limited in public health settings, this work provides a guiding point for future researchers to systematically assess these strategies and their effects on public health programming.