The dynamic influence of human resources on evidence-based intervention sustainability and population outcomes: an agent-based modeling approach

Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool

BACKGROUND

Available resources within an organization can determine the implementation success of an intervention. However, few studies have investigated how the required resources change over the phases of implementation. Using stakeholder interviews, we examined the changes in and interactions between available resources and implementation climate in the implementation and sustainment phases of a national implementation effort for a population health tool.

METHODS

We conducted a secondary analysis of the interviews with 20 anticoagulation professionals at 17 clinical sites in the Veterans Health Administration health system about their experiences with a population health dashboard for anticoagulant management. Interview transcripts were coded using constructs from the Consolidated Framework for Implementation Research (CFIR) and according to the phase of implementation (pre-implementation, implementation, and sustainment) as defined by the VA Quality Enhancement Research Initiative (QUERI) Roadmap. We analyzed the factors that may determine successful implementation by examining the co-occurrence patterns between available resources and implementation climate across different implementation phases. To illustrate the variations in these determinants across phases, we aggregated and scored coded statements using a previously published CFIR scoring system (- 2 to + 2). Key relationships between available resources and implementation climate were identified and summarized using thematic analysis.

RESULTS

The resources necessary to support the successful implementation of an intervention are not static; both the quantity and types of resources shift based on the phases of the intervention. Furthermore, increased resource availability does not guarantee the sustainment of intervention success. Users need different types of support beyond the technical aspects of an intervention, and this support varies over time. Specifically, available resources in the form of technological support and social/emotional support help users establish trust in a new technological-based intervention during the implementation phase. Resources that foster and maintain collaboration between users and other stakeholders help them stay motivated during sustainment.

CONCLUSIONS

Our findings highlight the dynamic nature of available resources and their impacts on the implementation climate across different phases of implementation. A better understanding of the dynamics of available resources over time from the users' perspectives will allow the adaptation of resources to better meet the needs of the intervention stakeholders.

Child and adolescent mental health services in a devolved healthcare system: a qualitative exploration of sustainable practices

BACKGROUND

The transference of research evidence into routine healthcare practice remains poorly understood. This includes understanding the prerequisites of longer-term viability. The present study investigated the sustainable practices of GM i-THRIVE, a programme which reconceptualizes mental health services for children and young people (CYP) in Greater Manchester, United Kingdom. We aimed to establish whether a sustainable future was likely, and to identify areas of focus to improve that likelihood.

METHODS

The NHS Sustainability Model, typically completed as a questionnaire measure, was converted into interview questions. The responses of nine professionals, from a variety of roles across the CYP mental health workforce, were explored using inductive thematic framework analysis. Selected participants completed the original questionnaire.

RESULTS

Five themes (communication; support; barriers to implementation; past, present, and future: the implementation journey; and the nuances of GM i-THRIVE) and 21 subthemes formed the final thematic framework. Relationships with senior leaders and with colleagues across the workforce were seen as important. Leaders' roles in providing meaning and fit were emphasized. Whilst training delivered the programme's aims well, monitoring its dissemination was challenging. Widespread issues with dedicating sufficient time to implementation were raised. The flexibility of the programme, which can be applied in multiple ways, was discussed positively. This flexibility links to the idea of GM i-THRIVE as a mindset change, and the uniqueness of this style of intervention was discussed. To varying degrees, themes were supported by responses to the quantitative measure, although several limitations to the use of the questionnaire were discovered. Consequently, they were used to infer conclusions to a lesser degree than originally intended.

CONCLUSIONS

Professionals involved with GM i-THRIVE reported many elements that indicate a positive future for the programme. However, they suggested that more attention should be given to embedding the core concepts of the model at the current stage of implementation. Limitations relating to its use within our study are discussed, but we conclude that the NHS Sustainability Model is a suitable way of guiding qualitative implementation research. It is especially valuable for localized interventions. The constraints of our small sample size on transferability are considered.

Understanding Misimplementation in U.S. State Health Departments: An Agent-Based Model

INTRODUCTION

The research goal of this study is to explore why misimplementation occurs in public health agencies and how it can be reduced. Misimplementation is ending effective activities prematurely or continuing ineffective ones, which contributes to wasted resources and suboptimal health outcomes.

METHODS

The study team created an agent-based model that represents how information flow, filtered through organizational structure, capacity, culture, and leadership priorities, shapes continuation decisions. This agent-based model used survey data and interviews with state health department personnel across the U.S. between 2014 and 2020; model design and analyses were conducted with substantial input from stakeholders between 2019 and 2021. The model was used experimentally to identify potential approaches for reducing misimplementation.

RESULTS

Simulations showed that increasing either organizational evidence-based decision-making capacity or information sharing could reduce misimplementation. Shifting leadership priorities to emphasize effectiveness resulted in the largest reduction, whereas organizational restructuring did not reduce misimplementation.

CONCLUSIONS

The model identifies for the first time a specific set of factors and dynamic pathways most likely driving misimplementation and suggests a number of actionable strategies for reducing it. Priorities for training the public health workforce include evidence-based decision making and effective communication. Organizations will also benefit from an intentional shift in leadership decision-making processes. On the basis of this initial, successful application of agent-based model to misimplementation, this work provides a framework for further analyses.

Utilising community volunteers can increase the detection and referral of Buruli ulcer cases in endemic communities in Southeast, Nigeria

BACKGROUND

Buruli ulcer (BU) is a debilitating neglected tropical disease which causes disability and mostly affects inhabitants in impoverished settings where access to medical care is challenging. This study aims to determine the effect of training community members as volunteers for or in the detection and referral of people who have Buruli ulcer to the hospital.

METHODS

The following study is a before and after study in the BU-endemic Local Government Areas (LGA) of Ebonyi State. A cluster random sampling technique was used to select 90 volunteers from three LGAs (30 from each LGA). In each LGA, the volunteers underwent a one-day training and six months field work to identify all those who have any form of ulcer on any part of their bodies. A short questionnaire was used to capture socio-demographic characteristics of the patient, site of the ulcer, duration of the ulcer, initial appearance of the ulcer, referral to hospital, result of laboratory investigation, and treatment received. The data was analysed using the Statistical Package for Social Sciences (SPSS) for Microsoft Windows version 20 software. The Z test statistic was used to compare the number of referred BU patients before and after the intervention by LGA. The Chi square test was used to examine the association between the dependent and independent variables.

RESULTS

The mean age of volunteers was 39 ± 9.5 while mean age of the patients was 42.3 ± 17.1. Most of the ulcers were on the legs (79.4%) and lasted 1-5 years (65.6%). There was a significant increase in the proportion of BU suspects identified by the community volunteers in all 3 LGAs (Afikpo north (p =  < 0.001), Abakaliki (p = 0.02), Ikwo (p = 0.001). The duration of the ulcer was associated with the detection and referral of the patients with higher levels of detection and referral among those whose ulcer had lasted 1-5 years in two of the LGAs (P < 0.001).

CONCLUSION

We recommend that program managers and stakeholders integrate and scale up the services of trained community health volunteers for the rapid detection of Buruli ulcer cases in rural endemic communities. Awareness and sensitization campaigns on BU preventive measures should be intensified.

Exploring the impact of workforce turnover on the sustainability of evidence-based programs: A scoping review

Background

Evidence-based programs (EBPs) are used across disciplines to integrate research into practice and improve outcomes at the individual and/or community level. Despite widespread development and implementation of EBPs, many programs are not sustained beyond the initial implementation period due to many factors, including workforce turnover. This scoping review summarizes research on the impact of workforce turnover on the sustainability of EBPs and recommendations for mitigating these impacts.

Methods

We searched 10 databases for articles that focused on an EBP and described an association between workforce turnover and the sustainment or sustainability of the program. We created a data abstraction tool to extract relevant information from each article and applied the data abstraction tool to all included articles to create the dataset. Data were mapped and analyzed using the program sustainability framework (PSF).

Results and Discussion

A total of 30 articles were included in this scoping review and mapped to the PSF. Twenty-nine articles described impacts of workforce turnover and 18 articles proposed recommendations to address the impacts. The most frequent impacts of workforce turnover included increased need for training, loss of organizational knowledge, lack of EBP fidelity, and financial stress. Recommendations to address the impact of workforce turnover included affordable and alternative training modalities, the use of champions or volunteers, increasing program alignment with organizational goals, and generating diverse funding portfolios.

Conclusion

The sustainment of EBPs is critical to ensure and maintain the short- and long-term benefits of the EBP for all participants and communities. Understanding the impacts of workforce turnover, a determinant of sustainability, can create awareness among EBP-implementing organizations and allow for proactive planning to increase the likelihood of program sustainability.

Agent-Based Modeling for Implementation Research: An Application to Tobacco Smoking Cessation for Persons with Serious Mental Illness

Background

Implementation researchers have sought ways to use simulations to support the core components of implementation, which typically include assessing the need for change, designing implementation strategies, executing the strategies, and evaluating outcomes. The goal of this paper is to explain how agent-based modeling could fulfill this role.

Methods

We describe agent-based modeling with respect to other simulation methods that have been used in implementation science, using non-technical language that is broadly accessible. We then provide a stepwise procedure for developing agent-based models of implementation processes. We use, as a case study to illustrate the procedure, the implementation of evidence-based smoking cessation practices for persons with serious mental illness (SMI) in community mental health clinics.

Results

For our case study, we present descriptions of the motivating research questions, specific models used to answer these questions, and a summary of the insights that can be obtained from the models. In the first example, we use a simple form of agent-based modeling to simulate the observed smoking behaviors of persons with SMI in a recently completed trial (IDEAL, Comprehensive Cardiovascular Risk Reduction Trial in Persons with SMI). In the second example, we illustrate how a more complex agent-based approach that includes interactions between patients, providers and site administrators can be used to provide guidance for an implementation intervention that includes training and organizational strategies. This example is based in part on an ongoing project focused on scaling up evidence-based tobacco smoking cessation practices in community mental health clinics in Maryland.

Conclusion

In this paper we explain how agent-based models can be used to address implementation science research questions and provide a procedure for setting up simulation models. Through our examples, we show how what-if scenarios can be examined in the implementation process, which are particularly useful in implementation frameworks with adaptive components.

Building resource constraints and feasibility considerations in mathematical models for infectious disease: A systematic literature review

•

Mathematical model capabilities to explore complex systems now enable priority-setting to consider local resource constraints.

•

Common objectives of model-based analyses incorporating constraints are to assess real-world feasibility or allocate resources efficiently.

•

Constraints may be incorporated via (i) model-based estimation; (ii) linkage of mathematical and health system models; or (iii) optimisation.

•

Models can then project constrained intervention effects and costs and resource requirement s for delivering interventions at full scale.

•

'Health system constraints' should be systematically defined for routine operationalisation in model-based priority-setting.

Priority setting for infectious disease control is increasingly concerned with physical input constraints and other real-world restrictions on implementation and on the decision process. These health system constraints determine the ‘feasibility’ of interventions and hence impact. However, considering them within mathematical models places additional demands on model structure and relies on data availability. This review aims to provide an overview of published methods for considering constraints in mathematical models of infectious disease.

We systematically searched the literature to identify studies employing dynamic transmission models to assess interventions in any infectious disease and geographical area that included non-financial constraints to implementation. Information was extracted on the types of constraints considered and how these were identified and characterised, as well as on the model structures and techniques for incorporating the constraints.

A total of 36 studies were retained for analysis. While most dynamic transmission models identified were deterministic compartmental models, stochastic models and agent-based simulations were also successfully used for assessing the effects of non-financial constraints on priority setting. Studies aimed to assess reductions in intervention coverage (and programme costs) as a result of constraints preventing successful roll-out and scale-up, and/or to calculate costs and resources needed to relax these constraints and achieve desired coverage levels. We identified three approaches for incorporating constraints within the analyses: (i) estimation within the disease transmission model; (ii) linking disease transmission and health system models; (iii) optimising under constraints (other than the budget).

The review highlighted the viability of expanding model-based priority setting to consider health system constraints. We show strengths and limitations in current approaches to identify and quantify locally-relevant constraints, ranging from simple assumptions to structured elicitation and operational models. Overall, there is a clear need for transparency in the way feasibility is defined as a decision criteria for its systematic operationalisation within models.

Using a Modeling-Based Approach to Assess and Optimize HIV Linkage to Care Services

Evidence-based linkage to care interventions (LTCs) help recently diagnosed HIV+ individuals engage in care in a timely manner yet are heavily impacted by the systems in which they are embedded. We developed a prototype agent-based model informed by data from an established LTC program targeting youth and young adults aged 13-24 in Memphis, Tennessee. We then tested two interventions to improve LTC in a simulated environment: expanding testing sites versus using current testing sites but improving direct referral to LTC staff from organizations providing testing, to understand the impact on timely linkage to care. Improving direct referral to the LTC program decreased days to successful linkage from an average of 30 to 23 days but expanding testing sites increased average days to 31 days unless those sites also made direct referrals. We demonstrated how LTC is impacted by the system and interventions for shortening days to linkage to care.