Measuring sustainment of prevention programs and initiatives: a study protocol

Inequity of Access: Scoping the Barriers to Assisted Reproductive Technologies

Infertility impacts millions of people of reproductive age worldwide, with approximately 10-15% of couples affected. When infertility is present, there are many potential barriers to treatment, leading to inequity of access. Assisted reproductive technologies (ART) are the mainstay of medical treatment for infertility and include procedures such as in vitro fertilisation. This scoping review aims to explore the barriers to accessing assisted reproductive technologies to highlight a potential role for the pharmacist in addressing these barriers. Five databases, including CINAHL, Emcare, Medline, Scopus, and Web of Science, were searched using keywords that resulted in 19 studies that explored barriers to initially accessing or continuing ART. Studies identified more than one barrier to accessing ART, with the most mentioned barrier being the geographic location of the patient, with others themed as psychological, financial, minority groups, educational level, and the age of the patient. Recommendations were made to address barriers to accessing ART, which included changes to government regulations to increase health education and promotion of infertility. Pharmacists' accessibility, even in geographically remote locations, places them in an ideal position to address many of the challenges experienced by people accessing infertility treatment to improve outcomes for these people.

Results from an effectiveness-implementation evaluation of a postpartum depression prevention intervention delivered in home visiting programs

BACKGROUND

Building on growing research examining lay health professionals delivering postpartum depression preventive interventions, we conducted a hybrid effectiveness-implementation Type 2 trial to examine implementation metrics and determine whether pregnant women receiving Mothers and Babies 1-on-1 delivered by lay home visitors exhibit greater reductions in depressive symptoms and perceived stress than women receiving usual home visiting.

METHODS

1229 (672 control, 557 intervention) pregnant women were enrolled, with intervention participants receiving Mothers and Babies 1-on-1 delivered by a lay home visitor and control participants receiving usual home visiting services. Baseline and six-month follow-up assessments measured client mental health outcomes, with management information system data collected to assess intervention dosage. Surveys were administered nine months post-training to agency managers as well as home visitors who delivered any intervention content.

RESULTS

Intent-to-treat analyses indicated a significant reduction in perceived stress among intervention participants compared to controls, while as-treated analyses showed significant reductions in perceived stress and depressive symptoms. Although all study sites adopted the intervention, <50 % of eligible women received the intervention. Over two-thirds of home visitors made at least one fidelity-consistent adaptation, with client recruitment and retention in home visiting highlighted as challenges to delivery.

LIMITATIONS

Unmeasured historical events may have affected study outcomes and caution should be used generalizing to perinatal women from different racial/ethnic groups and home visiting models.

CONCLUSION

Mothers and Babies 1-on-1 delivered by lay home visitors leads to reductions in perceived stress and depressive symptoms, suggesting task shifting to non-mental health professionals is viable when appropriate training and supervision is provided.

A Review of the Prevention of Drug Addiction: Specific Interventions, Effectiveness, and Important Topics

BACKGROUND

This study was a narrative review on prevention of addictive behaviors with an effort to clarify the major concepts of prevention and its dimensions.

METHODS

The PsycInfo and MedInfo databases were used to identify prevention. Α total of 65 articles were found 46 of which were included in the review. Τhe data were analyzed using content analysis.

FINDINGS

For a better understanding of the results, it was deemed advisable to categorize them into: (a) the types of major interventions and the theoretical approaches on which they were based, (b) the results of the review articles according to (1) variables and factors concerning the prevention proved important through the review and (2) a number of more general important topics which arose in the prevention of addictive behaviors.

CONCLUSION

The results highlighted the greater effectiveness of community-based programs, applied primarily in broader contexts but also secondarily in high-risk groups. Τhe results were discussed based on the combination of the emerged factors with each other and prevention experience.

Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial

OBJECTIVE

Integrated treatment services are the gold standard for addressing co-occurring mental and substance use disorders, yet they are not readily available. The Network for the Improvement of Addiction Treatment (NIATx) was hypothesized to be an effective strategy to implement and sustain integrated mental health and substance use care in addiction treatment programs. This study examined sustainment of integrated services for up to 2 years after the active implementation phase.

METHODS

The effectiveness of NIATx strategies to implement and sustain integrated services was evaluated by using a cluster-randomized, waitlist control group design. Forty-nine addiction treatment organizations were randomly assigned to either NIATx1 (active implementation strategy) or NIATx2 (waitlist control). The Dual Diagnosis Capability in Addiction Treatment Index was used to evaluate organizations' capability to provide integrated care. The NIATx Stages of Implementation Completion scale was used to assess participation in and adherence to the NIATx implementation process. Linear mixed-effects modeling was used to evaluate changes from baseline to end of the sustainment period.

RESULTS

Both cohorts sustained their capability to provide integrated treatment services. Both groups achieved successful implementation and sustained integrated services to a similar degree, regardless of sustainment year. Sustainment did not vary as a function of NIATx adherence.

CONCLUSIONS

The delivery of integrated treatment services was sustained for 2 years after receipt of active implementation support. Future research should consider how contextual factors may predict, mediate, and moderate sustainment outcomes.

How do Lay Health Worker Engage Caregivers? A Qualitative Study to Enhance Equity in Evidence-Based Parenting Programs

Engaging caregivers in their children's mental health treatment is critical for delivering high quality, evidence-based care, particularly for young children with externalizing behaviors. Lay health workers (LHWs), including peer providers and promotoras de salud, have been identified as important workforces in addressing structural and stigma-related barriers to engagement in mental health services. Importantly, research has suggested that LHWs may be integral in efforts to address engagement disparities in evidence-based behavioral parent training programs (BPTs) for Latinx caregivers. The purpose of the study was to understand how different LHW workforces engage caregivers within their usual services, in order to inform strategies that improve access to and engagement in BPTs. Qualitative interviews were conducted with two different LHW workforces: volunteer LHWs (i.e., promotoras de salud) (n = 14), who were part of a community-embedded network, and paid LHWs (i.e., parent support partners, home visitors) (n = 9) embedded within children's mental health agencies. Participants were predominately Latinx (79%) and female (96%). Qualitative analyses revealed three primary themes related to engagement strategies used by LHWs to address barriers to care: 1.) Building Trust, 2.) Empowerment, 3.) Increasing Access. Although the majority of themes and sub-themes were consistent across the two LHW workforces, agency-embedded LHWs often discussed having the means to provide resources through their organizations, whereas community-embedded LHWs discussed acting as a bridge to services by providing information and conducting outreach. Findings have implications for partnering with different workforces of LHWs to increase equity in access to BPTs.

A framework for the implementation of certification procedures in nurse level: a mixed approach study

Background

The implementation of certification procedures across healthcare systems is an essential component of the management process. Several promising approaches were developed toward a successful implementation of such policies; however, a precise adaptation and implementation to each local context was essential. Local activities must be considered in order to generate more pragmatic recommendations for managers. In this study, we built a framework for the implementation of certification procedures at nurse activity level. This was developed using two objectives: the identification of key implementation process components, and the integration of these components into a framework which considered the local socio-material context of nurses’ work.

Methods

We used a two-step mixed approach. The first was inductive and consisted of a qualitative case study conducted between April and December 2019. Here, we analyzed the implementation of certification procedures in a French teaching hospital. Data were collected using semi-structured interviews and observations. In the second approach, emerging data were deductively analyzed using the Quality Implementation Tool (QIT) and Translational Mobilization Theory (TMT). Analyses were combined to construct an implementation framework.

Results

Sixteen interviews were conducted with participants from different organizational levels, managers, mid-managers, and nurses. Additionally, 83 observational hours were carried out in two different wards. Our results showed that, (1) All retrieved elements during the process were successfully captured by the QIT components, only one component was not applicable. (2) We identified elements related to the local activity context, with the different interrelationships between actors, actions, and contexts using the TMT. (3) Our analyses were integrated and translated into a framework that presents the implementation of certification procedures in healthcare facilities, with a specific interest to the nurse/mid-manager level. By initially using QIT, the framework components took on a transversal aspect which were then adapted by TMT to the local work context.

Conclusions

We successfully generated a framework that supports the implementation of certification procedures at the activity level. Our approach identified a broader vision of the interactions between proximity managers, teams, and contexts during change mobilization, which were not encompassed by transversal framework only, such as QIT. In the future, more empirical studies are needed to test this framework.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06940-0.

Train-to-Sustain: Predictors of Sustainment in a Large-Scale Implementation of Parent-Child Interaction Therapy

Sustainment of evidence-based practices is necessary to ensure their public health impact. The current study examined predictors of sustainment of Parent-Child Interaction Therapy (PCIT) within a large-scale system-driven implementation effort in Los Angeles County. Data were drawn from PCIT training data and county administrative claims between January 2013 and March 2018. Participants included 241 therapists from 61 programs. Two sustainment outcomes were examined at the therapist- and program-levels: 1) PCIT claim volume and 2) PCIT claim discontinuation (discontinuation of claims during study period; survival time of claiming in months). Predictors included therapist- and program-level caseload, training, and workforce characteristics. On average, therapists and programs continued claiming to PCIT for 17.7 and 32.3 months, respectively. Across the sustainment outcomes, there were both shared and unshared significant predictors. For therapists, case-mix fit (higher proportions of young child clients with externalizing disorders) and participation in additional PCIT training activities significantly predicted claims volume. Furthermore, additional training activity participation was associated with lower likelihood of therapist PCIT claim discontinuation in the follow-up period. Programs with therapists eligible to be internal trainers were significantly less likely to discontinue PCIT claiming. Findings suggest that PCIT sustainment may be facilitated by implementation strategies including targeted outreach to ensure eligible families in therapist caseloads, facilitating therapist engagement in advanced trainings, and building internal infrastructure through train-the-trainer programs.

The prospects for sustaining evidence-based responses to the US opioid epidemic: state leadership perspectives

BACKGROUND

The US 21st Century Cures Act provided $7.5 billion in grant funding to states and territories for evidence-based responses to the opioid epidemic. Currently, little is known about optimal strategies for sustaining these programs beyond this start-up funding.

METHODS

Using an inductive, conventional content analysis, we conducted key informant interviews with former and current state leaders (n = 16) about barriers/facilitators to sustainment and strategies for sustaining time-limited grants.

RESULTS

Financing and reimbursement, service integration, and workforce capacity were the most cited barriers to sustainment. Status in state government structure, public support, and spending flexibility were noted as key facilitators. Effective levers to increase chances for sustainment included strong partnerships with other state agencies, workforce and credentialing changes, and marshalling advocacy through public awareness campaigns.

CONCLUSIONS

Understanding the strategies that leaders have successfully used to sustain programs in the past can inform how to continue future time-limited, grant-funded initiatives.

Measurement of sustainment of prevention programs and initiatives: the sustainment measurement system scale

Background

Enhancing the sustainability of evidence-based prevention programs for mental and behavioral health requires tools for measuring both sustainability determinants and sustainment outcomes. The aim of this study was to develop the Sustainment Measurement System Scale (SMSS) and to assess its reliability and construct validity for measuring both determinants and outcomes of efforts to sustain prevention programs and initiatives.

Methods

A 42-item scale comprised of items identified from qualitative data collected from 45 representatives of 10 programs and 8 SAMHSA program officers was administered to 186 representatives of 145 programs funded by 7 SAMHSA prevention grant initiatives. Cronbach’s alphas were used to determine inter-item reliability. Convergent validity was assessed by comparisons of a global measure of sustainment with current SAMHSA-funding status and continued operation in the same form. Discriminant validity was assessed by comparisons of sustainability determinants with whether or not the program had undergone adaptations.

Results

Confirmatory factor analysis provided support for a 35-item model fit to the data. Cronbach’s alpha was .84 for the sustainment outcome construct and ranged from .70 to .93 for the sustainability determinant constructs. All of the determinant constructs were significantly associated with sustainment outcome individual and global measures for the entire sample (p < 0.01 to 0.001) and for community-based programs and programs with a substance abuse focus (p < 0.05 to 0.001). Convergent validity was supported by significant associations between the global sustainment measure and current SAMHSA funding status and continued operation in the same form (p < 0.001). Four of the sustainability determinant constructs (responsive to community needs; coalitions, partnerships, and networks; organizational staff capability; and evaluation, feedback, and program outcomes) were also significantly associated with current SAMHSA funding status (p < 0.5 to 0.01). With the exception of organizational staff capability, all sustainability determinants were unrelated to program adaptation as predicted.

Conclusions

The SMSS demonstrated good reliability and convergent and discriminant validity in assessing likelihood of sustainment of SAMHSA funded prevention programs and initiatives. The measure demonstrates potential in identifying predictors of program sustainment and as a tool for enhancing the likelihood of successful sustainment through ongoing evaluation and feedback.

Conceptualizing and measuring sustainability of prevention programs, policies, and practices

A large knowledge gap exists regarding the measurement of sustainability of evidence-based prevention programs for mental and behavioral health. We interviewed 45 representatives of 10 grantees and 9 program officers within 4 Substance Abuse and Mental Health Services Administration prevention grant initiatives to identify experiences with implementation and sustainability barriers and facilitators; what "sustainability" means and what it will take to sustain their programs; and which Consolidated Framework for Implementation Research (CFIR) elements are important for sustainability. Lists of sustainability determinants and outcomes were then compiled from each data set and compared with one another. Analysis of themes from interviews and free lists revealed considerable overlap between sustainability determinants and outcomes. Four sustainability elements were identified by all three data sets (ongoing coalitions, collaborations, and networks and partnerships; infrastructure and capacity to support sustainability; community need for program; and ongoing evaluation of performance and outcomes), and 11 elements were identified by two of three data sets (availability of funding; consistency with organizational culture; evidence of positive outcomes; development of a plan for implementation and sustainment; presence of a champion; institutionalization and integration of program; institutional support and commitment; community buy-in and support; program continuity; supportive leadership; and opportunities for staff training). All but one of the CFIR domain elements (pressure from other states, tribes, or communities) were endorsed as important to sustainability by 50% or more of participants. It may be more important to implement a standardized process of eliciting determinants and outcomes of sustainability than to implement a single standardized instrument.

Conceptualizing and measuring sustainability of prevention programs, policies, and practices

A large knowledge gap exists regarding the measurement of sustainability of evidence-based prevention programs for mental and behavioral health. We interviewed 45 representatives of 10 grantees and 9 program officers within 4 Substance Abuse and Mental Health Services Administration prevention grant initiatives to identify experiences with implementation and sustainability barriers and facilitators; what "sustainability" means and what it will take to sustain their programs; and which Consolidated Framework for Implementation Research (CFIR) elements are important for sustainability. Lists of sustainability determinants and outcomes were then compiled from each data set and compared with one another. Analysis of themes from interviews and free lists revealed considerable overlap between sustainability determinants and outcomes. Four sustainability elements were identified by all three data sets (ongoing coalitions, collaborations, and networks and partnerships; infrastructure and capacity to support sustainability; community need for program; and ongoing evaluation of performance and outcomes), and 11 elements were identified by two of three data sets (availability of funding; consistency with organizational culture; evidence of positive outcomes; development of a plan for implementation and sustainment; presence of a champion; institutionalization and integration of program; institutional support and commitment; community buy-in and support; program continuity; supportive leadership; and opportunities for staff training). All but one of the CFIR domain elements (pressure from other states, tribes, or communities) were endorsed as important to sustainability by 50% or more of participants. It may be more important to implement a standardized process of eliciting determinants and outcomes of sustainability than to implement a single standardized instrument.

Innovations in Mixed Methods Evaluations

Mixed methods research-i.e., research that draws on both qualitative and quantitative methods in varying configurations-is well suited to address the increasing complexity of public health problems and their solutions. This review focuses specifically on innovations in mixed methods evaluations of intervention, program or policy (i.e., practice) effectiveness, and implementation. The article begins with an overview of the structure, function, and process of different mixed methods designs and then provides illustrations of their use in effectiveness studies, implementation studies, and combined effectiveness-implementation hybrid studies. The article then examines four specific innovations: procedures for transforming (or "quantitizing") qualitative data, application of rapid assessment and analysis procedures in the context of mixed methods studies, development of measures to assess implementation outcomes, and strategies for conducting both random and purposive sampling, particularly in implementation-focused evaluation research. The article concludes with an assessment of challenges to integrating qualitative and quantitative data in evaluation research.

Examination of sustainability indicators for fall prevention strategies in three states

With 1-in-4 older adults suffering a fall each year, fall prevention efforts have emerged as a public health priority. Multi-level, evidence-based fall prevention programs have been promoted by the CDC and other government agencies. To ensure participants and communities receive programs' intended benefits, organizations must repeatedly deliver the programs over time and plan for program sustainability as part of 'scaling up' the initiative. The State Falls Prevention Project (SFPP) began in 2011 when the CDC provided 5 years of funding to State Departments of Health in Colorado, New York, and Oregon to simultaneously implement four fall prevention strategies: 1) Tai Chi: Moving for Better Balance; 2) Stepping On; 3) Otago Exercise Program; and 4) STEADI (STopping Elderly Accidents, Deaths, and Injuries) toolkit. Surveys were performed to examine systems change and perceptions about sustainability across states. The purposes of this study were to: 1) examine how funding influenced the capacity for program implementation and sustainability within the SFPP; and 2) assess reported Program Sustainability Assessment Tool (PSAT) scores to learn about how best to sustain fall preventing efforts after funding ends. Data showed that more organizations offered evidence-based fall prevention programs in participants' service areas with funding, and the importance of programming implementation, evaluation, and reporting efforts were likely to diminish once funding concluded. Participants' reported PSAT scores about perceived sustainability capacity did not directly align with previously reported perceptions about PSAT domain importance or modifiability. Findings suggest the importance of grantees to identify potential barriers and enablers influencing program sustainability during the planning phase of the programs.

Agency Leaders' Assessments of Feasibility and Desirability of Implementation of Evidence-Based Practices in Youth-Serving Organizations Using the Stages of Implementation Completion

Background: This study examined influences on the decisions of administrators of youth-serving organizations to initiate and proceed with implementation of an evidence-based practice (EBP).

Methods: Semi-structured interviews, developed using the Stages of Implementation Completion (SIC) as a framework, were conducted with 19 agency chief executive officers and program directors of 15 organizations serving children and adolescents.

Results: Agency leaders' self-assessments of implementation feasibility and desirability prior to implementation (Pre-implementation) were influenced by intervention affordability, feasibility, requirements, validity, reliability, relevance, cost savings, positive outcomes, and adequacy of information; availability of funding, support from sources external to the agency, and adequacy of technical assistance; and staff availability and attitudes toward innovation in general and EBPs in particular, organizational capacity, fit between the EBP and agency mission and capacity, prior experience with implementation, experience with seeking evidence, and developing consensus. Assessments during the Implementation phase included intervention flexibility and requirements; availability of funding, adequacy of training and technical assistance, and getting sufficient and appropriate referrals; and staffing and implementing with fidelity. Assessments during the Sustainment phase included intervention costs and benefits; availability of funding, support from sources outside of the agency, and need for the EBP; and the fit between the EBP and the agency mission.

Discussion: The results point to opportunities for using agency leader models to develop strategies to facilitate implementation of evidence-based and innovative practices for children and adolescents. The SIC provides a standardized framework for guiding agency leader self-assessments of implementation.

Addressing gaps in international blood availability and transfusion safety in low- and middle-income countries: a NHLBI workshop

In April 2017, a workshop sponsored by the National Heart, Lung, and Blood Institute, Division of Blood Diseases and Resources, and the Center for Translation Research and Implementation Science was held to discuss blood availability and transfusion safety in low- and middle-income countries (LMICs). The purpose of the workshop was to identify research opportunities for implementation science (IS) to improve the availability of safe blood and blood components and transfusion practices in LMICs. IS describes the late stages of the translational research spectrum and studies optimal and sustainable strategies to deliver proven-effective interventions. Regional working groups were formed to focus on opportunities and challenges in East Africa, Central/West Africa, Middle East and North Africa, Latin America and the Caribbean, Southeast Asia, Western Pacific Asia, Eastern Europe, and Central Asia. The need for an "adequate supply of safe blood" emerged as the major overriding theme. Among the regional working groups, common cross-cutting themes were evident. The majority of research questions, priorities, and strategies fell into the categories of blood availability, blood transfusion safety, appropriate use of blood, quality systems, health economics and budgeting, and training and education in IS. The workshop also brought into focus inadequate country-level data that can be used as the basis for IS initiatives. A mixed approach of needs assessment and targeted interventions with sufficient evidence base to move toward sustainment is an appropriate next step for blood availability and transfusion safety research in LMICs.

Reducing Cardiovascular Disparities Through Community-Engaged Implementation Research: A National Heart, Lung, and Blood Institute Workshop Report

Cardiovascular disparities remain pervasive in the United States. Unequal disease burden is evident among population groups based on sex, race, ethnicity, socioeconomic status, educational attainment, nativity, or geography. Despite the significant declines in cardiovascular disease mortality rates in all demographic groups during the last 50 years, large disparities remain by sex, race, ethnicity, and geography. Recent data from modeling studies, linked micromap plots, and small-area analyses also demonstrate prominent variation in cardiovascular disease mortality rates across states and counties, with an especially high disease burden in the southeastern United States and Appalachia. Despite these continued disparities, few large-scale intervention studies have been conducted in these high-burden populations to examine the feasibility of reducing or eliminating cardiovascular disparities. To address this challenge, on June 22 and 23, 2017, the National Heart, Lung, and Blood Institute convened experts from a broad range of biomedical, behavioral, environmental, implementation, and social science backgrounds to summarize the current state of knowledge of cardiovascular disease disparities and propose intervention strategies aligned with the National Heart, Lung, and Blood Institute mission. This report presents the themes, challenges, opportunities, available resources, and recommended actions discussed at the workshop.

An Appraisal of Social Network Theory and Analysis as Applied to Public Health: Challenges and Opportunities

The use of social network theory and analysis methods as applied to public health has expanded greatly in the past decade, yielding a significant academic literature that spans almost every conceivable health issue. This review identifies several important theoretical challenges that confront the field but also provides opportunities for new research. These challenges include (a) measuring network influences, (b) identifying appropriate influence mechanisms, (c) the impact of social media and computerized communications, (d) the role of networks in evaluating public health interventions, and (e) ethics. Next steps for the field are outlined and the need for funding is emphasized. Recently developed network analysis techniques, technological innovations in communication, and changes in theoretical perspectives to include a focus on social and environmental behavioral influences have created opportunities for new theory and ever broader application of social networks to public health topics.