Building capacity for dissemination and implementation research: one university's experience

Implementation Feasibility and Hidden Costs of Statewide Scaling of Evidence-Based Therapies for Children and Adolescents

OBJECTIVE

State mental health systems are retraining their workforces to deliver services supported by research. Knowledge about evidence-based therapies (EBTs) for child and adolescent disorders is robust, but the feasibility of their statewide scaling has not been examined. The authors reviewed implementation feasibility for 12 commonly used EBTs, defining feasibility for statewide scaling as an EBT having at least one study documenting acceptability, facilitators and barriers, or fidelity; at least one study with a racially and ethnically diverse sample; an entity for training, certification, or licensing; and fiscal data reflecting the costs of implementation.

METHODS

The authors reviewed materials for 12 EBTs being scaled in New York State and conducted a literature review with search terms relevant to their implementation. Costs and certification information were supplemented by discussions with treatment developers and implementers.

RESULTS

All 12 EBTs had been examined for implementation feasibility, but only three had been examined for statewide scaling. Eleven had been studied in populations reflecting racial-ethnic diversity, but few had sufficient power for subgroup analyses to demonstrate effectiveness with these samples. All had certifying or licensing entities. The per-clinician costs of implementation ranged from $500 to $3,500, with overall ongoing costs ranging from $100 to $6,000. A fiscal analysis of three EBTs revealed hidden costs ranging from $5,000 to $24,000 per clinician, potentially limiting sustainability.

CONCLUSIONS

The evidence necessary for embedding EBTs in state systems has notable gaps that may hinder sustainability. Research-funding agencies should prioritize studies that focus on the practical aspects of scaling to assist states as they retrain their workforces.

Adapting, Implementing, and Maintaining a Group Cognitive Behavioral Therapy Program at an Inpatient Addiction Treatment Facility

Background

Quality training is an oft-cited barrier to effective implementation and ongoing delivery of high-quality evidence-based practice (EBP) across fields. This is especially true in the addiction field, but there is little cited evidence for optimal methods to improve EBP in inpatient addiction facilities with minimal resources.

Objective

The current paper focuses on evaluating the state of our facility's group CBT manual and clinical training on the manual in a "realistic" (ie, non-RCT, non-grant-funded) inpatient addiction treatment setting.

Methods

Five full-time clinicians volunteered to take part in the study (woman = 60%; Mage = 36.20 years). The study involved a mix of semi-structured interviews and surveys designed to measure seven outcomes (barriers, feasibility, useability, appropriateness, acceptability, burden, trialability).

Results

Three themes emerged from the data that impacted the group CBT manual: training, timing, and functionality. Addressing these themes allowed for a new, optimal manual and training procedure to be put into place.

Conclusion

The current study highlights that under-resourced inpatient addiction facilities can still methodically utilize implementation approaches to study their EBP, namely CBT. Such an approach will ensure that the highest quality care is being delivered to patients and actively addresses known training barriers that prevent proper EBP delivery.

Process evaluation of an academic dissemination and implementation science capacity building program

The UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center (DISC) launched in 2020 to provide dissemination and implementation science (DIS) training, technical assistance, community engagement, and research advancement. DISC developed a program-wide logic model to inform a process evaluation of member engagement and impact related to DISC services. The DISC Logic Model (DLM) served as the framework for a process evaluation capturing quantitative and qualitative information about scientific activities, outputs, and outcomes. The evaluation involved a multimethod approach with surveys, attendance tracking, feedback forms, documentation of grant outcomes, and promotions metrics (e.g., Twitter engagement). There were 540 DISC Members at the end of year 2 of the DISC. Engagement in the DISC was high with nearly all members endorsing at least one scientific activity. Technical assistance offerings such as DISC Journal Club and consultation were most frequently used. The most common scientific outputs were grant submission (65, 39%), formal mentoring for career award (40, 24%), and paper submission (34, 21%). The DLM facilitated a comprehensive process evaluation of our center. Actionable steps include prioritizing technical assistance, strengthening networking opportunities, identifying streamlined approaches to facilitate DIS grant writing through writing workshops, as well as "office hours" or organized writing leagues.

Protocol for a scoping review of health equity frameworks and models applied in empirical studies of chronic disease prevention and control

BACKGROUND

Chronic diseases, such as cancers and cardiovascular diseases, present the greatest burden of morbidity and mortality worldwide. This burden disproportionately affects historically marginalized populations. Health equity is rapidly gaining increased attention in public health, health services, and implementation research, though many health inequities persist. Health equity frameworks and models (FM) have been called upon to guide equity-focused chronic disease and implementation research. However, there is no clear synthesis of the health equity FM used in chronic disease research or how these are applied in empirical studies. This scoping review seeks to fill this gap by identifying and characterizing health equity FM applied in empirical studies along the chronic disease prevention and control continuum, describing how these FM are used, and exploring potential applications to the field of implementation science.

METHODS

We follow established guidance for conducting scoping reviews, which includes six stages: (1) identify the research question; (2) identify relevant studies; (3) select studies for inclusion; (4) data extraction; (5) collating, summarizing, and reporting the results; and (6) consultation. This protocol presents the iterative, collaborative approach taken to conceptualize this study and develop the search strategy. We describe the criteria for inclusion in this review, methods for conducting two phases of screening (title and abstract, full text), data extraction procedures, and quality assurance approaches taken throughout the project.

DISCUSSION

The findings from this review will inform health-equity focused chronic disease prevention and control research. FM identified through this review will be added to an existing website summarizing dissemination and implementation science frameworks, and we will offer case examples and recommendations for utilizing a health equity FM in empirical studies. Our search strategy and review methodology may serve as an example for scholars seeking to conduct reviews of health equity FM in other health disciplines.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework Registration https://doi.org/10.17605/OSF.IO/SFVE6.

A systematic review of dissemination and implementation science capacity building programs around the globe

BACKGROUND

Research centers and programs focused on dissemination and implementation science (DIS) training, mentorship, and capacity building have proliferated in recent years. There has yet to be a comprehensive inventory of DIS capacity building program (CBP) cataloging information about activities, infrastructure, and priorities as well as opportunities for shared resources, collaboration, and growth. The purpose of this systematic review is to provide the first inventory of DIS CBPs and describe their key features and offerings.

METHODS

We defined DIS CBPs as organizations or groups with an explicit focus on building practical knowledge and skills to conduct DIS for health promotion. CBPs were included if they had at least one capacity building activity other than educational coursework or training alone. A multi-method strategy was used to identify DIS CBPs. Data about the characteristics of DIS CBPs were abstracted from each program's website. In addition, a survey instrument was developed and fielded to gather in-depth information about the structure, activities, and resources of each CBP.

RESULTS

In total, 165 DIS CBPs met our inclusion criteria and were included in the final CBP inventory. Of these, 68% are affiliated with a United States (US) institution and 32% are internationally based. There was one CBP identified in a low- and middle-income country (LMIC). Of the US-affiliated CBPs, 55% are embedded within a Clinical and Translational Science Award program. Eighty-seven CBPs (53%) responded to a follow-up survey. Of those who completed a survey, the majority used multiple DIS capacity building activities with the most popular being Training and Education (n=69, 79%) followed by Mentorship (n=58, 67%), provision of DIS Resources and Tools (n=57, 66%), Consultation (n=58, 67%), Professional Networking (n=54, 62%), Technical Assistance (n=46, 52%), and Grant Development Support (n=45, 52%).

CONCLUSIONS

To our knowledge, this is the first study to catalog DIS programs and synthesize learnings into a set of priorities and sustainment strategies to support DIS capacity building efforts. There is a need for formal certification, accessible options for learners in LMICs, opportunities for practitioners, and opportunities for mid/later stage researchers. Similarly, harmonized measures of reporting and evaluation would facilitate targeted cross-program comparison and collaboration.

Peer mentorship to build research capacity among members of the International Student Surgical Network (InciSioN): a proof of concept study

BACKGROUND

International Student Surgical Network (InciSioN) is the largest student and trainee global surgery interest group worldwide and its members have contributed significantly to global surgery research. The InciSioN Research Capacity-Building (IReCaB) program aimed to enhance the research skills and confidence of participants via a peer mentorship model.

METHODS

After an open call to members of InciSioN to enroll, participants' knowledge of research methods and the process was evaluated through a test to assign mentor and mentee roles, with mentors being those who scored ≥ 20/25. Mentors then delivered a series of four webinars to help disseminate research methodology to mentees. Finally, mentees were tested on their knowledge of research and their satisfaction with the program was also evaluated.

RESULTS

Fifty-two participants, mostly from LMICs (n = 23/52, 44.2%) were enrolled, and 36 completed the program. There was a significant improvement in the proportion of questions answered correctly on the post-program test (R = 0.755, p < 0.001). Post-IReCaB test scores were greater than pre-IReCaB scores (p < 0.001). The difference in confidence after the course was also significant (p < 0.001). IReCaB participants successfully designed, implemented, and published an international cross-sectional study.

CONCLUSION

This study showed improvements in participants' understanding of theoretical components of scientific research. We offer a model for research capacity building that can be implemented, modeled, and further refined by similar organizations with academic research goals.

Dissemination and implementation science resources, training, and scientific activities provided through CTSA programs nationally: Opportunities to advance D&I research and training capacity

Introduction

Clinical and Translational Science Award (CTSA) Program hubs are well-positioned to advance dissemination and implementation (D&I) research and training capacity nationally, though little is known about what D&I research support and services CTSAs provide. To address this gap, the CTSA Dissemination, Implementation, and Knowledge Transfer Working Group conducted an environmental scan of CTSAs (2017-2018).

Methods

Of 67 CTSA institutions, we contacted 43 that previously reported delivering D&I research services. D&I experts from these institutions were emailed a survey assessing D&I resources, services, training, and scientific projects. Responses were categorized and double-coded by study authors using a content analysis approach.

Results

Thirty-five of the 43 D&I experts (81.4%) responded. Challenges to CTSAs in developing and supporting D&I science activities were related to inadequate D&I science workforce (45.7%) and lack of understanding of D&I science (25.7%). Services provided included consultation/mentoring programs (68%), pilot funding/grants (50%), and workshops/seminars/conferences (46%). Training and workforce development in D&I were frequently identified as future priorities. Recommendations included increase training to meet demand (68.6%), accessible D&I tools/resources (34.3%), greater visibility/awareness of D&I methods (34.3%), consultation services (22.9%), and expand D&I science workforce (22.9%).

Conclusions

CTSAs have tremendous potential to support the advancement and impact of D&I science across the translational continuum. Despite the growing presence of D&I science in CTSAs, continued commitment and prioritization are needed from CTSA and institutional leadership to raise awareness of D&I science and its value, meet training demands, and develop necessary infrastructure for conducting D&I science.

A network analysis of dissemination and implementation research expertise across a university: Central actors and expertise clusters

Background:

Although dissemination and implementation (D&I) science is a growing field, many health researchers with relevant D&I expertise do not self-identify as D&I researchers. The goal of this work was to analyze the distribution, clustering, and recognition of D&I expertise in an academic institution.

Methods:

A snowball survey was administered to investigators at University of Rochester with experience and/or interest in D&I research. The respondents were asked to identify their level of D&I expertise and to nominate others who were experienced and/or active in D&I research. We used social network analysis to examine nomination networks.

Results:

Sixty-eight participants provided information about their D&I expertise. Thirty-eight percent of the survey respondents self-identified as D&I researchers, 24% as conducting D&I under different labels, and 38% were familiar with D&I concepts. D&I researchers were, on average, the most central actors in the network (nominated most by other survey participants) and had the highest within-group density, indicating wide recognition by colleagues and among themselves. Researchers who applied D&I under different labels had the highest within-group reciprocity (25%), and the highest between-group reciprocity (29%) with researchers familiar with D&I. Participants significantly tended to nominate peers within their departments and within their expertise categories.

Conclusions:

Identifying and engaging unrecognized clusters of expertise related to D&I research may provide opportunities for mutual learning and dialog and will be critical to bridging across departmental and topic area silos and building capacity for D&I in academic settings.

Barriers and Facilitators for Conducting Implementation Science in German-Speaking Countries: Findings from the Promote ImpSci Interview Study

On a global scale, implementation science has developed considerably as a discipline in recent years. In German-speaking countries, the field has been gaining significance as well, but respective efforts in building capacity and advancing the research infrastructure are still rare. The present study investigates barriers and facilitators for conducting implementation science in German-speaking countries with the goal of formulating recommendations for creating a more supportive research infrastructure. We conducted an interview study with nine well-established implementation researchers affiliated with universities in Austria, Germany, or Switzerland. The interviews were held via Zoom or phone in November and December 2020, transcribed verbatim and analyzed using thematic analysis. Barriers that relate to characteristics of the discipline were difficulties in building a common understanding of the field and the complexity of implementation research projects. Although supportive scientific networks were important facilitators, interviewees mentioned challenges in connecting with likeminded researchers. A further barrier was the lack of opportunities for education and training in implementation science, especially in the German language. Also, participants reported a missing readiness in academia for establishing implementation science that should be addressed by advocacy of the discipline toward academic decision makers. Moreover, since most national funding agencies prioritize basic research over applied research, some interviewees named flexibility in handling research funds as a facilitator for implementation research. The results inform an agenda for promoting implementation science in German-speaking countries and can be beneficial to other countries that are currently advancing their implementation research capacity and infrastructure.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-022-00046-3.

Getting To Implementation (GTI)-Teach: A seven-step approach for teaching the fundamentals of implementation science

Introduction:

Implementation Science (IS) is a complex and rapidly evolving discipline, posing challenges for educators. We developed, implemented, and evaluated a novel, pragmatic approach to teach IS.

Methods:

Getting To Implementation (GTI)-Teach was developed as a seven-step educational model to guide students through the process of developing, conducting, and sustaining an IS research project. During the four-week online course, students applied the steps to self-selected implementation problems. Students were invited to complete two online post-course surveys to assess course satisfaction and self-reported changes in IS knowledge and relevance of GTI-Teach Steps to their work. Results were summarized using descriptive statistics; self-reported post-course changes in IS knowledge were compared using paired t-tests.

Results:

GTI-Teach was developed to include seven Steps: 1. Define the implementation problem; 2. Conceptualize the problem; 3. Prioritize implementation barriers and facilitators; 4. Select and tailor implementation strategies; 5. Design an implementation study; 6. Evaluate implementation; 7. Sustain implementation. Thirteen students, ranging in experience from medical students to full professors, enrolled in and completed the first GTI-Teach course. Of the seven students (54%) completing an end-of course survey, six (86%) were very satisfied with the course. Ten students (77%) responded to the tailored, 6-month post-course follow-up survey. They retrospectively reported a significant increase in their knowledge across all steps of GTI-Teach (1.3–1.8 points on a 5-point Likert scale) and rated each of the Steps as highly relevant to their work.

Conclusions:

GTI-Teach is a seven-step model for teaching IS fundamentals that students reported increased their knowledge and was relevant to their work.

A mixed methods study to develop a tool to assess institutional readiness to conduct knowledge translation activities in low-income and middle-income countries

OBJECTIVE

This paper describes the development of a tool for assessing organisational readiness to conduct knowledge translation (KT) among academic institutions in low-income and middle-income countries (LMICs).

DESIGN

A literature review and stakeholder consultation process were conducted to identify constructs relevant for assessing KT readiness in LMICs. These were face-validated with LMIC stakeholders and organised into a Likert-scale questionnaire.

PARTICIPANTS

The questionnaire was distributed to researchers based at six LMIC academic institutions and members of a global knowledge-to-action thematic working group.

OUTCOME MEASURES

An exploratory factor analysis was used to identify underlying dimensions for assessing institutional readiness to conduct KT.

RESULTS

111 respondents with varied KT experiences from 10 LMICs were included in the analysis. We selected 5 factors and 23 items, with factor loadings from 0.40 to 0.77. These factors include (1) institutional climate, (2) organisation change efficacy, (3) prioritisation and cosmopolitanism, (4) self-efficacy, and (5) financial resources. These factors accounted for 69% of the total variance, with Cronbach's alpha coefficients of 0.78, 0.73, 0.62, 0.68 and 0.52, respectively.

CONCLUSIONS

This study identifies a tool for assessing readiness of LMIC academic institutions to conduct KT and unique opportunities for building capacity. The organisational focus of these factors underscores the need for strategies that address organisational systems and structures in addition to individual skills. Future research will be conducted to understand determinants of these factors and develop a comprehensive set of capacity building strategies responsive to academic institutions in LMICs.

Opportunities and challenges in translational science

The mission of translational science is to bring predictivity and efficiency to the development and dissemination of interventions that improve human health. Ten years ago this year, the National Center for Advancing Translational Sciences was founded to embody, conduct, and support this new discipline. The Center's first decade has brought substantial progress across a broad range of translational areas, from diagnostic and drug development to clinical trials to implementation science to education. The origins of the translational science and advances to this point are reviewed here and allow the establishment of an ambitious future research agenda for the field.

Evaluating an Implementation Science Training Program: Impact on Professional Research and Practice

Real-world application and implementation of evidence-based practice continue to be a challenge across multiple sectors, including behavioral health settings. Providing the opportunity for future researchers and practitioners to gain capacity and knowledge through structured experiential learning in implementation science is critical to closing the research to practice gap. The Institute for Translational Research Education in Adolescent Drug Abuse (ITRE) is a graduate certificate program that offers specific coursework, a large-scale service-learning project based in the community, and mentorship related to implementation science research and practice. The purpose of this evaluation was to examine, from the perspective of ITRE scholars, the perceived impact on the development of professional research and practice skills once graduated from the ITRE program. Fifty-eight semi-structured interviews across five cohorts were selected randomly for in-depth thematic analysis (n = 58). Results suggest that the ITRE provides a unique approach grounded in implementation science for building robust and transferable skills for future researchers and practitioners working in a variety of behavioral healthcare settings.

Translating the learning sciences into practice: A primer for clinical and translational educators

The learning sciences have yielded a wealth of insights about the mechanisms and conditions that promote learning, yet the findings from this body of research often do not make their way into educational practice. This fundamentally translational problem is one we believe that educators from translational fields, with their evidence-based orientation and familiarity with the challenges and importance of translation, are well-positioned to address. Here, we provide a primer on the learning sciences to guide educators in the Clinical and Translational Science Institutes and other organizations that train translational researchers. We (a) describe the unique teaching and learning environment in which this training occurs, and why it necessitates attention to learning research and its appropriate application, (b) explain what the learning sciences are, (c) distill the complex science of learning into core principles, (d) situate recent developments in the field within these principles, and (e) explain, in practical terms, how these principles can inform our teaching.

A mapping review of NIDA-funded implementation research studies on treatments for opioid and/or stimulant use disorders

BACKGROUND

The biomedical research enterprise invests greatly in discovery-oriented science, but significantly less in how to implement the most effective of these innovations. The return on investment in public health benefit is therefore low. In the context of substance-related overdose epidemics, presently with opioids and/or stimulants, the gap in proven treatments and routine access is amplified. Implementation research is designed to deepen understanding of how best to scale-up proven treatments. This study assessed how implementation research has been deployed in the National Institute on Drug Abuse (NIDA) efforts to address the opioid and stimulant epidemics.

METHODS

Adapting a procedure developed to categorize HIV-focused research, a four-stage systematic mapping review of NIDA-funded R01, R34, R61, and U studies pertaining to opioids and/or stimulants funded between 2015 and 2019 was performed. Abstracts were retrieved using NIH Research Portfolio Online Reporting Tools. Key study characteristics were abstracted and coded by two independent reviewers.

RESULTS

An initial search across NIH institutes yielded 5963 relevant records. Of these, 666 (11.2 %) were NIDA funded. One-hundred-and-thirty-four (20.1 %) of the 666 studies were opioid and/or stimulant treatment related. Of these, 28 (4.2 %) were categorized as Implementation Preparation (IP), and 16 (2.4 %) were categorized as Implementation Research (IR). Over the five-year period, there was a gradual increase in both IP and IR studies.

CONCLUSIONS

Implementation research is a small but slowly growing component of the federal portfolio to address substance-related public health issues. To more effectively respond to contemporary overdose epidemics, implementation research must take on an even more significant role.

Barriers and facilitators to knowledge translation activities within academic institutions in low- and middle-income countries

The barriers and facilitators of conducting knowledge translation (KT) activities are well-established but less is known about the institutional forces that drive these barriers, particularly in low resource settings. Understanding organizational readiness has been used to assess and address such barriers but the employment of readiness assessments has largely been done in high-income countries. We conducted a qualitative study to describe the institutional needs and barriers in KT specific to academic institutions in low- and middle-income countries. We conducted a review of the grey and published literature to identify country health priorities and established barriers and facilitators for KT. Key-informant interviews (KII) were conducted to elicit perceptions of institutional readiness to conduct KT, including experiences with KT, and views on motivation and capacity building. Participants included representatives from academic institutions and Ministries of Health in six countries (Bangladesh, Democratic Republic of the Congo, Ethiopia, India, Indonesia, Nigeria). We conducted 18 KIIs, 11 with members of academic institutions and 7 with policymakers. KIIs were analysed using a deductive and inductive coding approach. Our findings support many well-documented barriers including lack of time, skills and institutional support to conduct KT. Three additional institutional drivers emerged around soft skills and the complexity of the policy process, alignment of incentives and institutional missions, and the role of networks. Participants reflected on often-lacking soft-skills needed by researchers to engage policy makers. Continuous engagement was viewed as a challenge given competing demands for time (both researchers and policy makers) and lack of institutional incentives to conduct KT. Strong networks, both within the institution and between institutions, were described as important for conducting KT but difficult to establish and maintain. Attention to the cross-cutting themes representing barriers and facilitators for both individuals and institutions can inform the development of capacity building strategies that meet readiness needs.

Implementing Implementation Research: Teaching Implementation Research to HIV Researchers

PURPOSE OF REVIEW

Given the growth in HIV-related implementation research, there is a need to expand the workforce and rigor through implementation science (IS) training and mentorship. Our objective is to review IS training opportunities for HIV-focused researchers and describe the approach and lessons learned from a recent HIV-related implementation research training initiative.

RECENT FINDINGS

IS training opportunities range from degree programs to short- and longer-term professional development institutes and community-focused institutional trainings. Until recently, there have not been extensive dedicated opportunities for implementation research training for HIV-focused investigators. To meet this gap, an inter-Center for AIDS Research IS Fellowship for early-stage investigators was launched in 2019, building on lessons learned from dissemination and implementation training programs. Key components of the HIV-focused IS fellowship include didactic training, mentorship, grant-writing, and development of HIV-IS collaborative networks. Fellows to-date were two-thirds junior faculty and one-third post-doctoral fellows, the majority (69%) with prior public health training. Perceived value of the program was high, with a median rating of 9 [IQR 8-9] on a 10-point scale. Overall, 22/27 (81%) Fellows from the first cohort submitted IS-related grants within 12 months of Fellowship completion, and by 1 year 13 grants had been funded among 10 investigators, 37% overall among Fellows. Mentors identified framing of IS questions as the top-ranked training priority for HIV-investigators. Increasing knowledge of the utility of IS may support more grants focused on optimal implementation of HIV treatment and prevention strategies. Experiences from mentors and trainees engaged in an IS-focused fellowship for HIV investigators demonstrate the demand and value of a dedicated training program and reinforce the importance of mentorship.

Implementation Science and Nursing Leadership: Improving the Adoption and Sustainability of Evidence-Based Practice

Increasingly, nurse executives are embracing evidence-based practice (EBP). Yet, it takes 17 years to move only 14% of research evidence into practice. Implementation science (IS) aims to reduce this gap through the study of factors that influence dissemination, adoption, implementation, and sustainability of evidence-based interventions. Nurse leaders aiming for EBP must be well equipped to support IS. This article introduces IS, leadership competencies essential for supporting implementation research and practice, and preparing an IS-ready workforce.

The FRAME-IS: a framework for documenting modifications to implementation strategies in healthcare

BACKGROUND

Implementation strategies are necessary to ensure that evidence-based practices are successfully incorporated into routine clinical practice. Such strategies, however, are frequently modified to fit local populations, settings, and contexts. While such modifications can be crucial to implementation success, the literature on documenting and evaluating them is virtually nonexistent. In this paper, we therefore describe the development of a new framework for documenting modifications to implementation strategies.

DISCUSSION

We employed a multifaceted approach to developing the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS), incorporating multiple stakeholder perspectives. Development steps included presentations of initial versions of the FRAME-IS to solicit structured feedback from individual implementation scientists ("think-aloud" exercises) and larger, international groups of researchers. The FRAME-IS includes core and supplementary modules to document modifications to implementation strategies: what is modified, the nature of the modification (including the relationship to core elements or functions), the primary goal and rationale for the modification, timing of the modification, participants in the modification decision-making process, and how widespread the modification is. We provide an example of application of the FRAME-IS to an implementation project and provide guidance on how it may be used in future work.

CONCLUSION

Increasing attention is being given to modifications to evidence-based practices, but little work has investigated modifications to the implementation strategies used to implement such practices. To fill this gap, the FRAME-IS is meant to be a flexible, practical tool for documenting modifications to implementation strategies. Its use may help illuminate the pivotal processes and mechanisms by which implementation strategies exert their effects.

Consulting the Community on Strategies to Strengthen Social Capital for Community Disease Control

Excluding communities in planning and implementing research maximizes internal risks that are otherwise visible and avoidable when there is adequate community consultation. Communities might not meaningfully use research results if majority of the researched people have minimal or no participation in information generation and dissemination. However, effective participation of researched communities in research is key to transferring knowledge to action. Using a qualitative approach, the study identified barriers to, and relevant strategies for improving health research uptake, particularly for schistosomiasis (commonly known as bilharzia) in the Ingwavuma area, uMkhanyakude District of KwaZulu-Natal. Data was collected through modified ethnography using participant observation, focus group discussions, unstructured in-depth interviews, and ethnographic conversational interviews. Results reveal that research uptake is inhibited by reduced opportunities for habitual interaction between residents, a paucity of innovative and inclusive health education activities and unsafe recreational facilities. The community's strategies on strengthening social capital for disease control include using existing social systems and power hierarchies to mobilise and organise and using the performing arts to facilitate habitual interaction and knowledge sharing. The study recommends a community consultation flow which facilitates openness about the benefits and the community's role in research, a pre-condition for community wide efforts in local disease prevention and control.

Traveling Without a Map: An Incomplete History of the Road to Implementation Science and Where We May Go from Here

This editorial provides a brief history of mental health services research over the last 30 years and how findings from large-scale studies shocked the field and led to the lines of inquiry culminating in current implementation science research. I review the manuscripts published in this special issue of Administration and Policy in Mental Health in light of that history and usethese studies as a way to assess the state of the field. Finally, I present five takeaways extracted from these articles that may be useful in considering future directions for implementation research.

Roles dissemination and implementation scientists can play in supporting research teams

The field of dissemination and implementation (D&I) science is rapidly growing, with many scientists seeking to apply D&I science to enhance and expand the impact of their work. As the D&I field grows and collaborations of implementation scientists with other fields flourish, a description for the roles for D&I scientists as they collaborate with researchers from other fields could be beneficial. This paper exemplifies how the D&I scientist/researcher collaborative process might work and important elements to consider in doing so, as well as provide an outline on how collaborations might progress for different project needs. This is discussed through example scenarios to consider an implementation scientists' engagement in a research project and describe potential roles for implementation scientists in supporting research teams. We then discuss characteristics to consider when incorporating a D&I expert into a team and considerations in navigating the scenarios.

Building capacity in dissemination and implementation science: a systematic review of the academic literature on teaching and training initiatives

BACKGROUND

The field of dissemination and implementation (D&I) science has grown significantly over recent years. Alongside this, an increased demand for training in D&I from researchers and implementers has been seen. Research describing and evaluating D&I training opportunities, referred to here as 'capacity building initiatives' (CBIs), can help provide an understanding of different methods of training as well as training successes and challenges. However, to gain a more detailed understanding of the evidence-base and how D&I CBIs are being reported in publications, a field-wide examination of the academic literature is required.

METHODS

Systematic review to identify the type and range of D&I CBIs discussed and/or appraised in the academic literature. EMBASE, Medline and PsycINFO were searched between January 2006 and November 2019. Articles were included if they reported on a D&I CBI that was developed by the authors (of each of the included articles) or the author's host institution. Two reviewers independently screened the articles and extracted data using a standardised form.

RESULTS

Thirty-one articles (from a total of 4181) were included. From these, 41 distinct D&I CBIs were identified which focussed on different contexts and professions, from 8 countries across the world. CBIs ranged from short courses to training institutes to being part of academic programmes. Nearly half were delivered face-face with the remainder delivered remotely or using a blended format. CBIs often stipulated specific eligibility criteria, strict application processes and/or were oversubscribed. Variabilities in the way in which the D&I CBIs were reported and/or evaluated were evident.

CONCLUSIONS

Increasing the number of training opportunities, as well as broadening their reach (to a wider range of learners), would help address the recognised deficit in D&I training. Standardisation in the reporting of D&I CBIs would enable the D&I community to better understand the findings across different contexts and scientific professions so that training gaps can be identified and overcome. More detailed examination of publications on D&I CBIs as well as the wider literature on capacity building would be of significant merit to the field.

Transforming the practice of medicine through team science

Background

The translation of biomedical research discoveries into clinical practice is marked by extended timelines (averaging 17 years) and multiple sequential process steps. However, even after a drug, device, diagnostic tool or unique therapeutic procedure successfully navigates through clinical testing to approval, real barriers remain in applying and scaling the innovation in practice.

Methods

Mayo Clinic initiated the Transform the Practice programme to facilitate multidisciplinary team and convergence science to continuously reinvent solutions to address unmet patient needs and accelerate the application of next-generation healthcare solutions. During a 5-year period, 24 programme teams received financial resources, barrier-removing engagement from clinical and research leadership, and enhanced administrative support, including dedicated project managers.

Results

The approach created value in facilitating consistent progress toward project objectives and resulted in multiple publications, new extramural funding sources, and implementation of new tests and services into the clinical practice. This report describes the concentrated institutional effort to accelerate the discovery–translation–application continuum in an academic medical centre and highlights successful applications and persistent obstacles.

Conclusions

The Transform the Practice approach is effective in moving high-potential research discoveries closer to implementation in the clinical practice. Its concepts, including the application of structured project management methodology, may be quickly integrated to shorten an organisation’s time to implementing its most important discoveries.

Developing implementation research capacity: longitudinal evaluation of the King's College London Implementation Science Masterclass, 2014-2019

BACKGROUND

Despite an increasing number of training opportunities in implementation science becoming available, the demand for training amongst researchers and practitioners is unmet. To address this training shortfall, we developed the King's College London 'Implementation Science Masterclass' (ISM), an innovative 2-day programme (and currently the largest of its kind in Europe), developed and delivered by an international faculty of implementation experts.

METHODS

This paper describes the ISM and provides delegates' quantitative and qualitative evaluations (gathered through a survey at the end of the ISM) and faculty reflections over the period it has been running (2014-2019).

RESULTS

Across the 6-year evaluation, a total of 501 delegates have attended the ISM, with numbers increasing yearly from 40 (in 2014) to 147 (in 2019). Delegates represent a diversity of backgrounds and 29 countries from across the world. The overall response rate for the delegate survey was 64.5% (323/501). Annually, the ISM has been rated 'highly' in terms of delegates' overall impression (92%), clear and relevant learning objectives (90% and 94%, respectively), the course duration (85%), pace (86%) and academic level 87%), and the support provided on the day (92%). Seventy-one percent of delegates reported the ISM would have an impact on how they approached their future work. Qualitative feedback revealed key strengths include the opportunities to meet with an international and diverse pool of experts and individuals working in the field, the interactive nature of the workshops and training sessions, and the breadth of topics and contexts covered.

CONCLUSIONS

Yearly, the UK ISM has grown, both in size and in its international reach. Rated consistently favourably by delegates, the ISM helps to tackle current training demands from all those interested in learning and building their skills in implementation science. Evaluation of the ISM will continue to be an annual iterative process, reflective of changes in the evidence base and delegates changing needs as the field evolves.

Perceptions of barriers and facilitators to engaging in implementation science: a qualitative study

OBJECTIVE

Understanding barriers and facilitators to engaging with implementation science (IS) research can provide insight into how to improve efforts to encourage more researchers to participate in IS research.

STUDY DESIGN

The study design used is a grounded theory qualitative study.

METHODS

We conducted semistructured telephone interviews with 20 health researchers familiar with IS that both report engaging in IS research and those that do not. We explored perceptions of barriers and facilitators to engaging in IS research. Themes surrounding difficulties defining IS, lack of training availability, and obstacles to forming research partnerships were discussed as barriers to engaging IS research. Interview topics were informed by the result of an online survey of health researchers in the US.

RESULTS

Themes surrounding difficulties defining IS, lack of training availability, and obstacles to forming research partnerships were discussed as barriers to engaging IS research. While accessible mentorship, exposure to formative experiences that develop interest in IS research and an increasing IS visibility were described as motivators for engaging in IS research.

CONCLUSIONS

These results highlight the importance of mentorship and exposure to IS ideas in motivating engagement in IS research and the presence of training and methodological barriers to engagement. Future research should expand this line of inquiry to include the perspectives of more junior researchers and students to better reflect the current IS environment.

Scale-up of the Accrual to Clinical Trials (ACT) network across the Clinical and Translational Science Award Consortium: a mixed-methods evaluation of the first 18 months

Introduction

The Clinical and Translational Science Award (CTSA) Program is a Consortium of nearly 60 academic medical research centers across the USA and a natural network for evaluating the spread and uptake of translational research innovation across the Consortium.

Methods

Dissemination of the Accrual to Clinical Trials (ACT) Network, a federated clinical informatics data network for population-based cohort discovery, began January 2018 across the Consortium. Diffusion of innovation theory guided dissemination design and evaluation. Mixed-methods assessed the spread and uptake across the Consortium through July 1, 2019 (n = 48 CTSAs). Methods included prospective time activity tracking (Kaplan-Meier curves), and survey and qualitative interviews.

Results

Within 18 months, nearly 80% of CTSAs had joined the data network and two-thirds of CTSAs achieving technical readiness had initiated launch to local clinical investigators. Over 10,000 ACT Network queries are projected for 2019; and by 2020, nearly all CTSAs will have joined the network. Median time-from-technical-readiness-to-local-launch was 154 days (interquartile range: 87-225 days]. Quality improvement processes reduced time-to-launch by 35.2% (64 days, p = 0.0036). Lessons learned include: (1) conceptualize dissemination as two-stage adoption demonstrating value for both CTSA hub service providers and clinical investigators; (2) include institutional trial into dissemination strategies so CTSA hubs can refine internal workflows and gather local user feedback endorsement; (3) embrace designing-for-dissemination during technology development; and (4) sustain adaptive dissemination and customer relationship management to keep CTSA hubs and users engaged.

Conclusions

Scale-up and spread of the ACT Network provides lessons learned for others disseminating innovation across the CTSA Consortium. The Network is primed for embedded implementation research.

The TDR MOOC training in implementation research: evaluation of feasibility and lessons learned in Rwanda

BACKGROUND

Hypertension (HTN) affects nearly 1 billion people globally and is a major cause of morbidity and mortality. In low- and middle-income countries (LMICs), HTN represents an unmet health care gap that can be addressed by strengthening national health care systems. The National Heart, Lung, and Blood Institute recently funded the T4 Translation Research Capacity Building Initiative in Low Income Countries (TREIN) program to build capacity in dissemination and implementation (D&I) research in HTN in LMICs. The Special Programme for Research and Training in Tropical Diseases (TDR) at the World Health Organization (WHO) recently developed a massive open online course (MOOC) to train in D&I. Herein, we report on the use of the TDR WHO MOOC in D&I for the TREIN program in Rwanda, assessing feasibility of the MOOC and D&I competencies after MOOC training.

METHODS

Participants in one-group MOOC training completed pre- and post-training questionnaires to assess dissemination and implementation (D&I) competency outcomes and feasibility. D&I competencies were measured by use of a scale developed for a US-based training program, with the change in competency scores assessed by paired t test. Feasibility was measured by completion of homework and final project assignment and analyzed using descriptive statistics.

RESULTS

Of the 92 trainees enrolled, 35 (38%) completed all MOOC components. D&I competency scores showed strong evidence of improvements from pre- to post-test. The full-scale average score improved by an average of 1.09 points, representing an effect size of 1.25 (CI 0.48-2.00); all four subscales also showed strong evidence of improvements. Trainees reported challenges to MOOC course completion that included technological issues (i.e., limited internet access) and competing demands (i.e., work, family).

CONCLUSIONS

In the context of LMIC training, the MOOC course was feasible and course completion showed improvement in D&I competency scores. While the program was designed with a focus on training for tropical diseases, there is potential for scalability to a wider audience of health care researchers, workers, administrators, and policymakers in LMIC interested in D&I research in non-communicable diseases.

The University of California San Francisco (UCSF) Training Program in Implementation Science: Program Experiences and Outcomes

Purpose: We evaluated outcomes of trainees who have completed the Certificate program in Implementation Science at the University of California San Francisco.

Methods: All students who completed the in-person Certificate Program between 2008 and 2015 (n = 71), or the online Certificate Program between 2016 and 2017 (n = 13), were eligible for our study. We assessed the potential impact of the Certificate Program on the professional development of trainees, through participant surveys on their self-reported level of comfort with pre-defined competencies, and on academic productivity.

Results: Of eligible trainees, 54 in-person (77%) and 13 online (100%) Certificate Program participants completed surveys. In-person trainees reported a total of 147 implementation science-related publications in peer-reviewed journals (median 3 publications/trainee, IQR 1–15). Thirty-four trainees (63%) reported being a Principal Investigator (PI) of 64 funded implementation science-related grants (median 2 grants/trainee, IQR 1–4). Fifteen percent (15%, n = 8) of participants reported receiving an NIH grant on which they were the PI, including R01 or P01 level funding (n = 4, 7%) and K awards (n = 3, 6%). Both in-person and online trainees reported median high to moderate confidence for all 12 competencies assessed. Confidence waned in skills aligning with later stages of implementation research for all trainees.

Conclusion: The moderate to high confidence in all competencies assessed and reported high level of academic productivity support the benefits of intensive, graduate-level training focused on applied methods to support career development of implementation scientists.

Designs and methods for implementation research: Advancing the mission of the CTSA program

INTRODUCTION

The US National Institutes of Health (NIH) established the Clinical and Translational Science Award (CTSA) program in response to the challenges of translating biomedical and behavioral interventions from discovery to real-world use. To address the challenge of translating evidence-based interventions (EBIs) into practice, the field of implementation science has emerged as a distinct discipline. With the distinction between EBI effectiveness research and implementation research comes differences in study design and methodology, shifting focus from clinical outcomes to the systems that support adoption and delivery of EBIs with fidelity.

METHODS

Implementation research designs share many of the foundational elements and assumptions of efficacy/effectiveness research. Designs and methods that are currently applied in implementation research include experimental, quasi-experimental, observational, hybrid effectiveness-implementation, simulation modeling, and configurational comparative methods.

RESULTS

Examples of specific research designs and methods illustrate their use in implementation science. We propose that the CTSA program takes advantage of the momentum of the field's capacity building in three ways: 1) integrate state-of-the-science implementation methods and designs into its existing body of research; 2) position itself at the forefront of advancing the science of implementation science by collaborating with other NIH institutes that share the goal of advancing implementation science; and 3) provide adequate training in implementation science.

CONCLUSIONS

As implementation methodologies mature, both implementation science and the CTSA program would greatly benefit from cross-fertilizing expertise and shared infrastructures that aim to advance healthcare in the USA and around the world.

Advancing health equity through CTSA programs: Opportunities for interaction between health equity, dissemination and implementation, and translational science

Dissemination and implementation (D&I) science is dedicated to studying how to effectively translate and apply research in real-world contexts. There has been increasing interest in health equity within the D&I field to ensure the equitable implementation of evidence-based programs/practices across a range of diverse populations and settings. At the same time, health equity researchers recognize the potential of D&I science to promote the more widespread dissemination, implementation, and sustainment of evidence-based interventions to address health inequities. The National Center for Accelerating Clinical and Translational Science Clinical and Translational Science Award (CTSA) Program has been a champion for community engagement and translational scholarship in its mission to improve individual and population health. The overall CTSA infrastructure and resources within and among CTSA hubs are well-equipped to facilitate a health equity focus to D&I across the phases of translational research. This paper proposes a framework that demonstrates the interaction and opportunities between health equity and D&I science and highlights how CTSAs can support and facilitate wider efforts in translational research with a focus on equitable D&I.

Dissemination and stakeholder engagement practices among dissemination & implementation scientists: Results from an online survey

INTRODUCTION

There has been an increasing focus on disseminating research findings, but less about practices specific to disseminating and engaging non-researchers. The present project sought to describe dissemination practices and engagement of stakeholders among dissemination & implementation (D&I) scientists.

METHODS

Methods to disseminate to and engage non-research stakeholders were assessed using an online survey sent to a broad, diverse sample of D&I scientists.

RESULTS

Surveys were received from 210 participants. The majority of respondents were from university or research settings in the United States. (69%) or Canada (13%), representing a mix of clinical (28%) and community settings (34%). 26% had received formal training in D&I. Respondents indicated routinely engaging in a variety of dissemination-related activities, with academic journal publications (88%), conference presentations (86%), and reports to funders (74%) being the most frequent. Journal publication was identified as the most impactful on respondents' careers (94%), but face-to-face meetings with stakeholders were rated as most impactful on practice or policy (40%). Stakeholder involvement in research was common, with clinical and community-based researchers engaging stakeholder groups in broadly similar ways, but with critical differences noted between researchers with greater seniority, those with more D&I training, those based in the United States vs. Canada, and those in community vs. clinical research settings.

CONCLUSIONS

There have been increases in stakeholder engagement, but few other practices since the 2012 survey, and some differences across subgroups. Methods to engage different stakeholders deserve more in-depth investigation. D&I researchers report substantial misalignment of incentives and behaviors related to dissemination to non-research audiences.

Cardiovascular Nursing Science Priorities: A Statement From the American Heart Association Council on Cardiovascular and Stroke Nursing

BACKGROUND

The American Heart Association's (AHA) Council on Cardiovascular and Stroke Nursing (CVSN) plays a critical role in advancing the mission of the AHA in the discovery of new scientific knowledge. The aim was to identify priority research topics that would promote and improve cardiovascular (CV) health, provide direction for the education of future nurse scientists, and serve as a resource and catalyst for federal and organizational funding priorities.

METHODS

A Qualtrics survey, which included 3 questions about priorities for CVSN nurse researchers, was sent to the CVSN Leadership Committee and all CVSN Fellows of the AHA (n = 208). Responses to the questions were reviewed for word repetitions, patterns, and concepts and were then organized into thematic areas. The thematic areas were reviewed within small groups at the November (2016) in-person CVSN leadership meeting.

RESULTS

Seventy-three surveys were completed. Five thematic areas were identified and included (1) developing and testing interventions, (2) assessment and monitoring, (3) precision CV nursing care, (4) translational and implementation science, and (5) big data. Topic areas noted were stroke, research methods, prevention of stroke and CV disease, self-management, and care and health disparities.

CONCLUSION

Five thematic areas and 24 topic areas were identified as priorities for CV nursing research. These findings can provide a guide for CV nurse scientists and for federal and foundational funders to use in developing funding initiatives. We believe additional research and discovery in these thematic areas will help reduce the rising global burden of CV disease.

Dissemination and implementation science activities across the Clinical Translational Science Award (CTSA) Consortium: Report from a survey of CTSA leaders

INTRODUCTION

Dissemination and implementation (D&I) science is not a formal element of the Clinical Translational Science Award (CTSA) Program, and D&I science activities across the CTSA Consortium are largely unknown.

METHODS

The CTSA Dissemination, Implementation, and Knowledge Translation Working Group surveyed CTSA leaders to explore D&I science-related activities, barriers, and needed supports, then conducted univariate and qualitative analyses of the data.

RESULTS

Out of 67 CTSA leaders, 55.2% responded. CTSAs reported directly funding D&I programs (54.1%), training (51.4%), and projects (59.5%). Indirect support (e.g., promoted by CTSA without direct funding) for D&I activities was higher - programs (70.3%), training (64.9%), and projects (54.1%). Top barriers included funding (39.4%), limited D&I science faculty (30.3%), and lack of D&I science understanding (27.3%). Respondents (63.4%) noted the importance of D&I training and recommended coordination of D&I activities across CTSAs hubs (33.3%).

CONCLUSION

These findings should guide CTSA leadership in efforts to raise awareness and advance the role of D&I science in improving population health.

Training in Implementation Practice Leadership (TRIPLE): evaluation of a novel practice change strategy in behavioral health organizations

Background

Effective leadership for organizational change is critical to the implementation of evidence-based practices (EBPs). As organizational leaders in behavioral health organizations often are promoted from within the agency for their long-standing, effective work as counselors, they may lack formal training in leadership, management, or practice change. This study assesses a novel implementation leadership training designed to promote leadership skills and successful organizational change specific to EBP implementation.

Methods

We conducted a pre-post outcome evaluation of the Training in Implementation Practice Leadership (TRIPLE), delivered via three in-person, half-day training sessions, with interim coaching and technical support. Sixteen mid-level leaders (75% female, 94% Caucasian, mean age 37 years) from 8 substance abuse treatment agencies participated. Professional roles included clinical managers, quality improvement coordinators, and program directors. Participants completed surveys prior to the first and following the final session. At both time points, measures included the Implementation Leadership Scale, Implementation Climate Scale, and Organizational Readiness for Implementing Change Scale. At post-test, we added the Training Acceptability and Appropriateness Scale (TAAS), assessing participant satisfaction with the training. Qualitative interviews were conducted 6 to 8 months after the training.

Results

Most participants (86% and 79%, respectively) reported increased implementation leadership skills and implementation climate; paired samples t tests indicated these pre-post increases were statistically significant. Implementation leadership scores improved most markedly on the Proactive and Knowledgeable subscales. For implementation climate, participants reported the greatest increases in educational support and recognition for using EBP. Post-test scores on the TAAS also indicated that participants found the training program to be highly acceptable and appropriate for their needs. Qualitative results supported positive outcomes of training that resulted in both increased organizational implementation as well as leadership skills of participants.

Conclusions

This training program represents an innovative, effective, and well-received implementation strategy for emerging behavioral healthcare leaders seeking to adopt or improve the delivery of EBPs. Reported implementation leadership skills and implementation climate improved following the training program, suggesting that TRIPLE may have helped fulfill a critical need for emerging behavioral healthcare leaders.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0906-2) contains supplementary material, which is available to authorized users.

Toolkits for dissemination and implementation research: Preliminary development

The Dissemination and Implementation Research Core, a research methods core from the Clinical and Translation Science Award at Washington University in St. Louis Institute of Clinical and Translational Sciences, developed toolkits about dissemination and implementation (D&I) concepts (e.g., D&I outcomes, strategies). This paper reports on the development of the toolkits. These toolkits respond to 3 identified needs for capacity building in D&I research: resources for investigators new to the D&I field, consolidation of tools, and limitations in local contexts.

Barriers to the evaluation of evidence-based public health policy

Public health policy has the potential to produce great benefits for individuals and communities. There is growing demand that such efforts be rigorously evaluated to ensure that the expected benefits are, in fact, realised. Commonly, public health policy is evaluated by consumer acceptability, reach, or changes in knowledge and attitudes. Non-robust research designs are often used. But these approaches to evaluation do not answer three critical questions: Has a change in the desired outcome occurred? Was it a consequence of the policy and not some extraneous factor? Was the size of the change considered significant and cost-effective? We, a team of government and academic scholars working in research and evaluation, have examined some of the more common impediments to robust evaluation: political impediments, a lack of investment in evaluation capacity within bureaucracy, and the failure of academic researchers to understand the need for the evaluation of public health policy.

Longitudinal evaluation of a course to build core competencies in implementation practice

BACKGROUND

Few training opportunities are available for implementation practitioners; we designed the Practicing Knowledge Translation (PKT) to address this gap. The goal of PKT is to train practitioners to use evidence and apply implementation science in healthcare settings. The aim of this study was to describe PKT and evaluate participant use of implementation science theories, models, and frameworks (TMFs), knowledge, self-efficacy, and satisfaction and feedback on the course.

METHODS

PKT was delivered to implementation practitioners between September 2015 and February 2016 through a 3-day workshop, 11 webinars. We assessed PKT using an uncontrolled before and after study design, using convergent parallel mixed methods. The primary outcome was use of TMFs in implementation projects. Secondary outcomes were knowledge and self-efficacy across six core competencies, factors related to each of the outcomes, and satisfaction with the course. Participants completed online surveys and semi-structured interviews at baseline, 3, 6, and 12 months.

RESULTS

Participants (n = 15) reported an increase in their use of implementation TMFs (mean = 2.11; estimate = 2.11; standard error (SE) = 0.4; p = 0.03). There was a significant increase in participants' knowledge of developing an evidence-informed, theory-driven program (ETP) (estimate = 4.10; SE = 0.37; p = 0.002); evidence implementation (estimate = 2.68; SE = 0.42; p < 0.001); evaluation (estimate = 4.43; SE = 0.36; p < 0.001); sustainability, scale, and spread (estimate = 2.55; SE = 0.34; p < 0.001); and context assessment (estimate = 3.86; SE = 0.32; p < 0.001). There was a significant increase in participants' self-efficacy in developing an ETP (estimate = 3.81; SE = 0.34; p < 0.001); implementation (estimate = 3.01; SE = 0.36; p < 0.001); evaluation (estimate = 3.83; SE = 0.39; p = 0.002); sustainability, scale, and spread (estimate = 3.06; SE = 0.46; p = 0.003); and context assessment (estimate = 4.05; SE = 0.38; p = 0.016).

CONCLUSION

Process and outcome measures collected indicated that PKT participants increased use of, knowledge of, self-efficacy in KT. Our findings highlight the importance of longitudinal evaluations of training initiatives to inform how to build capacity for implementers.

Evaluation of the "Foundations in Knowledge Translation" training initiative: preparing end users to practice KT

BACKGROUND

Current knowledge translation (KT) training initiatives are primarily focused on preparing researchers to conduct KT research rather than on teaching KT practice to end users. Furthermore, training initiatives that focus on KT practice have not been rigorously evaluated and have focused on assessing short-term outcomes and participant satisfaction only. Thus, there is a need for longitudinal training evaluations that assess the sustainability of training outcomes and contextual factors that may influence outcomes.

METHODS

We evaluated the KT training initiative "Foundations in KT" using a mixed-methods longitudinal design. "Foundations in KT" provided training in KT practice and included three tailored in-person workshops, coaching, and an online platform for training materials and knowledge exchange. Two cohorts were included in the study (62 participants, including 46 "Foundations in KT" participants from 16 project teams and 16 decision-maker partners). Participants completed self-report questionnaires, focus groups, and interviews at baseline and at 6, 12, 18, and 24 months after the first workshop.

RESULTS

Participant-level outcomes include survey results which indicated that participants' self-efficacy in evidence-based practice (F(1,8.9) = 23.7, p = 0.001, n = 45), KT activities (F(1,23.9) = 43.2, p < 0.001, n = 45), and using evidence to inform practice increased over time (F(1,11.0) = 6.0, p = 0.03, n = 45). Interviews and focus groups illustrated that participants' understanding of and confidence in using KT increased from baseline to 24 months after the workshop. Interviews and focus groups suggested that the training initiative helped participants achieve their KT project objectives, plan their projects, and solve problems over time. Contextual factors include teams with high self-reported organizational capacity and commitment to implement at the start of their project had buy-in from upper management that resulted in secured funding and resources for their project. Training initiative outcomes include participants who applied the KT knowledge and skills they learned to other projects by sharing their knowledge informally with coworkers. Sustained spread of KT practice was observed with five teams at 24 months.

CONCLUSIONS

We completed a longitudinal evaluation of a KT training initiative. Positive participant outcomes were sustained until 24 months after the initial workshop. Given the emphasis on implementing evidence and the need to train implementers, these findings are promising for future KT training.

Training scholars in dissemination and implementation research for cancer prevention and control: a mentored approach

BACKGROUND

As the field of D&I (dissemination and implementation) science grows to meet the need for more effective and timely applications of research findings in routine practice, the demand for formalized training programs has increased concurrently. The Mentored Training for Dissemination and Implementation Research in Cancer (MT-DIRC) Program aims to build capacity in the cancer control D&I research workforce, especially among early career researchers. This paper outlines the various components of the program and reports results of systematic evaluations to ascertain its effectiveness.

METHODS

Essential features of the program include selection of early career fellows or more experienced investigators with a focus relevant to cancer control transitioning to a D&I research focus, a 5-day intensive training institute, ongoing peer and senior mentoring, mentored planning and work on a D&I research proposal or project, limited pilot funding, and training and ongoing improvement activities for mentors. The core faculty and staff members of the MT-DIRC program gathered baseline and ongoing evaluation data regarding D&I skill acquisition and mentoring competency through participant surveys and analyzed it by iterative collective reflection.

RESULTS

A majority (79%) of fellows are female, assistant professors (55%); 59% are in allied health disciplines, and 48% focus on cancer prevention research. Forty-three D&I research competencies were assessed; all improved from baseline to 6 and 18 months. These effects were apparent across beginner, intermediate, and advanced initial D&I competency levels and across the competency domains. Mentoring competency was rated very highly by the fellows--higher than rated by the mentors themselves. The importance of different mentoring activities, as rated by the fellows, was generally congruent with their satisfaction with the activities, with the exception of relatively greater satisfaction with the degree of emotional support and relatively lower satisfaction for skill building and opportunity initially.

CONCLUSIONS

These first years of MT-DIRC demonstrated the program's ability to attract, engage, and improve fellows' competencies and skills and implement a multicomponent mentoring program that was well received. This account of the program can serve as a basis for potential replication and evolution of this model in training future D&I science researchers.