Developing a framework of core competencies in implementation research for low/middle-income countries

Multimethods study to develop tools for competency-based assessments of implementation research training programmes in low and middle-income countries

INTRODUCTION

Methods and validated tools for evaluating the performance of competency-based implementation research (IR) training programmes in low-middle-income countries (LMICs) are lacking. In this study, we developed tools for assessing the performance of IR training programmes based on a framework of IR core competencies in LMICs.

METHODS

We developed self-assessment and objective-assessment tools drawing on the IR competency framework. We used exploratory factor analyses and a one-parameter logistic model to establish construct validity and internal consistency of the tools drawing on a survey conducted in 2020 with 166 trainees before and after an index IR course across five universities and LMICs under the Special Program for Research and Training in Tropical Diseases postgraduate IR training scheme. We conducted key informant interviews (KII) with 20 trainees and five trainers to reflect on the usefulness of the tools and framework for guiding IR training in LMICs.

RESULTS

Two 16-item tools for self-assessment of IR knowledge and self-efficacy and a 40-item objective assessment tool were developed. The factor loadings of items in the self-assessment tools were 0.65-0.87 with Cronbach's alpha (α) of 0.97, and 0.77-0.91 with α of 0.98 for the IR knowledge and self-efficacy tools, respectively. The distribution of item difficulty in the objective-assessment tool was consistent before and after the index IR course. Pearson correlation coefficient (r) between the self-assessed and objectively assessed IR knowledge before the index IR course was low, r=0.27 (p value: <0.01), with slight improvements after the index IR course, r=0.43 (p value: <0.01). All KII respondents reported the assessment tools and framework were valid for assessing IR competencies.

CONCLUSION

The IR competency framework and tools developed for assessing IR competencies and performance of IR training programmes in LMICs are reliable and valid. Self-assessment methods alone may not suffice to yield a valid assessment of performance in these settings.

Closing the gap: advancing implementation science through training and capacity building

In their article on "Navigating the Field of Implementation Science Towards Maturity: Challenges and Opportunities," Chambers and Emmons describe the rapid growth of implementation science along with remaining challenges. A significant gap remains in training and capacity building. Formats for capacity building include university degree programs, summer training institutes, workshops, and conferences. In this letter, we describe and amplify on five key areas, including the need to (1) identify advanced competencies, (2) increase the volume and reach of trainings, (3) sustain trainings, (4) build equity focused trainings, and (5) develop global capacity. We hope that the areas we highlight will aid in addressing several key challenges to prioritize in future efforts to build greater capacity in implementation science.

Regulatory approval of clinical trials: is it time to reinvent the wheel?

Less than 1% of all clinical trials are conducted in Africa. In 2019, only six of 26 oncology clinical trials conducted in Africa were conducted in countries with subjects of African ancestry. There are multiple barriers that hinder the conduct of cancer clinical trials in Africa. Time to trial activation (TTA) is the administrative and regulatory process required before a study can be activated-an important metric and often a major barrier for site selection. In Kenya, TTA involves review by Institutional Review Board (IRB), Pharmacy and Poisons Board, National Commission for Science, Technology and Innovation and Ministry of Health, all in a sequential fashion. We performed a prospective review of TTA for all clinical trials initiated and began enrolment at the Aga Khan University-Clinical Research Unit between June 2020 and November 2022. TTA was defined as total time from submission of study documents (to regulatory bodies) to site activation by the sponsor. A total of 12 studies were submitted for regulatory review. Eleven (nine industry sponsored and two investigator initiated) were approved for activation. Three were COVID-19-related studies and eight were non-COVID-19-related studies. Mean TTA for COVID-related studies was 80 days (range 40-120). Mean TTA for non-COVID-related studies was 259 days (range 190-399). This TTA difference was statistically significant (p=0.02). TTA remains a significant barrier to the efficient regulatory approval of and subsequent conduct of clinical trials in Africa. COVID-19 pandemic revealed that parallel processing and expedited review of clinical trials allows efficient TTA without compromising human subject safety or data integrity. These lessons need to be applied to all clinical trials in order for African sites to become competitive and contribute data from African patients to global knowledge.

The development and application of a two-step surveillance process for Healthy China Initiative based on wide coverage interagency data

BACKGROUND

Healthy China is a nationwide health strategy aiming at improving health from diverse dimensions, and strengthening high-quality assessment is essential for its stimulation. However, there is limited evidence in the surveillance of the actual performance of the initiative at regional levels. This study innovatively proposes a two-step surveillance process which comprehensively monitors Healthy China Initiative based on regional realities, thus provides guidance for policymaking.

METHODS

A flexible indicator system was firstly developed basing on Delphi survey and focus group discussions. And then the Analysis Hierarchical Process and the TOPSIS method were used to determine the weights of indicators and calculate comprehensive indexes as the surveillance outcomes. A pilot study was conducted in a typical area in China to verify the applicability of the process.

RESULTS

Following the surveillance process and basing on the implementation of Healthy China Initiative in the target region, an indicator system comprised of 5 domains and 23 indicators with weights was first developed specifically for the pilot area. Then 1848 interagency data of the study area were collected from 8 provincial institutions/departments to calculate the indexes and ranks of the five domains which were health level, healthy living, disease prevention and control, health service, and healthy environment. The outcomes showed that Healthy China Initiative in the pilot area had been constantly improved since the strategy proposed, while there were still issues to be tackled such as the deficient monitoring mechanisms and unevenly development progress.

CONCLUSIONS

This study proposed a pragmatic surveillance process with indicators which could be tailored for specific context of target regions and produce meaningful surveillance outcomes to inform decision-making for policymakers, and also provided a theoretical foundation as well as empirical evidence for further health strategies and plannings assessment studies.

Developing Capacity in Dissemination and Implementation Research in the Eastern Mediterranean Region: Evaluation of a Training Workshop

As the demand for dissemination and implementation (D&I) research grows globally, there is a need for D&I capacity building in regions where D&I science is underrepresented. The Workshop on Dissemination and Implementation Research in Health (WONDIRH) was aimed for participants in the Eastern Mediterranean region to (1) appreciate the complex process of bridging research and practice in a variety of real-world settings, and (2) develop research that balances rigor with relevance and employs study designs and methods appropriate for the complex processes involved in D&I. The present exploratory study investigates participants' satisfaction with the workshop, the enhancement of their self-rated confidence in D&I skills, as well as their intention to apply the learned content into practice. The workshop included four weekly 90-min virtual interactive training sessions in conjunction with open access content from the National Cancer Institute Training Institute in Implementation and Dissemination Research in Cancer (TIDIRC). We applied a one-group pre-post design for the evaluation of workshop. Participants were invited to self-rate their confidence in D&I competencies (15 items, pre and post workshop). At the end of the workshop, participants additionally were asked to rate their satisfaction (5 items, 1-5 scales), and their intention to apply the learned content into practice (4 items, 1-5 scales). Of the 77 workshop participants, 34 completed the evaluation. Confidence improved between pre- and post-workshop assessments in all 15 self-rated D&I competencies. Respondents were generally satisfied with the workshop (mean satisfaction range 3.82-4.26 across the 5 items) and endorsed intentions to apply workshop topics (mean intention range 4.03-4.35 across the 4 items). This initial workshop demonstrated the ability to attract and engage participants to enhance their confidence in D&I research competencies and skills and to build capacity in D&I research. Future efforts should consider offering targeted training for researchers at different stages and to clearly articulate learning objectives.

Supplementary Information

The online version contains supplementary material available at 10.1007/s43477-022-00067-y.

Diversifying Implementation Science: A Global Perspective

We present a joint global perspective about the urgent need to diversify the loci of knowledge creation and sharing in global implementation science. We underscore the imperative of addressing implementation research questions relevant to practitioners, policy makers, and researchers from low- and middle-income countries.

Learning needs assessment for multi-stakeholder implementation science training in LMIC settings: findings and recommendations

Background

Despite significant progress in the field of implementation science (IS), current training programs are inadequate to meet the global need, especially in low-and middle-income countries (LMICs). Even when training opportunities exist, there is a “knowledge-practice gap,” where implementation research findings are not useful to practitioners in a field designed to bridge that gap. This is a critical challenge in LMICs where complex public health issues must be addressed. This paper describes results from a formal assessment of learning needs, priority topics, and delivery methods for LMIC stakeholders.

Methods

We first reviewed a sample of articles published recently in Implementation Science to identify IS stakeholders and assigned labels and definitions for groups with similar roles. We then employed a multi-step sampling approach and a random sampling strategy to recruit participants (n = 39) for a semi-structured interview that lasted 30–60 min. Stakeholders with inputs critical to developing training curricula were prioritized and selected for interviews. We created memos from audio-recorded interviews and used a deductively created codebook to conduct thematic analysis. We calculated kappa coefficients for each memo and used validation techniques to establish rigor including incorporating feedback from reviewers and member checking.

Results

Participants included program managers, researchers, and physicians working in over 20 countries, primarily LMICs. The majority had over 10 years of implementation experience but fewer than 5 years of IS experience. Three main themes emerged from the data, pertaining to past experience with IS, future IS training needs, and contextual issues. Most respondents (even with formal training) described their IS knowledge as basic or minimal. Preferences for future training were heterogeneous, but findings suggest that curricula must encompass a broader set of competencies than just IS, include mentorship/apprenticeship, and center the LMIC context.

Conclusion

While this work is the first systematic assessment of IS learning needs among LMIC stakeholders, findings reflect existing research in that current training opportunities may not meet the demand, trainings are too narrowly focused to meet the heterogeneous needs of stakeholders, and there is a need for a broader set of competencies that moves beyond only IS. Our research also demonstrates the timely and unique needs of developing appropriately scoped, accessible training and mentorship support within LMIC settings. Therefore, we propose the novel approach of intelligent swarming as a solution to help build IS capacity in LMICs through the lens of sustainability and equity.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-021-00238-2.

Increasing Knowledge Translation Capacity in Low- and Middle-Income Countries: A Model for Implementation Research Training

Most health professionals lack the training and expertise to translate clinical innovations into actionable programs. Even though some public health expert communities understand that even widely proven solutions need to be adapted to the demands and characteristics of diverse health systems and societies to be successful, such knowledge has yet to inform routine public health approaches and practices. Therefore, it should not be a surprise that the “know-do” gap between clinical innovations and their on-the-ground application that implementation research seeks to bridge is pervasive and enduring, particularly in low- and middle-income countries. This article draws on a study of implementation research training courses to highlight the various competencies needed to translate different types of knowledge into action, many of which are not adequately addressed in existing curricula. We utilized a four-phase modified Delphi methodology that included a review of the academic and grey literature, one-on-one interviews with experts, virtual dialogue series with key stakeholders, and peer review of the synthesized results. The resulting areas in need of further development include the ability of learners to work as part of a multidisciplinary team, engage various stakeholders, and communicate research findings to decision-makers. Based on these insights, it is argued that knowledge translation in implementation research is a multi-faceted, multi-level sensemaking and communication activity that takes place throughout the research and research-to policy-processes.

Building Implementation Science Capacity in Academic Pharmacy: Report of the 2020-2021 AACP Research and Graduate Affairs Committee

EXECUTIVE SUMMARY The 2020-2021 AACP Research and Graduate Affairs Committee (RGAC) continued the work begun by the 2019-2020 RGAC to increase awareness of and capacity for implementation research to advance practice transformation in academic pharmacy. AACP President Anne Lin charged the RGAC with developing resources and programs for training faculty and graduate students in implementation science. The committee was further charged with developing a mechanism to pair pharmacy faculty and implementation experts on practice advancement projects. In its work, the committee focused on generating near-term opportunities for pharmacy practice faculty to pursue projects while developing programs that would support ongoing career development and future implementation practice and research by pharmacy faculty and trainees.

Translating Implementation Experiences and Lessons Learned From Polio Eradication Into a Global Health Course: Insights From an International Consortium

Using international collaborations to develop educational materials presents several challenges but offers enormous benefits in gleaning a wealth of information, perspectives, and context. The global course that resulted from this collaboration mirrors the goals of implementation science more broadly—to bring the findings of research into routine practice to improve health services.

Lessons learned from one global health program can inform responses to challenges faced by other programs. One way to disseminate these lessons is through courses. However, such courses are often delivered by and taught to people based in high-income countries and thus may not present a truly global perspective. The Synthesis and Translation of Research and Innovations from Polio Eradication (STRIPE) is a consortium of 8 institutions in Afghanistan, Bangladesh, the Democratic Republic of the Congo, Ethiopia, India, Indonesia, Nigeria, and the United States that seeks to carry out such a transfer of the lessons learned in polio eradication. This short report describes the collaborative process of developing content and curriculum for an international course, the learnings that emerged, the barriers we faced, and recommendations for future similar efforts. Various parts of our course were developed by teams of researchers from countries across South Asia and sub-Saharan Africa. We held a series of regional in-person team meetings hosted in different countries to improve rapport and provide a chance to work together in person. The course content reflects the diversity of team members’ knowledge in a variety of contexts. Challenges to this effort included team coordination (e.g., scheduling across time zones); hierarchies across and between countries; and the coronavirus disease (COVID-19) pandemic. We recommend planning for these hierarchies ahead of time and ensuring significant in-person meeting time to make the most of international collaboration.

Barriers and facilitators to knowledge translation activities within academic institutions in low- and middle-income countries

The barriers and facilitators of conducting knowledge translation (KT) activities are well-established but less is known about the institutional forces that drive these barriers, particularly in low resource settings. Understanding organizational readiness has been used to assess and address such barriers but the employment of readiness assessments has largely been done in high-income countries. We conducted a qualitative study to describe the institutional needs and barriers in KT specific to academic institutions in low- and middle-income countries. We conducted a review of the grey and published literature to identify country health priorities and established barriers and facilitators for KT. Key-informant interviews (KII) were conducted to elicit perceptions of institutional readiness to conduct KT, including experiences with KT, and views on motivation and capacity building. Participants included representatives from academic institutions and Ministries of Health in six countries (Bangladesh, Democratic Republic of the Congo, Ethiopia, India, Indonesia, Nigeria). We conducted 18 KIIs, 11 with members of academic institutions and 7 with policymakers. KIIs were analysed using a deductive and inductive coding approach. Our findings support many well-documented barriers including lack of time, skills and institutional support to conduct KT. Three additional institutional drivers emerged around soft skills and the complexity of the policy process, alignment of incentives and institutional missions, and the role of networks. Participants reflected on often-lacking soft-skills needed by researchers to engage policy makers. Continuous engagement was viewed as a challenge given competing demands for time (both researchers and policy makers) and lack of institutional incentives to conduct KT. Strong networks, both within the institution and between institutions, were described as important for conducting KT but difficult to establish and maintain. Attention to the cross-cutting themes representing barriers and facilitators for both individuals and institutions can inform the development of capacity building strategies that meet readiness needs.

Designing the next generation of implementation research training for learners in low- and middle-income countries

Efficacious health interventions tested through controlled trials often fail to show desired impacts when implemented at scale. These challenges can be particularly pervasive in low- and middle-income settings where health systems often lack the capacity and mechanisms required for high-quality research and evidence translation. Implementation research is a powerful tool for identifying and addressing the bottlenecks impeding the success of proven health interventions. Implementation research training initiatives, although growing in number, remain out of reach for many investigators in low- and middle-income settings, who possess the knowledge required to contextualize challenges and potential solutions in light of interacting community- and system-level features. We propose a realigned implementation research training model that centers on team-based learning, tailored didactic opportunities, learning-by-doing, and mentorship.

An assessment of implementation science research capacity in Uganda

Background

In Uganda and other resource-poor countries, relevant research findings face a tortuous path to translation into policy and routine practice. Implementation science (ImSc) research could facilitate faster translation. Presently it is unclear what ImSc research capacity and possible training needs exist among Ugandan researchers. To assess both components, we interviewed potential trainees in Kampala, Uganda.

Methods

We used a cross-sectional design to survey potential ImSc trainees who had some research training and involvement in generating or utilizing research. Using a questionnaire, we documented eligibility for ImSc training, knowledge and interest in training, existing self-assessed confidence in initiating clinical research (SCICR) and self-assessed confidence in initiating ImSc research (SCIIR), availability for training and preferred modes of training. We developed scores from the Likert scales and used descriptive statistics, logistic regression and ordinal logistic regression to evaluate predictors of SCIIR.

Results

Between November 2016 and April 2017, we interviewed 190 participants; 60% were men, with a median age of 37 years. Among participants, 33% comprised faculty, 37% were graduate students and 30% were project staff. The majority of respondents knew about ImSc (73%) and were research-trained (80%). Only 9% reported any ImSc-related training. Previous ImSc training was associated with higher odds of a SCIIR score ≥ 75th percentile. Previous ImSc training compared to not having any training was associated with higher odds of reporting abilities in behaviour change theory integration (OR: 3.3, 95% CI: 1.3–8.5, p = 0.01) and framework use in intervention design and implementation (OR: 2.9, 95% CI: 1.1–7.4, p = 0.03), accounting for age, sex and current employment. In addition, 53% of participants preferred in-person (face-to-face) short ImSc courses compared to a year-long training, while 33% preferred online courses. Participants reported median availability of 6 hours per week (IQR: 4, 10) for training.

Conclusion

Most participants had some understanding of ImSc research, had research training and were interested in ImSc training. Those with previous ImSc training had better skills and SCIIR, compared to those without previous training. A hybrid approach with modular face-to-face training and online sessions would suit the preferences of most potential trainees.

Identifying core competencies for practicing public health professionals: results from a Delphi exercise in Uttar Pradesh, India

BACKGROUND

Ensuring the current public health workforce has appropriate competencies to fulfill essential public health functions is challenging in many low- and middle-income countries. The absence of an agreed set of core competencies to provide a basis for developing and assessing knowledge, skills, abilities, and attitudes contributes to this challenge. This study aims to identify the requisite core competencies for practicing health professionals in mid-level supervisory and program management roles to effectively perform their public health responsibilities in the resource-poor setting of Uttar Pradesh (UP), India.

METHODS

We used a multi-step, interactive Delphi technique to develop an agreed set of public health competencies. A narrative review of core competency frameworks and key informant interviews with human resources for health experts in India were conducted to prepare an initial list of 40 competency statements in eight domains. We then organized a day-long workshop with 22 Indian public health experts and government officials, who added to and modified the initial list. A revised list of 54 competency statements was rated on a 5-point Likert scale. Aggregate statement scores were shared with the participants, who discussed the findings. Finally, the revised list was returned to participants for an additional round of ratings. The Wilcoxon matched-pairs signed-rank test was used to identify stability between steps, and consensus was defined using the percent agreement criterion.

RESULTS

Stability between the first and second Delphi scoring steps was reached in 46 of the 54 statements. By the end of the second Delphi scoring step, consensus was reached on 48 competency statements across eight domains: public health sciences, assessment and analysis, policy and program management, financial management and budgeting, partnerships and collaboration, social and cultural determinants, communication, and leadership.

CONCLUSIONS

This study produced a consensus set of core competencies and domains in public health that can be used to assess competencies of public health professionals and revise or develop new training programs to address desired competencies. Findings can also be used to support workforce development by informing competency-based job descriptions for recruitment and performance management in the Indian context, and potentially can be adapted for use in resource-poor settings globally.

Harnessing the health systems strengthening potential of quality improvement using realist evaluation: an example from southern Tanzania

Quality improvement (QI) is a problem-solving approach in which stakeholders identify context-specific problems and create and implement strategies to address these. It is an approach that is increasingly used to support health system strengthening, which is widely promoted in Sub-Saharan Africa. However, few QI initiatives are sustained and implementation is poorly understood. Here, we propose realist evaluation to fill this gap, sharing an example from southern Tanzania. We use realist evaluation to generate insights around the mechanisms driving QI implementation. These insights can be harnessed to maximize capacity strengthening in QI and to support its operationalization, thus contributing to health systems strengthening. Realist evaluation begins by establishing an initial programme theory, which is presented here. We generated this through an elicitation approach, in which multiple sources (theoretical literature, a document review and previous project reports) were collated and analysed retroductively to generate hypotheses about how the QI intervention is expected to produce specific outcomes linked to implementation. These were organized by health systems building blocks to show how each block may be strengthened through QI processes. Our initial programme theory draws from empowerment theory and emphasizes the self-reinforcing nature of QI: the more it is implemented, the more improvements result, further empowering people to use it. We identified that opportunities that support skill- and confidence-strengthening are essential to optimizing QI, and thus, to maximizing health systems strengthening through QI. Realist evaluation can be used to generate rich implementation data for QI, showcasing how it can be supported in ‘real-world’ conditions for health systems strengthening.

Strengthening the core health research capacity of national health systems helps build country resilience to epidemics: a cross-sectional survey

Background:  TDR, The Special Programme for Research and Training hosted at the World Health Organization, has long supported Low- and Middle-Income Countries in strengthening research capacity through three training programmes: the Postgraduate Training Scheme (PGTS), the Clinical Research and Development Fellowship (CRDF), and the Structured Operational Research Training InitiaTive (SORT IT). In the advent of the COVID-19 pandemic, we assessed whether those trained through these programmes were involved in the COVID-19 response and if so, in which area(s) of the emergency response they were applying their skills. Methods: From the records for each training programme, we identified the individuals who had completed training during the relevant timespan of each programme: 1999-2018 for the CRDF scheme, 2015-2020 for PGTS, and 2009-2019 for SORT-IT. Between March and April 2020, we sent trainees an online questionnaire by e-mail. Results: Out of 1254 trained, 1143 could be contacted and 699 responded to the survey. Of the latter, 411 were involved with the COVID-19 response, of whom 315 (77%) were applying their acquired skills in 85 countries. With some overlap between programmes, 84% of those trained through CRDF were applying their skills in 27 countries, 91% of those trained through PGTS were applying their skills in 19 countries, and through SORT IT, this was 73% in 62 countries.  Skills were being applied in various areas of the emergency response, including: emergency preparedness, situation analysis/surveillance, infection control and clinical management, data generation, mitigating the effect of COVID on the health system, and research.  Depending on the type of training programme, 26-74% were involved in implementation, operational or clinical research. Conclusion: Research training programmes build research capacity and equip health workers with transferable core competencies and skillsets prior to epidemics. This becomes invaluable in building health system resilience at a time of pandemics.

Strengthening the core health research capacity of national health systems helps build country resilience to epidemics: a cross-sectional survey

Background:  TDR, The Special Programme for Research and Training hosted at the World Health Organization, has long supported Low- and Middle-Income Countries in strengthening research capacity through three training programmes: the Postgraduate Training Scheme (PGTS), the Clinical Research and Development Fellowship (CRDF), and the Structured Operational Research Training InitiaTive (SORT IT). In the advent of the COVID-19 pandemic, we assessed whether those trained through these programmes were involved in the COVID-19 response and if so, in which area(s) of the emergency response they were applying their skills. Methods: From the records for each training programme, we identified the individuals who had completed training during the relevant timespan of each programme: 1999-2018 for the CRDF scheme, 2015-2020 for PGTS, and 2009-2019 for SORT-IT. Between March and April 2020, we sent trainees an online questionnaire by e-mail. Results: Out of 1254 trained, 1143 could be contacted and 699 responded to the survey. Of the latter, 411 were involved with the COVID-19 response, of whom 315 (77%) were applying their acquired skills in 85 countries. With some overlap between programmes, 84% of those trained through CRDF were applying their skills in 27 countries, 91% of those trained through PGTS were applying their skills in 19 countries, and through SORT IT, this was 73% in 62 countries.  Skills were being applied in various areas of the emergency response, including: emergency preparedness, situation analysis/surveillance, infection control and clinical management, data generation, mitigating the effect of COVID on the health system, and research.  Depending on the type of training programme, 26-74% were involved in implementation, operational or clinical research. Conclusion: Research training programmes build research capacity and equip health workers with transferable core competencies and skillsets prior to epidemics. This becomes invaluable in building health system resilience at a time of pandemics.