Evaluations of training programs to improve human resource capacity for HIV, malaria, and TB control: a systematic scoping review of methods applied and outcomes assessed

Evaluation of the Training Program to Train HIV Treatment Center Staff in Pakistan

Introduction In Pakistan, HIV training programs, especially for health professionals working in HIV treatment centers, are limited. Consequently, there is little data about HIV awareness among physicians and allied health workers and how it may affect their care for people living with HIV (PLWH). Recently, the Global Fund to Fight AIDS, Tuberculosis, and Malaria (GFATM) grant Principal Recipient UNDP engaged an NGO experienced in HIV/AIDS training, on a competitive basis, to develop a training manual and conduct training of all categories of HIV treatment centers staff. The goal of this study was to assess the training program's influence on trainees' (both physicians and allied health staff) knowledge and abilities and describe its major lessons. Methodology This was a one-group pre-post test study, carried out between January 17 and February 22, 2023. The study was carried out in three phases. In the first phase, a team of experts developed an antiretroviral treatment (ART) training manual. In the second phase, 9- and three-day training workshops were conducted in six different cities of Pakistan, which were attended by physicians and allied health staff working in different HIV treatment centers across Pakistan. The workshops had plenary lectures, discussions, role plays, video cases, and case studies. In the third phase, a quiz, comprising multiple/best choice questions (MCQs/BCQs) and true and false questions, was administered before (pre) and after the workshop (post) to assess the impact of these training sessions in enhancing the level of HIV knowledge, especially related to ART. The workshop was attended by a total of 256 health workers from different cities in Pakistan. The participants had backgrounds in medical science, psychology, laboratory science, nursing, and computer science. Pre-and post-test responses were statistically analyzed to determine the impact of the training program on participant's knowledge. For this, the Shapiro-Wilk test was applied to test data normality, followed by the application of paired t-test or Wilcoxon Signed Rank Test for normally and non-normally distributed data, respectively. Finally, a chi-square test was applied to examine the significant (p<0.05) association between training workshops and improvement in the participant's level of understanding of HIV. In all statistical tests, p<0.05 was considered significant.  Results The results from our study showed that before the training session, both physicians and allied staff possessed limited knowledge about HIV-related domains. After the workshops, participants from all cities demonstrated a uniform enhancement of knowledge related to different HIV-related domains, evident from the improvement in post-test scores compared to pre-test scores (p<0.0001). The chi-square test showed a significant association between training workshops and improvement in the participant's level of understanding about HIV (p-values for BCQ, MCQ, and true and false: 0.001, 0.0047, and 0.0024, respectively). Conclusions Pre- and post-test evaluation provides an objective, data-driven method for measuring the impact of educational interventions in improving healthcare workers' awareness about HIV. The results emphasize the role of continuous workshops and training programs in enhancing the knowledge and understanding of healthcare and allied health workers regarding HIV.

Training strategies practiced for TB elimination

Training is the backbone of any public health program and it is true for a vast program like TB. It is urgent when the program is aiming to End TB. The strategy that is followed in India for capacity building of TB workers is presented in this article. Various types of trainings that are needed are described in detail. Also enlisted are the different trainings undertaken at NTI for the last five years. Recent times the effect of Covid-19 has resulted in the acceleration of the effort of going for digital platforms and onlinetrainings and is described.

Longitudinal analysis of the capacities of community health workers mobilized for seasonal malaria chemoprevention in Burkina Faso

Background

Seasonal malaria chemoprevention (SMC) relies on community health workers to distribute drugs. This study assessed: (1) the capacity of community-based distributors (CBDs) at the start and end of a campaign and from one campaign to another after training or refresher courses before each round; (2) to what extent CBDs’ experience over several campaigns contributed to measurable increase in their capacities; and (3) to what extent the training and experience of committed CBDs helped the less productive to catch up.

Methods

A longitudinal analysis was conducted in one Burkina Faso health district during the 2017 and 2018 campaigns. A panel including all CBDs was created. Their capacities were observed after: (1) initial training for the 2017 season; (2) refresher training for that year’s fourth round; and (3) initial training for the 2018 season. All were invited to complete a questionnaire at the end of training with 27 multiple-choice questions on their main tasks. Observers noted content coverage and conditions under which training sessions were conducted.

Results

The 612 CBDs showed, on average, high understanding of their tasks from the start of the annual campaigns. Tasks related to communicating with parents and reporting were best mastered. Their capacities grew from round to round and campaign to campaign, after most had undergone training and been supervised by head nurses. The greatest progress was in the technical components, considered more complex, which involved selecting eligible children, choosing the correct drug packet, and referring children to health professionals. Retaining CBDs from one round to the next benefited everyone, whatever their starting level. Groups that initially obtained the lowest scores (women, illiterates, youngest/oldest) progressed the most.

Conclusion

These results confirm the potential of using CBDs under routine programme implementation. Mandating CBDs with targeted tasks is a functional model, as they achieve mastery in this context where investments are made in training and supervision. Losing this specificity by extending CBDs’ mandates beyond SMC could have undesirable consequences. The added value of retaining committed CBDs is high. It is suggested that motivation and commitment be considered in recruitment, and that a supportive climate be created to foster retention.

The influence of quality of work life on motivation and retention of local government tuberculosis control programme supervisors in South-eastern Nigeria

Introduction

Significant gap exists in knowledge about employee-centred human resources practices that address motivation and retention of local government tuberculosis control programme supervisors (LGTBS) in Nigeria. The study examined the role of quality of worklife (QWL) in motivating and retaining LGTBS.

Materials and methods

The study was conducted in south-eastern region of Nigeria comprising five states and 95 local government areas. The design was mixed-methods. We used cross-sectional survey to collect quantitative data on socio-demographic factors, QWL, motivation and retention from a total sample of LGTBS. The qualitative component involved focus group discussions (n = 3) with 26 LGTBS. Quantitative data were analysed using exploratory factor analysis, descriptive statistics, Spearman correlation, Mann-Whitney test, Kruskal-Wallis test and multiple linear regression. Qualitative data were analysed using a thematic framework approach.

Results

The final 40-item QWL scale was found to be valid and reliable. The LGTBS had high QWL (M = 5.15, SD = 0.88) and motivation (M = 5.92, SD = 1.08), but low intention to leave their jobs (M = 2.68, SD = 1.59). Education significantly predicted satisfaction with overall QWL, work-family balance and work design; but tenure predicted satisfaction with work context. Work design and work-family balance significantly predicted motivation of LGTBS. Motivation mediated the relationship between QWL and intention to leave and accounted for 29% variance in intention to leave. Whereas LGTBS were motivated by responsibility, learning opportunities, achievement and recognition; they were dissatisfied with lack of flexible work schedules, involvement in non-TB tasks, long hours at work, limited opportunities for vacation, resource inadequacy, work-related stigma, lack of promotional opportunities, and pay disparity and delay.

Conclusion

Addressing work design, work-family balance and working conditions may increase the motivation and retention of LGTBS and improve human resources for TB at the district level and performance of the TB control programme.

Training for Tuberculosis Elimination in Indonesia: Achievements, Reflections, and Potential for Impact

Indonesia has the third highest tuberculosis (TB) caseload internationally. A cornerstone for strengthening health systems to respond to TB is a well-trained workforce. In a partnership between Indonesian and Australian institutions, TB training was run during 2018 to strengthen the local capacity to meet End TB strategy targets. This paper aims to report on course design, delivery, training outcomes, and reflections. Seventy-six Indonesian healthcare workers, program staff, researchers, and policy-makers were selected from over 800 applicants. The structure comprised three trainings, each with a pre-course workshop (in Indonesia) to identify learning needs, a two-week block (Australia), and a post-course workshop (Indonesia). The training content delivered was a combination of TB technical knowledge and program/project theory, design, and logic, and the training utilised multiple teaching and learning methods. An innovative element of the training was participant-designed TB workplace projects focusing on context-specific priorities. Evaluation was undertaken using participant surveys and appraisal of the projects. Participants rated the course highly, while success in project implementation varied. Reflections include the importance of involving Indonesian experts in delivery of training, the need to understand participant learning requirements and adapt the training content accordingly, and the challenge of measuring tangible training outputs.

Quality of Malaria Case Management and Reporting at Public Health Facilities in Six Health Districts in Guinea, 2018

Data on fever and malaria cases reported by health facilities are used for tracking incidence and quantification of malaria commodity needs in Guinea. Periodic assessments of the quality of malaria case management and routine data are a critical activity for the malaria program. In May-June 2018, survey teams visited 126 health facilities in six health districts purposefully selected to represent a spectrum (Stratum 1-high, Stratum 2-intermediate, and Stratum 3-low) of perceived quality of case management and reporting, as assessed from an a priori analysis of routine data. Surveyors performed exit interviews with 939 outpatients and compared results with registry data for interviewed patients. Availability of rapid diagnostic tests (RDTs) and artemisinin-based combination therapies (ACTs) was 100% in Strata 1 and 2, compared with 82% (95% CI: 63-92%) for RDTs and 86% (68-95%) for any formulation of ACT in Stratum 3. Correct case management for suspect malaria cases was 85% in both Stratum 1 (95% CI: 78-90%) and Stratum 2 (79-89%), but only 52% (37-67%) in Stratum 3. Concordance between exit interviews and registry entries for key malaria indicators was significantly higher in Strata 1 and 2 than in Stratum 3. Both adherence to national guidelines for testing and treatment and data quality were high in Strata 1 and 2, but substandard in Stratum 3. The survey results reflected the trends seen in the routine data, suggesting that analysis of routine data can identify areas requiring more attention to improve malaria case management and reporting.

Capacity Building for Public Health: Participant-Guided Training

Growth in the demand for public health services, along with limited funding, makes workforce collaboration and capacity building imperative. The faculty and staff of the Midwestern Public Health Training Center, with two Robert Wood Johnson Public Health Nurse Leaders, postulated that training could be more effective, and public health workers more effective in the field, if workers contributed to training format and content. The learning paradigm was tested on diabetes prevention and self-management programs. Public health professionals were surveyed on infrastructure, practices, roles, and gaps in diabetes-related services. Responses influenced the format and content of a one-day diabetes summit training program. Participants submitted evaluations immediately afterward. Eight months postsummit, participants were surveyed to self-assess behavioral changes attributed to the training. Using the Kirkpatrick model for evaluation, participants (n = 112) stated that the training met their expectations and that knowledge gained was consistent with stated training objectives. Qualitative postsummit survey results indicated that improvements in participants' delivery of diabetes prevention services to the public could be attributed to the training they received at the summit. Results suggest that training about specific programs and practices, as well as facilitated sessions of collaboration, can yield individual and organizational change.

Is knowledge retained by healthcare providers after training? A pragmatic evaluation of drug-resistant tuberculosis management in China

OBJECTIVES

Considering the urgent need of training to improve standardised management of drug-resistant infectious disease and the lack of evidence on the impact of training, this study evaluates whether training participants' knowledge on multidrug-resistant tuberculosis (MDR-TB) is improved immediately and a year after training.

SETTING AND PARTICIPANTS

The study involved 91 MDR-TB healthcare providers (HCPs), including clinical doctors, nurses and CDC staff, who attended a new MDR-TB HCP training programme in Liaoning and Jiangxi provinces, China.

MAIN OUTCOME MEASURES

A phone-based assessment of participants' long-term retention of knowledge about MDR-TB management was conducted in July 2017, approximately 1 year after training. The proportion of correct responses in the long-term knowledge assessment was compared with a pretraining test and an immediate post-training test using a χ2 test. Factors influencing participants' performance in the long-term knowledge assessment were analysed using linear regression.

RESULTS

Across both provinces, knowledge of definitions of drug-resistant TB, standardised MDR-TB case detection protocols and laboratory diagnosis was improved 1 year after the training by 14.5% (p=0.037), 32.4% (p<0.001) and 31% (p<0.001) relative to pretraining. However, compared with immediately after training, the knowledge of the three topics declined by 26.5% (p=0.003), 19.8% (p=0.018) and 52.7% (p<0.001) respectively in Jiangxi, while no significant decline was observed in Liaoning. Additionally, we found that obtaining a higher score in the long-term knowledge assessment was associated with longer years of clinical experience (coefficient=0.51; 95 CI% 0.02 to 0.99; p=0.041) and attending training in Liaoning (coefficient=0.50; 95% CI 0.14 to 0.85; p=0.007).

CONCLUSION

Our study, the first to assess knowledge retention of MDR-TB HCPs 1 year after training, showed an overall positive long-term impact of lecture-style group training on participants' knowledge. Knowledge decline 1 year after training was observed in one province, Jiangxi, and this may be partly addressed by targeted support to HCPs with fewer years of clinical experience.

Designing evaluation studies to optimally inform policy: what factors do policy-makers in China consider when making resource allocation decisions on healthcare worker training programmes?

BACKGROUND

In light of the gap in evidence to inform future resource allocation decisions about healthcare provider (HCP) training in low- and middle-income countries (LMICs), and the considerable donor investments being made towards training interventions, evaluation studies that are optimally designed to inform local policy-makers are needed. The aim of our study is to understand what features of HCP training evaluation studies are important for decision-making by policy-makers in LMICs. We investigate the extent to which evaluations based on the widely used Kirkpatrick model - focusing on direct outcomes of training, namely reaction of trainees, learning, behaviour change and improvements in programmatic health indicators - align with policy-makers' evidence needs for resource allocation decisions. We use China as a case study where resource allocation decisions about potential scale-up (using domestic funding) are being made about an externally funded pilot HCP training programme.

METHODS

Qualitative data were collected from high-level officials involved in resource allocation at the national and provincial level in China through ten face-to-face, in-depth interviews and two focus group discussions consisting of ten participants each. Data were analysed manually using an interpretive thematic analysis approach.

RESULTS

Our study indicates that Chinese officials not only consider information about the direct outcomes of a training programme, as captured in the Kirkpatrick model, but also need information on the resources required to implement the training, the wider or indirect impacts of training, and the sustainability and scalability to other settings within the country. In addition to considering findings presented in evaluation studies, we found that Chinese policy-makers pay close attention to whether the evaluations were robust and to the composition of the evaluation team.

CONCLUSIONS

Our qualitative study indicates that training programme evaluations that focus narrowly on direct training outcomes may not provide sufficient information for policy-makers to make decisions on future training programmes. Based on our findings, we have developed an evidence-based framework, which incorporates but expands beyond the Kirkpatrick model, to provide conceptual and practical guidance that aids in the design of training programme evaluations better suited to meet the information needs of policy-makers and to inform policy decisions.

Strengthening health systems in Africa: a case study of the Kenya field epidemiology training program for local frontline health workers

The logistical and operational challenges to improve public health practice capacity across Africa are well documented. This report describes Kenya's Field Epidemiology and Laboratory Training Program's (KFELTP) experience in implementing frontline public health worker training to transfer knowledge and practical skills that help strengthen their abilities to detect, document, respond to, and report unusual health events. Between May 2014 and May 2015, KFELTP hosted five training courses across the country to address practice gaps among local public health workers. Participants completed a 10-week process: two 1-week didactic courses, a 7-week field project, and a final 1-week course to present and defend the findings of their field project. The first year was a pilot period to determine whether the program could fit into the existing 2-year KFELTP model and whether this frontline-level training would have an impact on local practice. At the end of the first year, KFELTP certified 167 frontline health workers in field epidemiology and data management. This paper concludes that local, national, and international partnerships are critical for improving local public health response capacity and workforce development training in an African setting.