Training and capacity building evaluation: Maximizing resources and results with Success Case Method

Evaluation of the Care Pathway in the Context of the Dispensing of Emicizumab (Hemlibra) in Community Pharmacies in France: Protocol for a Cross-sectional Study Based on the Kirkpatrick Model

BACKGROUND

Commercialized since 2019, emicizumab (Hemlibra) was available only in French hospital pharmacies for prophylaxis of hemophilia A with or without inhibitors. Since June 15, 2021, patients can choose between a hospital and community pharmacy. These changes in the care pathway have important organizational consequences for patients, their relatives, and health professionals. Two training programs are available for community pharmacists: the "HEMOPHAR" training program proposed by the national reference center for hemophilia and the Roche training program proposed by the laboratory that markets the product.

OBJECTIVE

The PASODOBLEDEMI study aims to evaluate the direct impact of the training programs provided to community pharmacists in the context of the dispensing of emicizumab, and to evaluate patients' satisfaction with their treatment whether they choose dispensation from a community pharmacy or retained dispensation from the hospital pharmacy.

METHODS

We designed a cross-sectional study based on the 4-level Kirkpatrick evaluation model: the immediate reaction of community pharmacists following training (Reaction), the knowledge acquired during the training (Learning), the professional practice of community pharmacists during dispensing of the product (Behavior), and patients' satisfaction related to the treatment whether it is dispensed from a hospital or from a community pharmacy (Results).

RESULTS

Considering that single outcome measures cannot adequately reflect the complexity of this new organization, the Kirkpatrick evaluation model provides 4 distinct outcomes: the immediate reaction after the HEMOPHAR training program, the level of knowledge acquired after the HEMOPHAR training program, the impact of training on professional practice, and patient satisfaction with access to emicizumab. We developed specialized questionnaires for each of the 4 levels of the Kirkpatrick evaluation model. All community pharmacists involved in dispensing emicizumab, whether they have followed the HEMOPHAR or the Roche training program or neither, were eligible for inclusion. All patients with severe hemophilia A were eligible, irrespective of inhibitor use, age, treatment with emicizumab, and whether they chose dispensation from a community pharmacy or retained dispensation from a hospital pharmacy.

CONCLUSIONS

The new organization for dispensing emicizumab to patients with hemophilia A in French community pharmacies must be accompanied by optimal safety and quality conditions due to the risk of serious and urgent bleeding situations in the management of rare bleeding diseases. The elaboration of the PASODOBLEDEMI protocol has already a positive impact with the commitment of all health professionals, physicians, hospital and community pharmacists, and the patient community. The results will be disseminated among the French authorities and will enable, if necessary, proposing this access model to other rare diseases.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05449197, https://clinicaltrials.gov/ct2/show/NCT05449197?term=NCT05449197; ClinicalTrials.gov NCT05450640, https://clinicaltrials.gov/ct2/show/NCT05450640?term=NCT05450640.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/43091.

Validating the evaluation capacity scale among practitioners in non-governmental organizations

The growing emphasis on demonstrating the effectiveness of social services through evaluation has heightened demand for nongovernmental organization (NGO) practitioners to enhance evaluation capacity. However, a lack of validated instruments in the NGO context has hampered efforts to assess NGO practitioners' current evaluation capacity and understand how capacity-building activities could be tailored to meet NGO practitioners' actual needs and enhance their evaluation capacity. Hence, this study aims to develop the Evaluation Capacity Scale (ECS), a self-reporting instrument of NGO practitioners' capacity to conduct an effective evaluation of their service programs. Validation data was derived from 439 NGO practitioners who attended the Jockey Club MEL Institute Project in Hong Kong, China. Exploratory factor analysis of the ECS revealed three factors-evaluation mindset, evaluation implementation, and evaluation communication-and confirmatory factor analysis further validated this three-factor structure. Moreover, MANCOVA analysis demonstrated the ECS's predictive validity. Overall, the ECS demonstrated satisfactory convergent validity, high internal consistency reliability, and predictive validity, and its factor structure was supported in subgroups based on gender, age, and level of education. Theoretical and practical implications of the findings are discussed.

The impact of a continuing training program on the perceived improvement in quality of health care delivered by health care professionals

There is abundant scientific literature concerning factors that affect patients' perceptions of the quality of health care. However, there are few published works that consider the opinions of health care professionals. This article aims to conjointly analyse two organisational strategies that determine professional health care practice: continuous training and quality of care. The objective is to examine the opinions of physicians and nurses on the improvement of the quality of care after a 'learning by doing' program. An evaluation method was designed that integrates the main variables that intervene in quality of care. An online questionnaire was utilised for collecting opinions on the effects of the training program. A total of 184 nurses and 180 other medical professionals participated in the program and all of them were asked to complete the questionnaire. A descriptive, and inferential statistical analysis was undertaken and results showed that there is a direct relationship between perceptions about: satisfaction, professional competence, training modality, optimisation of health resources and quality of care.

Assessing Participants' Experiences in a Service-Learning Program: A Psychometric Evaluation

There is currently a gap in the literature regarding the creation of psychometrically sound measurement tools assessing service-learning programs in health-related fields. Without comprehension of a survey's psychometric properties, evaluators cannot ensure that survey instruments are reliable or valid. This study describes the psychometric evaluation of the Public Health Associate Program (PHAP) Service-Learning Scale (PSLS). PSLS assesses participant experience in PHAP, a Centers for Disease Control and Prevention program. This paper explains survey development, scale validity and reliability, and the internal factor structure of the PSLS. The final scale consisted of 22 items with a high internal consistency (Cronbach's α=.90). Exploratory Factor Analysis (EFA) was used to determine the scale's factor structure; five factors comprising of all 22 items were retained. The factors, or subscales, were Learning Outcomes, Mentoring, Experiential Assignment, Self-Efficacy in Program Competency Domains, and Program Satisfaction. All were also found to have adequate internal consistency (Cronbach's α >.70). Service-learning is vital in developing the next generation of the workforce. These study findings suggest the PSLS fills a critical gap in the literature by providing a valid and reliable instrument to evaluate experiences and satisfaction in service-learning programs and other fellowships.

Early Evaluation Findings From a Federally Funded Training Program: The Public Health Associate Program

OBJECTIVE

The Centers for Disease Control and Prevention (CDC) created the Public Health Associate Program (PHAP) to establish a continuous source of public health professionals who can deliver frontline services at the federal, state, tribal, local, and territorial levels. The article describes preliminary evaluation findings for PHAP.

DESIGN

The evaluation's primary purposes are to assess the quality and effectiveness of PHAP, determine its value and impact, and provide information to continuously improve the program. Because the evaluation is both formative and summative and focuses on aggregate outputs and outcomes of PHAP, the methodology is complex and builds over time as different cohorts cycle into and out of the program. Results presented are outcomes of various Web-based surveys and reporting systems.

PARTICIPANTS

Four PHAP cohorts, consisting of 579 individuals, participated in 1 or more of the evaluation activities described in this article.

RESULTS

The majority of participants report satisfaction with their PHAP experiences, and 74% of recent graduates indicate they are continuing their careers or education in public health immediately after program completion. Seventy-eight percent of recent PHAP graduates who accept a job in public health are employed by the federal government. One year post-PHAP, 74% of alumni report that PHAP has been influential in their careers.

CONCLUSION

CDC's investment in PHAP has increased the capacity and capabilities of the public health workforce. Results presented are early indicators of program quality, effectiveness, and impact. Today's public health workers are asked to do more with less, in the face of a dynamic array of complex public health challenges. PHAP offers public health agencies assistance in tackling these losses and challenges.