Developing a framework for evaluating the impact of Healthcare Improvement Science Education across Europe: a qualitative study

Evaluation of cost-effectiveness of single-credit traffic safety course based on Kirkpatrick model: a case study of Iran

BACKGROUND

Training plays a role in reducing traffic accidents, and evaluating the effectiveness of training programs in managers' decision-making for training continuation is important. Thus, the present study aimed to evaluate the cost-effectiveness of a single-credit traffic safety course based on the four levels of the Kirkpatrick model in all Iranian universities.

METHODS

This interventional study aimed to evaluate the cost-effectiveness of a single-credit traffic safety course based on the Kirkpatrick model from 2016 to 2020 in Iran. The data were collected in three stages: (1) calculating the costs of offering traffic safety courses, (2) determining the effectiveness of providing such courses based on the levels of the Kirkpatrick model, and (3) evaluating the cost-effectiveness of administering traffic safety courses. Data were collected through researcher-made and standardized questionnaires. The research population included traffic safety course instructors and university students who could take this course. Finally, the data were analyzed with SPSS v. 23 and also calculations related to ICER, which shows the cost effectiveness of providing single credit course.

RESULTS

Scores of the students' reaction level to the traffic safety course was 41.8% before the course; this score was estimated at 67% after the course. At the level of learning, students' knowledge was 43.6% before the training course, which reached 73% after the course. At the level of behavior, the state of students' desirable traffic behaviors was 54% before the course, which reached 66.1% after the course. The educational effectiveness of the course presentation at the level of results was 58.2% before and 74.8% after the course. While assuming that the weights of all model levels were constant, the cost of a 1% increase in the overall educational effectiveness by using the Kirkpatrick model, compared to not providing the course (not administering the intervention) was 486.46 USD.

CONCLUSION

The results showcased the effectiveness of the traffic safety course in all four levels of The Kirkpatrick model. Therefore, policy-makers and officials in charge of delivering this program should strengthen it and resolve its deficiencies to realize all its educational goals at the highest level.

How, and under what contexts, do academic-practice partnerships collaborate to implement healthcare improvement education into preregistration nursing curriculums: a realist review protocol

INTRODUCTION

Internationally, healthcare improvement remains a clinical and educational priority. Consensus in Europe, Canada and the USA to implement quality improvement (QI) education into preregistration nursing curricula ensures students become equipped with the skills and knowledge required to improve practice. Now, New Zealand and Australia are beginning to implement QI education into their nursing curricula. However, QI education is complex; comprising multiple components, each influenced by the contexts under which they are developed and implemented. Evaluation studies of QI education unanimously acknowledge that academic and practice partnerships (APPs) are essential to optimally embed QI into preregistration curricula, yet it is not understood how, and under what contexts, APPs collaborate to achieve this.

METHODS AND ANALYSIS

A realist review to determine how, and under what contexts, APPs collaborate to implement QI education in pre-registration nursing will be conducted using the Realist and Meta-narrative Evidence Syntheses: Evolving Standards Guidelines. International stakeholders will be consulted at each stage which includes (1) clarifying the scope of the review through empirical literature and tacit expert knowledge, (2) searching for evidence in healthcare and social science databases/grey literature, (3) appraising studies using the Evidence for Policy and Practice Information and Co-ordinating Centre weight of evidence framework and extracting data using Standards for QUality Improvement Reporting Excellence in Education Publication Guidelines, (4) synthesising evidence and drawing conclusions through the creation of context, mechanism and outcome configurations and (5) disseminating findings through conferences and peer-reviewed publications.

ETHICS AND DISSEMINATION

Ethical approval was not required for this study. Findings will be disseminated to international nurse educators, leaders and front-line staff implementing QI education within their own academic and practice contexts through conferences and peer-reviewed publications.

PROSPERO REGISTRATION NUMBER

CRD42021282424.

Yemen Advanced Field Epidemiology Training Program: An Impact Evaluation, 2021

This is the first evaluation of the Yemen Field Epidemiology Training Program (Y-FETP) to assess if it met its objectives. We collected data using mixed methods including desk review, a focus group discussion with the Y-FETP staff, in-depth interviews with 21 program stakeholders, and an online survey for the program's graduates. We transcribed/analyzed qualitative data using explanatory quotations and survey data using descriptive methods. The desk review indicated that Y-FETP covers 18 (82%) out of 22 governorates and conducted >171 outbreak investigations, 138 surveillance system analyses/evaluations, 53 planned studies, published >50 articles and had >155 accepted conference abstracts. Qualitative findings showed Y-FETP helped save lives and reduced morbidity/mortality using building capacities in outbreak response; provided evidence-based data for decision-making; and increased awareness about public health issues. An online survey showed that Y-FETP helped 60 to 80% of graduates conduct outbreak investigations, surveillance analysis/evaluation, manage surveillance systems/projects, engage in public health communication (reports/presentation), and use basic statistical methods. However, the evaluation revealed that Y-FETP is primarily funded by donors; thus, it is not sustainable. Other challenges include low graduate retention and limited training in policy development and management. Y-FETP achieved its main objectives of increasing the number of epidemiologists in the workforce, making a positive impact on public health outcomes.

Evaluation of Advanced Field Epidemiology Training Programs in the Eastern Mediterranean Region: A Multi-Country Study

Field Epidemiology Training Programs (FETPs) are competency-based training programs aiming to strengthen the epidemiologic capacity of the public health workforce. This study aimed to evaluate the impact of the advanced FETPs in the Eastern Mediterranean region (EMR) and ascertain whether the expected objectives of the programs are met. A descriptive study was conducted based on Kirkpatrick's model for evaluating training programs. Data were collected from FETP graduates and FETP technical advisers on the practices of FETP graduates, their engagement in key areas of field epidemiology, and their perceived skills and capacity to perform such activities. A total of 166 FETP graduates responded to the online survey. Almost two-thirds of FETP graduates reported that they are often engaged in managing public health surveillance systems (n = 119, 71.7%), analyzing the surveillance data (n = 116, 69.9%), training public health professionals (n = 113, 68.1%), investigations on and response to outbreaks (n = 109, 65.7%), and managing staff and resources (n = 106, 63.9%). However, only 28.3% reported that they are often engaged in writing scientific research articles. More than two-thirds of graduates reported that the FETP helped them to perform most of the field epidemiology activities and rate their skills as good. In conclusion, the FETP graduates in the EMR were well engaged in many field epidemiology activities including managing public health surveillance systems, surveillance data analysis, training public health professionals, and investigations on and response to outbreaks. Therefore, the FETPs should continue supporting the graduates to work toward strengthening surveillance systems and investigating outbreaks and to participate in regional and global efforts as part of the Global Health Security.

Qualitative evaluation of an educational intervention about healthcare improvement for nursing students

Objective: To explore the meaning of the experience and knowledge acquired by nursing students trained in healthcare improvement. Material and methods: A phenomenological approach based on the Giorgi method was made through a group discussion of 21 European students from four countries in July 2019. The data was triangulated with the testimonies of key informants and with the use of an evaluation tool specific to this topic. The analysis was also triangulated with three experienced researchers who reduced the data by consensus to eight units of meaning. Results: Nursing empowerment and horizontal health organizations were two of the most recurrent units of meaning along with professional values such as teamwork and humanization of care. Conclusions: The knowledge, attitudes and values acquired are evidence of the shortcomings in the training of future nursing professionals in Europe. The theoretical and practical approach of the sessions, as well as the mixture of cultures, were strengths for the students. It is necessary to follow up on more cases that show the return on the training investment in substantial improvements in real contexts and its impact on the quality of care.

Transforming the Future Healthcare Workforce across Europe through Improvement Science Training: A Qualitative Approach

Healthcare improvement science (HIS) is the generation of knowledge to cultivate change towards improving health systems performance. Our purpose was to evaluate the experience of European nursing students after an intensive one-week summer program conducted in 2019 at the University of Alicante in Spain. The educational intervention combined theoretical and practical HIS contents, with students from different countries, educational programs, and health systems. The intervention was evaluated under a qualitative approach through the open discussion group technique based on the method of participatory action research (PAR), with a total of 25 students who reflected about their experiences and perceptions during the intervention. The responses were used to improve the program’s contents, its didactics, and organization. Nursing empowerment, professional recognition, and healthcare research were some of the seven main categories identified through the systematic content analysis method triangulated by three experienced researchers. According to the students’ replies, values like compassion, respect, or empathy were identified as key elements of care. Promoting international students’ networking emerged as the key to creating a positive provision for change and the generation of improvement initiatives. Building a HIS culture may potentially provide future healthcare professionals with critical thinking skills and the resources needed to improve their future work settings.

The United Kingdom Field Epidemiology Training Programme: meeting programme objectives

Introduction

Most evaluations of field epidemiology training programmes (FETP) are limited to process measures, but stakeholders may need evidence of impact.

Objective

To ascertain if the United Kingdom (UK) FETP met its objectives to: (i) strengthen capacity and provision of national epidemiology services, (ii) develop a network of highly skilled field epidemiologists with a shared sense of purpose working to common standards and (iii) raise the profile of field epidemiology through embedding it into everyday health protection practice.

Methods

The evaluation consisted of: (i) focus groups with training site staff, (ii) individual interviews with stakeholders and (iii) an online survey of FETP fellows and graduates. Findings were synthesised and triangulated across the three evaluation components to identify cross-cutting themes and subthemes.

Findings

Eight focus groups were undertaken with 38 staff, ten stakeholders were interviewed and 28 (76%) graduates and fellows responded to the survey. Three themes emerged: confidence, application and rigour. FETP was perceived to have contributed to the development, directly and indirectly, of a skilled workforce in field epidemiology, increasing stakeholders’ confidence in the service. Graduates applied their learning in practice, collaborating with a wide range of disciplines. Fellows and graduates demonstrated rigour by introducing innovations, supporting service improvements and helping supervisors maintain their skills and share good practice.

Conclusion

The UK FETP appears to have met its three key objectives, and also had wider organisational impact. FETPs should systematically and prospectively collect information on how they have influenced changes to field epidemiology practice.

Mapping the Status of Healthcare Improvement Science through a Narrative Review in Six European Countries

With the aim to explore how improvement science is understood, taught, practiced, and its impact on quality healthcare across Europe, the Improvement Science Training for European Healthcare Workers (ISTEW) project "Improvement Science Training for European Healthcare Workers" was funded by the European Commission and integrated by 7 teams from different European countries. As part of the project, a narrative literature review was conducted between 2008 and 2019, including documents in all partners' languages from 26 databases. Data collection and analysis involved a common database. Validation took place through partners' discussions. Referring to healthcare improvement science (HIS), a variety of terms, tools, and techniques were reported with no baseline definition or specific framework. All partner teams were informed about the non-existence of a specific term equivalent to HIS in their mother languages, except for the English-speaking countries. A lack of consensus, regarding the understanding and implementation of HIS into the healthcare and educational contexts was found. Our findings have brought to light the gap existing in HIS within Europe, far from other nations, such as the US, where there is a clearer HIS picture. As a consequence, the authors suggest further developing the standardization of HIS understanding and education in Europe.