Using the Kirkpatrick Model to evaluate the Maternity and Neonatal Emergencies (MANE) programme: Background and study protocol

Rural and regional maternity managers' and educators' views of the Maternity and Newborn Emergencies (MANE) education program in Victoria, Australia: a qualitative descriptive study

BACKGROUND

Australia has one of the lowest perinatal morbidity and mortality rates in the world, however a cluster of perinatal deaths at a regional health service in the state of Victoria in 2015 led to state-wide reforms, including the introduction of the Maternity and Newborn Emergencies (MANE) program. MANE was a 2-day interprofessional maternity education program delivered by external expert facilitators to rural and regional Victorian maternity service providers. An independent evaluation found that the MANE program improved the confidence and knowledge of clinicians in managing obstetric emergencies and resulted in changes to clinical practice. While there is a large volume of evidence that supports the use of interprofessional education in improving clinicians' clinical practice, the impact of these programs on the overall safety culture of a health service has been less studied. Managers and educators have an important role in promoting the safety culture and clinical governance of the heath service. The aim of this study, therefore, was to explore Victorian rural and regional maternity managers' and educators' views and experiences of the MANE program.

METHODS

Maternity managers and educators from the 17 regional and rural health services across Victoria that received the MANE program during 2018 and 2019 were invited to participate. Semi-structured interviews using mostly open-ended questions (and with a small number of fixed response questions) were undertaken. Qualitative data were transcribed verbatim and analysed thematically. Descriptive statistics were used for quantitative data.

RESULTS

Twenty-one maternity managers and educators from the 17 health services participated in the interviews. Overall, participants viewed the MANE program positively. Four themes were identified: the value of external facilitation in providing obstetric emergency training; improved awareness and understanding of clinical governance; improved clinical practice; and the importance of maintaining the program. Participants agreed that MANE had improved the confidence (94%) and skills (94%) of clinicians in managing obstetric emergencies, as well as confidence to escalate concerns (94%), and most agreed that it had improved clinical practice (70%) and teamwork among attendees (82%).

CONCLUSION

Maternity managers and educators were positive about MANE; they considered that it contributed to improving factors that impact the safety culture of health services, with delivery by external experts considered to be particularly important. Given the crucial role of maternity managers and educators on safety culture in health services, as well in program facilitation, these findings are important for future planning of maternity education programs across the state.

TRIAL REGISTRATION

Trial registration was not required for this study.

Evaluating the impact of a maternity and neonatal emergencies education programme in Australian regional and rural health services on clinician knowledge and confidence: a pre-test post-test study

INTRODUCTION

Almost 78 000 women gave birth in the state of Victoria, Australia, in 2019. While most births occurred in metropolitan Melbourne and large regional centres, a significant proportion of women birthed in rural services. In late 2016, to support clinicians to recognise and respond to clinical deterioration, the Victorian government mandated provision of an emergency training programme, called Maternity and Newborn Emergencies (MANE), to rural and regional maternity services across the state. This paper describes the evaluation of MANE.

DESIGN AND SETTING

A quasi-experimental study design was used; the Kirkpatrick Evaluation Model provided the framework.

PARTICIPANTS

Participants came from the 17 rural and regional Victorian maternity services who received MANE in 2018 and/or 2019.

OUTCOME MEASURES

Baseline data were collected from MANE attendees before MANE delivery, and at four time points up to 12 months post-delivery. Clinicians' knowledge of the MANE learning objectives, and confidence ratings regarding the emergencies covered in MANE were evaluated. The Safety Attitudes Questionnaire (SAQ) assessed safety climate pre-MANE and 6 months post-MANE among all maternity providers at the sites.

RESULTS

Immediately post-MANE, most attendees reported increased confidence to escalate clinical concerns (n=251/259). Knowledge in the non-technical and practical aspects of the programme increased. Management of perinatal emergencies was viewed as equally stressful pre-MANE and post-MANE, but confidence to manage these emergencies increased post-delivery. Pre-MANE SAQ scores showed consistently strong and poor performing services. Six months post-MANE, some services showed improvements in SAQ scores indicative of improved safety climate.

CONCLUSION

MANE delivery resulted in both short-term and sustained improvements in knowledge of, and confidence in, maternity emergencies. Further investigation of the SAQ across Victoria may facilitate identification of services with a poor safety climate who could benefit from frequent targeted interventions (such as the MANE programme) at these sites.

Effectiveness of Training Workshop on ICD-10 Cancer Coding Guidelines for Clinical Coders

Background: The accurate retrieval of cancer data in health information systems depends on the correct coding and classification of data. Thus, it is essential to provide continuous training for clinical coders and ensure the effectiveness of this training to achieve high-quality data. Objectives: The present study aimed at evaluating the effectiveness of the training workshop on ICD-10 cancer coding guidelines for clinical coders. Methods: The present study was performed to evaluate the effectiveness of coding guidelines training regarding the second chapter of ICD-10 (2016 ed.). Forty-five clinical coders have participated in the study. A researcher-made questionnaire was provided for guidelines data gathering, with the scoring system of Likert (0 to 5 points) at the reaction level, and by and pre-and post-test questionnaires at the learning level. The descriptive statistics were applied on the reaction level and the paired sample t-test was performed on the reaction level data. Results: The results of descriptive statistics revealed that the learners had 84.4% of the average satisfaction with the training courses The paired-samples test indicated that, at the learning level, a significant difference (P = 0.000) existed between the mean pre-and post-test scores. Conclusions: Based on Kirkpatrick’s model, the training program resulted in satisfaction and improved the clinical coders skills regarding cancer coding. It is recommended that health information managers hold effective training courses to enhance the coders’ knowledge and skills.

Evaluating the effect of an in-service training workshop on ICD-10 coding instructions of pregnancy, childbirth and the puerperium for clinical coders

The quality of the data coded based on the 10^th revision of the International Classification of Diseases (ICD-10) can be improved by providing continuous education and promoting the clinical coders’ knowledge and skills. Due to the significance of maternal health in promoting the health of society, the present study evaluated the effects of an in-service training workshop on ICD-10 coding instructions of pregnancy, childbirth, and the puerperium for clinical coders. This applied evaluation study was conducted to evaluate the effects of a coding instructions training course focusing on the 15^th chapter of the ICD-10. The statistical population comprised 45 clinical coders working in the hospitals. The data were collected by a researcher-made questionnaire scored on a five-point Likert scale at the reaction level and by pretest and posttest questionnaires at the learning level. The data were then analyzed by descriptive statistics at the reaction level and by a paired-samples t-test at the learning level. The participants’ satisfaction with the training course was 94.7% on average at the reaction level. At the learning level, the results of the paired-samples t-test showed a significant difference between the means of scores before and after the training course (p=0.000). The training course led to satisfaction and enhanced the capabilities of the clinical coders with regard to coding the 15^th chapter of ICD-10. Clinical Coders must receive training on the new changes and guidelines in the other chapters of ICD-10 based on its most recent revision and employ them in the workplace.