1
|
Robinson SJ, Yin Mar Oo, Ljuhar D, McLeod E, Pacilli M, Nataraja RM. A guide to outcome evaluation of simulation-based education programmes in low and middle-income countries. ANZ J Surg 2024; 94:1011-1020. [PMID: 38553885 DOI: 10.1111/ans.18987] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/13/2023] [Revised: 02/16/2024] [Accepted: 03/13/2024] [Indexed: 06/19/2024]
Abstract
Evaluation is a vital part of any learning activity and is essential to optimize and improve educational programmes. It should be considered and prioritized prior to the implementation of any learning activity. However, comprehensive programme evaluation is rarely conducted, and there are numerous barriers to high-quality evaluation. This review provides a framework for conducting outcome evaluation of simulation-based education programmes in low and middle-income countries (LMICs). The basis of evaluation, including core ideas of theory, purpose and structure are outlined, followed by an examination of the levels and healthcare applications of the Kirkpatrick model of evaluation. Then, methods of conducting evaluation of simulation-based education in LMICs are discussed through the lens of a successful surgical simulation programme in Myanmar, a lower-middle-income country. The programme involved the evaluation of 11 courses over 4 years in Myanmar and demonstrated evaluation at the highest level of the Kirkpatrick model. Reviewing this programme provides a bridge between evaluation theory and practical implementation. A range of evaluation methods are outlined, including surveys, interviews, and clinical outcome measurement. The importance of a mixed-methods approach, enabling triangulation of quantitative and qualitative analysis, is highlighted, as are methods of analysing data, including statistical and thematic analysis. Finally, issues and challenges of conducting evaluation are considered, as well as strategies to overcome these barriers. Ultimately, this review informs readers about evaluation theory and methods, grounded in a practical application, to enable other educators in low-resource settings to evaluate their own activities.
Collapse
Affiliation(s)
- Samuel Ja Robinson
- Department of Paediatrics, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Paediatric Surgery and Monash Children's Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
| | - Yin Mar Oo
- Department of Paediatric Surgery, Yangon Children's Hospital, Yangon, Myanmar
| | - Damir Ljuhar
- Department of Paediatrics, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Paediatric Surgery and Monash Children's Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
| | - Elizabeth McLeod
- Department of Paediatric and Neonatal Surgery, Royal Children's Hospital, Melbourne, Victoria, Australia
| | - Maurizio Pacilli
- Department of Paediatrics, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Paediatric Surgery and Monash Children's Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
| | - Ramesh M Nataraja
- Department of Paediatrics, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia
- Department of Paediatric Surgery and Monash Children's Simulation, Monash Children's Hospital, Melbourne, Victoria, Australia
| |
Collapse
|
2
|
Yu IC, Fang JT. Effects of Microlectures on Nursing Students' Understanding of Key Medication Administration Concepts: A Quasi-Experimental Design. Nurse Educ 2023; 48:E35-E40. [PMID: 36332228 PMCID: PMC9936970 DOI: 10.1097/nne.0000000000001316] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 09/24/2022] [Indexed: 11/06/2022]
Abstract
BACKGROUND Microlectures comprise short audio or video presentations designed to provide a concise explanation of key concepts or topics. PURPOSE The study aimed to develop a microlecture module for a nursing course in medication administration and test whether it could improve learning. METHODS The study had a quasi-experimental design. Four classes of sophomore nursing students were divided into the treatment (n = 109) and control groups (n = 84). Pretests, posttests, integrated examinations, and simulated experiences were employed. Analysis of covariance were used to analyze the students' pretest, posttest, and integrated examination performances. RESULTS The treatment group performed better than the control group. CONCLUSIONS This study provides nursing educators with a different approach to traditional teaching and sheds light on how to design a microlecture.
Collapse
Affiliation(s)
- I-Chen Yu
- Associate Professor (Dr Yu), Department of Nursing, Chang Gung University of Science and Technology, Linkou, Tao-Yuan, Taiwan; Associate Researcher (Dr Yu) and Doctor of Medicine (Dr Fang), Department of Nephrology, Chang Gung Memorial Hospital, Linkou, Tao-Yuan, Taiwan; and Professor (Dr Fang), School of Medicine, College of Medicine, Chang Gung University, Tao-Yuan, Taiwan
| | - Ji-Tseng Fang
- Associate Professor (Dr Yu), Department of Nursing, Chang Gung University of Science and Technology, Linkou, Tao-Yuan, Taiwan; Associate Researcher (Dr Yu) and Doctor of Medicine (Dr Fang), Department of Nephrology, Chang Gung Memorial Hospital, Linkou, Tao-Yuan, Taiwan; and Professor (Dr Fang), School of Medicine, College of Medicine, Chang Gung University, Tao-Yuan, Taiwan
| |
Collapse
|
3
|
Baayd J, Heins Z, Walker D, Afulani P, Sterling M, Sanders JN, Cohen S. Context Matters: Factors Affecting Implementation of Simulation Training in Nursing and Midwifery Schools in North America, Africa and Asia. Clin Simul Nurs 2023; 75:1-10. [PMID: 36743129 PMCID: PMC9859761 DOI: 10.1016/j.ecns.2022.10.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
Background Robust evidence supports the effectiveness of simulation training in nursing and midwifery education. Simulation allows trainees to apply newly-learned skills in a supportive environment. Method This study was conducted using the Consolidated Framework for Implementation Research (CFIR). We conducted in-depth individual interviews with simulation experts around the world. Results Findings from this study highlight best-practices in facilitating simulation implementation across resources settings. Universal accelerators included: (1) adaptability of simulation (2) "simulation champions" (3) involving key stakeholders and (4) culturally-informed, pre-implementation planning. Conclusions Shared constructs reported in diverse settings provide lessons to implementing evidence-based, flexible simulation trainings in pre-service curriculum.
Collapse
Affiliation(s)
- Jami Baayd
- ASCENT Center for Sexual and Reproductive Health, Department of Obstetrics and Gynecology, University of Utah School of Medicine, Salt Lake City, UT, USA
| | - Zoë Heins
- Global Medical Affairs Scientist, bioMerieux, Salt Lake City, UT, USA
| | - Dilys Walker
- School of Medicine, Institute for Global Health Sciences, University of California, San Francisco, CA, USA
| | - Patience Afulani
- Department of Epidemiology and Biostatistics and Institute for Global Health Sciences, University of California, San Francisco, CA, USA
| | | | - Jessica N. Sanders
- ASCENT Center for Sexual and Reproductive Health, Department of Obstetrics and Gynecology, University of Utah School of Medicine, Salt Lake City, UT, USA
| | - Susanna Cohen
- ASCENT Center for Sexual and Reproductive Health and LIFT Simulation Design Lab, Department of Obstetrics and Gynecology, University of Utah School of Medicine, Salt Lake City, UT, USA
| |
Collapse
|
4
|
Abstract
Abstract
Objective
To ensure that only competent graduates are licensed to practice nursing, councils conduct licensing examinations, which may include among others clinical competency assessment. This review explored current practices in clinical competency assessment of nursing students as part of a larger study aimed at developing an evidence-based, context-specific framework for clinical competency assessment in a sub-Saharan African (SSA) country.
Methods
A scoping guided by the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) was conducted.
Results
Findings from 28 out of 1151 studies identified from Scopus, PubMed, CINAHL, Wiley Online Library, and ProQuest were included and synthesized. Results show that a good assessment system must be valid, reliable, transparent, feasible, fair, objective, and must provide feedback and continually improve to have an educational impact. Clinical competency assessment systems must be developed on sound empirical evidence, pilot tested, and involve thorough training and evaluation of the examiners. Continuous evaluation of the assessment system is also essential to ensure the quality and relevance of the assessment system. Only one of the included studies was conducted in Africa.
Conclusions
The paucity of clinical competency assessment research in sub-Saharan Africa may lead to benchmarking assessment systems on research conducted outside the context. Sub-Saharan Africa has a set of circumstances that demand a context-specific clinical competency assessment framework to guide clinical competency assessment.
Collapse
|
6
|
A Cognitive Load Theory Simulation Design to Assess and Manage Deteriorating Patients. Int J Nurs Educ Scholarsh 2019; 16:ijnes-2019-0009. [DOI: 10.1515/ijnes-2019-0009] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2019] [Accepted: 06/05/2019] [Indexed: 11/15/2022]
Abstract
Abstract
Background
Cognitive load theory (CLT) uses an understanding of brain architecture for educational design, with implications for simulation. Since working memory is limited, minimising extraneous cognitive load improves learning of new tasks (intrinsic load) and use of existing knowledge (germane load). This study evaluates the effectiveness of low-fidelity simulation (LFS) utilising CLT principles in the assessment and management of the deteriorating patient (AMDP).
Method
CLT design principles informed the choice of LFS and simulation design. The self-rated ability of 13 undergraduate nurses across seven aspects of AMDP was measured in a pre-post design.
Results
Self-rated ability increased from 2.98 (SD = 0.19) to 4.47 (SD = 0.12) (p < 0.001), with improvements across all AMDP aspects.
Conclusion
This study indicates that CLT informed design has benefits for simulation. LFS may be preferred to high fidelity simulation for AMDP teaching and medical simulation for novice learners.
Collapse
|