An Updated Review of Published Simulation Evaluation Instruments

Simulation-based education for teaching aggression management skills to healthcare providers in acute healthcare settings: A systematic review

BACKGROUND

Behavioural emergencies involving aggression in acute care hospitals are increasing globally. Acute care staff are often not trained or confident in their prevention or management. Of available training options simulation-based education is superior for clinical medical education and is gaining acceptance for teaching clinical aggression management skills.

OBJECTIVE

The aim of this study was to conduct a systematic review of the effectiveness of simulation-based education for teaching aggression management skills for health professionals working in acute healthcare settings.

METHODS

The study protocol was prepared in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols (PRISMA-P) statement, registered (27/02/2020) and published. We included randomised controlled trials, non-randomised controlled trials, quasi-experimental studies, and observational studies involving healthcare professionals in acute hospital settings or trainee health professionals who received simulation-based training on managing patient aggression. Comprehensive searches were conducted in PubMed, Ovid MEDLINE, PsycINFO, CINAHL and The Cochrane Library. Two reviewers independently screened all records, extracted data and assessed risk of bias. The primary outcomes included patient outcomes, quality of care, and adverse effects. Secondary outcomes included workplace resource use, healthcare provider related outcomes, knowledge (de-escalation techniques), performance, attitudes, and satisfaction. A narrative synthesis of included studies was performed because substantial variation of interventions and outcome measures precluded meta-analyses.

RESULTS

Twenty-five studies were included with 2790 participants, 2585 (93 %) acute care hospital staff and 205 (7 %) undergraduate university students. Twenty-two studies combined simulation-based education with at least one other training modality. Three studies were randomised controlled trials, one was a pilot and feasibility cluster randomised controlled trial, one was a three-group post-test design and twenty were pre-/post-test design. Twenty-four studies were deemed to be high/critical or serious risk of bias. Four studies collected primary outcome data, all using different methods and with inconsistent findings. Twenty-one studies assessed performance in the test situation, seven studies provided objective ratings of performance and eighteen provided self-report data. Twenty-three studies reported objective or subjective improvements in secondary outcomes.

CONCLUSIONS

Acute healthcare staff who completed simulation-based education on managing clinical aggression showed statistically significant improvements in knowledge and self-reported confidence. However, there is a lack of evidence about the magnitude of these improvements and impact on patient outcomes.

REGISTRATION

PROSPERO Registration Number CRD42020151002.

TWEETABLE ABSTRACT

Simulation-based education improved acute healthcare clinician knowledge and confidence in managing aggression.

Criteria to evaluate graduate nurse proficiencies in obtaining a health history and perform physical assessment in simulation-based education: A narrative review

BACKGROUND

Simulation is a technique being used increasingly in healthcare education which offers opportunities to evaluate nursing proficiencies. The use of valid and reliable instruments is recognised as the foundation for a robust assessment, however competency-based health assessment courses for graduate nurses can consequently become reductionist in measuring proficiencies.

OBJECTIVE

The specific review question was: In simulation-based education, what are the criteria that evaluate graduate nursing student's competence in obtaining a health history and performance of patient assessment?

METHODS

Eleven studies were included in the review. Papers were critically appraised with The Joanna Briggs Institute quasi-experimental studies checklist. Bloom's taxonomy was used to structure this narrative review.

RESULTS

Seven papers evaluated cognition through questionnaires and two papers used a Likert-scale to determine self-perceived knowledge. Six papers evaluated psychomotor skills with a behavioural checklist. Diversity of application was factored into the studies when testing affective skills. Three papers used Likert-scales to evaluate preparedness, six papers used Likert-scales to evaluate self-confidence and one used a Likert-scale to evaluate autonomy. Three papers used a checklist to evaluate professionalism. Four papers used faculty member/ standardised patient feedback.

CONCLUSION

Reductionist evaluation instruments create a barrier when evaluating competency. The limited validity and reliability of assessment instruments in simulation, as well as the lack of standardisation of affective skills assessment, presents a challenge in simulation research. Affective skills encompass attitudes, behaviours and communication abilities, which pose a significant challenge for standardised assessments due to their subjective nature. This review of the simulation literature highlights a lack of robustness in the evaluation of the affective domain. This paper proposes that simulation assessment instruments should include the standardisation of affective domain proficiencies such as: adaptation to patients' cognitive function, ability to interpret and synthesise relevant information, ability to demonstrate clinical judgement, readiness to act, recognition of professional limitations and faculty/standardised-simulated patient feedback. The incorporation of the affective domain in standardised assessment instruments is important to ensure comprehensive assessment of simulation particularly in the development of health history and physical assessment proficiencies. Attention to all of the domains in Blooms taxonomy during simulation assessment has the potential to better prepare professionals for the patient care setting.

Competency-Based Virtual Clinical Practicum in Response to COVID-19 Pandemic: A Quality Improvement Project

PURPOSE

The purpose of the quality improvement (QI) project was to provide a learner and faculty preceptor high quality virtual clinical practicum (VCP) for Wound, Ostomy, Continence (WOC) Nurse learners during the COVID-19 pandemic and disseminate findings.

PARTICIPANTS AND SETTING

Project participants were registered nurses post-bachelor's degree or higher with a minimum of 1-year clinical experience who had completed and passed the didactic portion of a Wound Ostomy Continence Nursing Education Program (WOCNEP). Learners completing the WOCNEP's VCP and the faculty preceptors were the subjects for this QI report. The setting was a virtual platform allowing live time visibility and interactions between learners and faculty preceptors including the ability to share documents and procedural videos. An initial group of 62 learners were asked open-ended questions after completion of the VCP; the most frequently recurring themes were converted into 5-point Likert scale questions as the evaluation tool completed by an additional 234 learners. In total, 296 learners completing the VCP evaluated this novel 100% simulation virtual practicum.

APPROACH

This QI project evaluated a 100% simulation-based practicum at a WOCENP based in the United States. A competency based VCP used the Competency Outcomes and Performance Assessment (COPA) model. Learners were taught competencies via a preparatory virtual lab and then had to prepare documents and videos or pictures with step-by-step instructions prior to attending a Live Competency Assessment Lab with faculty preceptors and a small group of peer learners.

OUTCOMES

Appraisal of the VCP was based on course evaluations anonymously completed by each of the 296 learners. Faculty preceptors were queried regarding their VCP evaluation and certification exam pass rates were tallied. Findings indicate positive outcomes of the VCP competency-based practicum from student and faculty preceptor perspectives along with higher certification pass rates; specifically, wound care pass rates increased by 16%, ostomy care increased by 18% and continence care pass rates increased by 13%.

IMPLICATIONS FOR PRACTICE

Our reported experiences strongly suggest a VCP offers an attractive option for fulfilling practicum requirements while reducing barriers of the in-person split option practicum.

Cognitive and physiological evaluation of virtual reality training in nursing

Virtual reality (VR) has increasingly been used in several disciplines, including nursing, a profession in high demand that is now facing a shortage. This study investigated the effectiveness and efficacy of VR hybrid training over traditional training (TR) methods. Sixteen college students were recruited and randomly assigned to get 100% TR or 50% TR and 50% VR (VR50). Participants attended a three-day program with a registered nurse, consisting of lectures and practical lab sessions. Participants' performance, training time, cognitive development, physical development, mental workload, user experience, Students' Satisfaction and Self-Confidence, and team learning were evaluated. The results showed that the VR50 performed as well as, and sometimes even better than the TR group (p-value = 0.043). VR50 group significantly had higher cognitive development and found VR easy to use and attractive (p-value < 0.05). VR-integrated training makes nurses' training more affordable and accessible while providing instant and relevant feedback.

Practitioner Summary: This study assessed employment-integrated virtual training in nursing, particularly peri-care, by comparing the performance, cognitive, physical, and mental workload of traditional and integrated VR training groups. The findings of this study provide significant support for incorporating VR training into educational settings.

The Lasater Clinical Judgment Rubric: 17 Years Later

BACKGROUND

Nearly 17 years ago, the Lasater Clinical Judgment Rubric (LCJR) was published to provide a common language and trajectory of students' development to think like a nurse.

METHOD

This article traces the uses of the LCJR from creation to the present and cites lessons learned from its use.

RESULTS

During the intervening years, the LCJR has been used effectively as a debriefing guide in simulation and as a research instrument, as well as for formative assessment. The LCJR has been translated or is in process in 19 languages besides English.

CONCLUSION

This article provides evidence of the efficacy of the LCJR and discusses important lessons learned. [J Nurs Educ. 2024;63(3):149-155.].

The Effects of a Simulation-Based Patient Safety Education Program on Compliance with Patient Safety, Perception of Patient Safety Culture, and Educational Satisfaction of Operating Room Nurses

BACKGROUND

Operating rooms (ORs) are healthcare areas that are high-risk regarding patient safety (PS). The prevention of PS errors such as wrong-site surgery, medication errors, and patient falls is important in the OR. Causes such as having insufficient information, not taking enough care and precautions, and inattention may lead to errors. Ensuring PS in an organization depends on the composition of a PS culture.

METHOD

This study, as equivalent-control-group pretest-posttest research, aimed to develop and apply a simulation-based patient safety education program for operating room nurses and then to examine the influence of the program on patient safety management and compliance. Participants included a total of 45 operating room nurses: 22 in the experimental group and 23 in the control group. In the program, each member of the experimental and control groups underwent two 60 min sessions for a total of 120 min. Person-to-person individual lectures and simulation-based practice were provided to the experimental group, whereas booklets and person-to-person individual lectures were provided to the control group. Compliance with patient safety, the perception of patient safety culture, and satisfaction were measured using a structured Likert questionnaire. Intervention effects were analyzed using a t-test and ANCOVA.

RESULTS

As a result of the analysis, we found that the experimental group was significantly higher in terms of compliance with patient safety (p = 0.021), the perception of patient safety culture (p = 0.039), and education satisfaction (p < 0.001) than the control group.

CONCLUSIONS

The results indicate that implementing a simulation-based patient safety education program can improve the patient safety competency of operating room nurses and, ultimately, prevent patient safety accidents in the operating room.

Satisfaction and self-confidence among nursing students with simulation learning during COVID-19

AIM

This survey aimed to investigate nursing students' satisfaction and self-confidence in simulation in education during the COVID-19 pandemic. Along with comparing these levels based on selected students' characteristics.

DESIGN

A cross-sectional survey.

METHODS

The survey was conducted at the faculty of nursing of a private university in Jordan. Students' satisfaction and self-confidence levels in simulation learning were measured using the National League for Nursing (NLN) Student Satisfaction and Self-confidence in Learning Scales.

RESULTS

A total of 138 undergraduate nursing students participated in the survey. Students' satisfaction levels and self-confidence in simulation learning were lower (just above the scale's midpoint) than scores reported in similar surveys. The lowest student ratings were reported as "the variety of learning activities that can be done using simulation" and "the self-confidence to develop the needed skills and knowledge to be used in real clinical settings". The results also indicated that as students' progress in the bachelor's degree program, they develop higher levels of self-confidence in simulation-based learning.

CONCLUSIONS

Nursing students' experience of simulation learning was observed to be negatively affected by the COVID-19 pandemic. High-fidelity simulation, in particular, among other simulation modalities, can be more beneficial in similar situations. Education stakeholders are invited to invest in the resources of high-fidelity simulation to maximize its benefits and help in the recovery phase after the pandemic.

The Effect of Simulation Training on Enhancing Nursing Students' Perceptions to Incorporate Patients' Families Into Treatment Plans: A Randomized Experimental Study

INTRODUCTION

As clinical placement in bachelor's nursing programs becomes increasingly difficult, simulation is becoming increasingly common to enhance learning. Blended learning incorporating simulation videos provides students with the opportunity to observe and learn from exemplary practices while bridging the gap between theoretical knowledge and its practical application. This study aimed to investigate the effect of simulation training on enhancing nursing students' perception of integrating patient's families' assessments into their treatment plan.

METHODS

A quantitative, experimental research design was used, with a control (56) and intervention group (67) from levels 7 and 8 senior nursing students at King Saud Bin Abdulaziz University for Health Sciences, College of Nursing, Jeddah, assigned randomly to each group. The tool consists of three sections: personal information, a Van Gelderen family rubric, and a role-play survey. The validity and reliability of the tools were confirmed by the original developer. In the current study, the reported Cronbach's alpha was 95%.

RESULTS

A total of 123 students participated in the study. Their ages ranged between 19 and 23 years and 23 years and above, with a mean age of 21.3 ± 1.3 among the control group and 22.2 ± 1.1 among the experimental group. There was an improvement in the mean scores in the post-training phase compared to the pre-training phase in the experimental group, with a statistically significant difference at p < 0.05. However, there were no significant differences noted between the control and experimental groups in the pre-training phase compared to the statistically significant difference noted between the two groups in the post-training phase.

CONCLUSION AND RECOMMENDATIONS

The findings of the study indicated that the utilization of scenario-based standardized patient-simulated exercises, guided by dedicated faculty and accompanied by reflective debriefing exercises, proved to be an effective approach for bridging the gap between theoretical knowledge and its application in clinical practice. Therefore, the study prompts curriculum revisions to incorporate family assessment into nursing practices, as well as evidence-based strategies, such as learning activities that use standardized patient or high-fidelity simulation technology to address and possibly reduce the theory-practice gap for graduates when entering clinical practice.

Lasater clinical judgment rubric reliability for scoring clinical judgment after observing asynchronous simulation and feasibility/usability with learners

BACKGROUND

There is strong evidence supporting using the Lasater Clinical Judgment Rubric (LCJR) for scoring learners' clinical judgment during in-person simulation performance and clinical experience reflections. However, a gap exists for using LCJR to evaluate clinical judgment after observing asynchronous simulation.

OBJECTIVE

We aimed to determine the reliability, feasibility, and usability of LCJR for scoring learners' written reflections after observing expert-modeled asynchronous simulation videos.

DESIGN/SETTING/PARTICIPANTS

We used a one-group, descriptive design and sampled pre-licensure, junior-level bachelor's learners from the Southwestern United States.

METHODS

Participants observed eight expert-modeled asynchronous simulation videos over one semester and provided written responses to clinical judgment prompts. We scored clinical judgment using LCJR. We studied reliability by measuring internal consistency of 11 clinical judgment prompts and interrater reliability with two raters. This study also investigated feasibility and usability of the asynchronous simulation learning activity using descriptive statistics. Feasibility included time learners spent completing written responses and time raters spent evaluating written responses. Learners reported usability perceptions using an instructor-developed survey.

RESULTS

Sixty-three learners completed 504 written responses to clinical judgment prompts. Cohen's kappa ranged from 0.34 to 0.86 with a cumulative κ = 0.58. Gwet's AC ranged from 0.48 to 0.90, with a cumulative AC = 0.74. Cronbach's alpha was from 0.51 to 0.72. Learners spent on average 28.32 ± 12.99 min per expert-modeling video observation. Raters spent on average 4.85 ± 1.34 min evaluating written responses for each participant. Learners reported the asynchronous learning activity was usable.

CONCLUSIONS

Nurse educators can reliably use LCJR for scoring learners' clinical judgment after observing asynchronous expert-modeled simulation. Logistically, learners complete the reflective learning activity and faculty use LCJR to measure clinical judgment in feasible time. Further, participants perceived the asynchronous learning activity usable. Nurse educators should utilize this learning activity for evaluating and tracking observer clinical judgment development.

Effect of High-Fidelity Simulation on Self-Satisfaction and Self-Confidence Among Nursing Students

Introduction

Newly graduating nurses should be able to enter clinical practice as safe, accurate, competent, and compassionate professionals in health care settings that necessitate them to be learners who can easily get familiar with a broad diversity of clinical practice settings.

Purposes

The purpose of this study was to evaluate the effect of high-fidelity simulation experiences on self-satisfaction and self-confidence among the pediatric nursing students.

Methods

A quasi-experimental design, post-test, was carried out through the recruitment of a convenience sample of 150 nursing students from Arab American University. The students were randomly assigned to the study (n  =  75) or control (n  =  75) groups. Data were collected using learner satisfaction and self-confidence scales.

Results

The results of the post-test revealed that there was a significant difference between both groups in self-satisfaction (t (148)  =  11.14, P < .001) and self-confidence (t (148)  =  13.04, P < .001). The mean score of self-satisfaction for the study group (22.45  ±  2.27) was higher than control group (16.80  ±  3.76). Also, the study group' mean self-confidence score (35.72  ±  3.49) was higher than the control group (27.28  ±  4.39).

Conclusion

High-fidelity simulation was confirmed to be an helpful tool to keep an effective and safe learning method for pediatric nursing students, thus increasing their satisfaction and confidence.

Psychometric evaluation of critical incident video instruments for nursing education

The Critical Incident Video (CIV) Project is a mixed method longitudinal study that uses CIVs to prepare nursing faculty members to address common teaching challenges. CIVs are short videos that present unresolved teaching challenges. Given the lack of specific instruments for evaluating the usefulness of CIVs, the aim of this phase of the project was to develop and test two instruments measuring the pedagogical effects of CIVs. The CIV Preparation and Confidence Scale (CIVPCS©) and the CIV Simulation Experience Scale (CIVSES©) were assessed for validity and reliability. Using a Delphi method, a convenience sample of 23 nurse educators provided feedback enhancing the validity and clarity of the CIVPCS©. Reliability of the CIVPCS© was determined using Cronbach's alpha and test-retest method. No changes were recommended for the CIVSES©. Findings from the assessment of these newly developed CIV instruments are reported and implications for faculty development are discussed.

Perceptions of final year nursing students transer of clinical judgement skills from simulation to clinical practice: A qualitative study

AIM

To explore final year nursing student's ability to transfer clinical judgement skills to the clinical practice setting following immersive simulation.

BACKGROUND

Clinical judgement is considered a fundamental skill for nurses to ensure safe, quality care is delivered. In undergraduate nursing education where students have limited clinical experience, simulation-based education is an important educational strategy for introducing and developing these skills. Simulation allows for students to be exposed to repeated experiences and emotional responses to varying clinical situations. Despite this, there is a paucity of literature relating to students' ability to transfer clinical judgement skills from the simulated environment into clinical practice.

DESIGN

A naturalistic philosophical approach informed data collection in this qualitative phase of a larger study.

METHODS

Data were collected from students and nurse educators using semi-structured interviews as well as from facilitated simulation debriefs. Data were thematically analysed.

RESULTS

Four themes were identified which related to student knowledge, self-awareness and the clinical context: Safely collecting the data; Understanding the data to safely make decisions; Emotional intelligence; and Role variation. Students and educators held similar views on many of these elements.

CONCLUSIONS

Questioning was identified as a key component of nursing students' clinical judgements. There were challenges in assessing students' ability to link theory to practice in the clinical setting, despite evidence of this occurring in the simulated setting. Simulation prepares students for practice by exposing them to new experiences and stressors and therefore an effective educational technique for developing clinical judgement skills in this cohort.

Psychometric Evaluation of the Creighton Competency Evaluation Instrument in a Population of Working Nurses

BACKGROUND AND PURPOSE

This study explored the psychometric properties of the Creighton Competency Evaluation Instrument (C-CEI), previously validated for use with nursing students, to assess simulation performance among registered nurses working 12-hour shifts. Valid and reliable measurements are needed to test clinical and simulation competencies and characterize the effects of fatigue on nursing performance.

METHODS

Trained raters scored nurses' patient care performance in simulation scenarios using the C-CEI. We analyzed the instrument's principal components, internal reliability, and construct validity.

RESULTS

Internal reliability of the C-CEI aggregate score and the Clinical Decision-making component were high (>.70). The latter robustly correlated with predicted cognitive effectiveness, a measure of fatigue.

CONCLUSIONS

The C-CEI is a reliable measure for use among registered nurses and its further development will be important for testing performance of working nurses and fatigue-mitigation innovations.

The relationships of nursing students’ satisfaction and self-confidence after a simulation-based course with their self-confidence while practicing on real patients in Vietnam

PURPOSE

Simulation teaching refers to the replication of real-life scenarios, enabling students to practice nursing skills and learn actively in a safe environment. It also helps students control their anxiety and fears when caring for real patients. This study investigated the relationships of Vietnamese nursing students’ self-confidence in clinical practice with their satisfaction and self-confidence in simulation-based practice.

METHODS

This cross-sectional descriptive study included 182 nursing students. The data collection included 2 separate stages with 2 main questionnaires. The Student Satisfaction and Self-Confidence in Learning Scale was used to measure students’ satisfaction and self-confidence after learning in the simulation room. The Confidence Scale was used to measure students’ self-confidence when first performing techniques on actual patients. Data were analyzed by descriptive and Pearson correlation statistics.

RESULTS

Students’ satisfaction and self-confidence during the simulation course were quite high (mean±standard deviation [SD], 4.06±0.48 and 4.11±0.46 out of 5.0, respectively). In contrast, their confidence when first practicing on a patient was moderate (mean±SD, 3.19±0.62 out of 5.0). Students’ satisfaction showed moderate and weak positive correlations with self-confidence in pre-clinical practice and in clinical practice (r=0.33, P<0.001 and r=0.26, P<0.001, respectively).

CONCLUSION

Simulation has become an effective teaching strategy that can help nursing students be well-prepared for clinical placements in Vietnam. An effective nursing education strategy is needed to enhance the satisfaction and self-confidence of nursing students in simulation and then in clinical practice to help achieve professional engagement and development.

Time-to-Task Validation in Sequential Simulations as a Component of Nursing Student Competency

BACKGROUND

Time goals have not been tested as a component of nursing competence.

METHODS

Using a known-groups approach, a multisite study was performed on prelicensure nursing students (novices) and experienced nurses (experts) who participated in 2 simulations of patient deterioration with time goals. Video capture was achieved with an eye tracker.

RESULTS

While all novices did not achieve the time goals in both simulations, there was a significant improvement between the 2 simulations (P = .04). Experts were more successful in achieving the time-to-task goals in both simulations (44% and 95%, respectively). Statistical significance was found between the novice and expert groups in achieving the time goal for both simulations (P < .01).

CONCLUSIONS

This study was able to differentiate the expert group as being more proficient in a timed assessment, although the novices showed improvement between 2 simulations. This illustrates an opportunity for skill development among novices in managing patient deterioration amenable to time as an objective assessment.

Paramedic students' confidence and satisfaction with clinical simulations of an emergency medical care programme in South Africa: A cross-sectional study

Background

There has been an increase in the use of clinical simulations as instructional tools in healthcare education. This is because of their role in ensuring patients' safety and quality-care provision.

Aim

This study investigated the paramedic students' satisfaction and self-confidence in the clinical simulation of an emergency medical care programme.

Setting

The study was conducted at the Durban University of Technology in the KwaZulu-Natal Province of South Africa. The paramedic students' satisfaction and self-confidence in the clinical simulation of an emergency medical care programme were the focus of the study.

Methods

The study used a cross-sectional research design. A convenience sampling method was used to select the 83-paramedic students who participated in the study. Data were collected between July and September 2017 using a structured questionnaire. Descriptive statistics (frequencies and percentages and Spearman's rank-order correlation coefficient) and an inferential test, ordinal logistic regression analysis, were used for data analysis.

Results

High levels of paramedic students' satisfaction and self-confidence in simulation activities were reported. Generally, the paramedic students' demographics were associated with the satisfaction and self-confidence variables with p-values ≤ 0.04. Emergency medical care training undertaken by the paramedic students was significantly associated with self-confidence (p = 0.00).

Conclusion

Clinical simulation can bridge the theory-practice gap for paramedic students. It is a hands-on approach that promotes students learning of clinical skills through reflection.

An Innovative Simulation Curriculum to Train General Surgery Residents and Medical Students on Four Commonly Encountered Obstetric and Gynecologic Procedures

INTRODUCTION

Lack of obstetric and gynecologic (OBGYN) procedure exposure during general surgery residency was recognized as a training gap for military general surgery residents by U.S. Navy trauma and general surgeons serving as simulation leads for the Naval Medical Center Portsmouth general surgery resident program. Program faculty requested the authors develop and implement a recurring simulation curriculum to address this training gap. The primary goal of the simulation curriculum was to increase exposure to and confidence in performing four commonly encountered OBGYN procedures in a deployed setting: spontaneous vaginal delivery (SVD), Bartholin's cyst incision and drainage with Word catheter placement, cesarean delivery, and total abdominal hysterectomy (TAH). We hypothesized that trainees exposed to the new simulation curriculum would demonstrate an increase in knowledge and confidence in these four commonly encountered OBGYN procedures.

MATERIALS AND METHODS

The study utilized a pre- and postdesign to determine the efficacy of a newly developed simulation curriculum as measured by participants' knowledge and confidence levels. The annual curriculum was 4 hours in duration and included a targeted lecture followed by the administration of the four OBGYN simulation skills stations: SVD, Bartholin's, cesarean delivery, and total abdominal hysterectomy. Proctors for the simulation stations consisted of OBGYN teaching faculty who had at least 5 years of clinical teaching experience and were experienced simulation instructors. Precourse and postcourse knowledge and confidence were self-rated on a 5-point Likert scale. The learners rated the course using the Simulation Design Scale. Likert scale data were analyzed using the paired Wilcoxon rank sum test. Descriptive statistics were reported for the Simulation Design Scale. P < 0.05 was considered to be statistically significant. This project was classified as nonhuman subjects' research by the Naval Medical Center Portsmouth Institutional Review Board.

RESULTS

Over the 2 years since curriculum implementation, 34 trainees participated in the curriculum; no trainees have had the curriculum twice. All trainees confirmed training gaps on baseline self-assessment of knowledge and confidence of all four OBGYN procedures with the most confidence reported for SVD (2.83 mean). Self-rated knowledge and confidence levels improved significantly in all four of the simulated procedures for all 34 participants (P < 0.001). Postcurricular surveys (82% response rate, 28/34) indicated a high satisfaction with all areas of the simulation training (4.74/5.0).

CONCLUSIONS

The implemented curriculum increased general surgery trainees' knowledge and confidence in four commonly encountered OBGYN procedures and demonstrated a high level of learner satisfaction and sustainability. The curriculum exhibits high educational impact and could be a valuable adjunctive training for other non-OBGYN physicians who may need to provide OBGYN care in military environments.

A Key to Transforming a Nursing Curriculum: Integrating a Continuous Improvement Simulation Expansion Strategy

Introduction

Changes in nursing, health care, and education warrant continued pedagogical innovations. Faculty are challenged to develop many innovative strategies in the clinical and simulation laboratory setting. Intentional simulation-based learning experiences are one method to prepare new graduates for nursing practice.

Methods

One college integrated intentional simulation-based learning experiences as an improvement strategy in a newly transformed undergraduate nursing curriculum, from mapping through evaluation and revision. Simulation-based learning experiences that were intentionally mapped, organized, and interactive enhanced the teaching and learning needs of students and faculty.

Conclusion

The positive outcomes from this curricular transformation serve as a platform for continuous improvement for future approaches to nursing education. This affirmed that the key to transforming a nursing curriculum encompasses intentional mapping, evaluation, and revision.

Development of the Simulation-Positive Learning Evaluation (SIM-PLE) rubric for use in medical-surgical nursing high fidelity simulations

Clinical nurse educators in a BSN program developed the SIM-PLE (Simulation-Positive Learning Evaluation) rubric to evaluate student performance in simulation. A pilot study was conducted over two semesters with junior level medical-surgical classes made up of approximately 110-130 students each semester, and the tool was revised and re-evaluated. Inter-rater reliability testing utilized 69 SIM-PLE rubric student evaluations (N = 69). Inter-rater reliability was assessed with comparison of lead faculty with three other instructors utilizing two raters for each simulation experience. The intraclass correlation coefficient (ICC) suggested high inter-rater reliability. The t-test difference was not significant. One-way analysis of variance (ANOVA) showed a significant difference in the Collaboration subcategory score. Univariate ANOVA indicated both raters scored consistently with each other overall and that they both scored JR1 differently than JR2. The pilot study and testing of interrater reliability of the rubric revealed positive findings. The role of the rubric in providing formative feedback during debrief sessions empowered students to analyze their own performance and/or experience to self-correct or improve their future practice based on these experiences.

Evaluation of Simulation Outcomes

Simulation has been used in nursing education and training since Florence Nightingale's era. Over the past 20 years, simulation learning experiences (SLEs) have been used with increasing frequently to educate healthcare professionals, develop and increase the expertise of practicing professionals, and gain competency in key interprofessional skills. This chapter provides a brief overview of simulation evaluation history, beginning in the late 1990s, and the initial focus on learner self-report data. Using Kirkpatrick's Levels of Evaluation as an organizing model, four types of SLE evaluation are reviewed as well as suggestions for future research.

Psychometric testing of the Norwegian version of the Simulation Design Scale, the Educational Practices Questionnaire and the Student Satisfaction and Self-Confidence in Learning Scale in nursing education

Introduction

Simulation-based learning is a well-established technique in nursing education. However, there is a need for reliable and validated evaluation tools across both national boundaries and cultural conditions. Such evaluation tools may contribute in identifying areas for improvement in simulation-based learning from the nursing students' perspective.

Objectives

The aim of this study was to test three widely used American questionnaires - the Simulation Design Scale, the Educational Practices Questionnaire, and the Student Satisfaction and Self-Confidence in Learning Scale, for psychometric properties among Norwegian undergraduate nursing students.

Methods

A descriptive cross-sectional study was conducted at a university simulation center in southern part of Norway. A total of 105 undergraduate nursing students participated, giving a response rate of 77%. An exploratory factor analysis was used to examine construct validity. Cronbach's alpha was applied in order to establish the questionnaires' internal consistency.

Results

The exploratory factor analyses displayed the same number of extracted factors as the number of subscales in each of the original American questionnaires. However, the item-factor structure differed from the original item-subscales. The Cronbach's alpha was > 0.7 for all three questionnaires, indicating acceptable internal consistency.

Conclusion

Psychometric testing of the Norwegian versions of the three questionnaires, the Simulation Design Scale, the Educational Practices Questionnaire, and the Student Satisfaction and Self-Confidence in Learning Scale, could be used as valid instruments for nursing students to evaluate important aspects of simulation-based learning. This also makes it easier to compare evaluation results of SBL across languages and cultural boundaries. However, to confirm the construct validity of the factors extracted in this study, further multi-site studies are needed to perform a confirmatory factor analysis in a new, large sample.

Development and Evaluation of an Algorithm-Corresponding Instrument for Nursing Simulation

BACKGROUND

This study aimed to develop and validate an assessment instrument for students engaging with an algorithm-based simulation scenario addressing emergency measures for high-risk newborns with apnea in a neonatal intensive care unit.

METHOD

The study was conducted in two phases of development and evaluation of the algorithm-corresponding instrument. One hundred sixty-nine senior nursing students from two universities in South Korea were evaluated using the developed instrument.

RESULTS

The developed and validated instrument consisted of three dimensions (assessment points, nursing skills, and communication) measured through 13 items. The exploratory factor analysis revealed three factors of the instrument, and the confirmatory factor analysis demonstrated a better model fit for a three-factor instrument model than for other models.

CONCLUSION

The developed algorithm-corresponding assessment instrument is suitable for assessing the clinical decision-making ability of nursing students in a simulation scenario. [J Nurs Educ. 2020;59(11):617-626.].

Simulation-based education to promote confidence in managing clinical aggression at a paediatric hospital

BACKGROUND

An increasing number of incidents involving aggressive behaviour in acute care hospitals are being witnessed worldwide. Acute care hospital staff are often not trained or confident in managing aggression. Competent management of clinical aggression is important to maintain staff and patient safety. Training programmes for acute care staff are infrequently described in the literature and rarely reported for paediatric staff. Simulation training allows practice of skills without patient risk and may be more effective than traditional teaching formats for aggression management.

AIM AND DESIGN

The aim of this proof of concept study was to develop a simulation-based education session on aggression management for acute care paediatric staff based on best practice principles, to evaluate the acceptability of this training programme and to gain an understanding of the impact of the training on participants' perceived confidence in managing clinical aggression. Two separate simulation exercises were delivered as a 2-h component of a hospital management of clinical aggression (MOCA) training day. Participants completed a written survey immediately prior to, at completion of the simulation-based group training, and at 3-6 months following the simulation training.

FINDINGS

Nine training days were conducted in 2017 for nursing, medical, allied health, education and security staff with a total of 146 participants (83% were acute care nurses). Two thirds (68%) of participants had experienced clinical aggression as part of their routine work, with 51% overall reporting a lack of confidence managing these patients. Immediately following the simulation training, 80% of all participants reported feeling more confident in managing clinical aggression, 47% reported a 1-point increase in confidence, whilst 33% of participants reported a 2- or 3-point increase. At 3-6 months post-training, 66% of respondents (N = 44) reported continued confidence in managing aggression with 100% of participants stating they would recommend simulation training to colleagues.

CONCLUSIONS

Simulation training is an acceptable method of training and shows promise to improve staff-perceived confidence for managing behavioural emergencies in acute paediatric health care settings. In addition, there were potential enduring positive impacts at 3 months after the study. Whilst resource and time intensive, further research assessing the benefits of utilising simulation training in this setting is warranted in order to minimise staff burn-out and improve outcomes for these very vulnerable patients.

Using Simulation to Evaluate Clinical Performance and Reasoning in Adult-Geriatric Acute Care Nurse Practitioner Students

BACKGROUND

The use of simulation-based learning experiences (SBLE) with nurse practitioner (NP) students is largely unexplored. The purpose of this pilot project was to develop, implement, and evaluate a simulated patient (SP)-SBLE to improve confidence and evaluate clinical performance and reasoning in Adult-Geriatric Acute Care NP (AG-ACNP) students.

METHOD

A mixed-methods design was used to evaluate an SP-SBLE on AG-ACNP student clinical performance and reasoning.

RESULTS

Self-confidence scores and themes from reflections indicated that students perceived this as a valuable experience that enhanced learning and confidence.

CONCLUSION

Results supported the use of SP-SBLEs to improve confidence in clinical performance and reasoning with AG-ACNP students. [J Nurs Educ. 2019;58(10):599-603.].

Formal Training Efforts to Develop Simulation Educators: An Integrative Review

STATEMENT

Formal training for educators who use simulation-based education (SBE) is required by standards of best practice, simulation guidelines, regulatory, and accrediting bodies. Training efforts to establish educator competency for SBE are being offered. However, a systematic review of this body of literature has yet to be conducted. The purpose of this integrative review was to appraise formal training efforts of educators who use SBE. The aims were to summarize the training topics, describe the structure of training programs, and explore evaluation methods of educators. The New World Kirkpatrick Model guided the review. A PRISMA search approach yielded 2007 citations of which 38 met inclusion criteria. Analysis supports a formalized training process that uses a combination of didactic material, time for repetitive practice, and ongoing feedback with longitudinal and scaffolded delivery approaches. An identified gap in the literature is threshold levels for determining competency of educators. Recommendations for planning simulation training programs are provided.

Cautionary study on the effects of pay for performance on quality of care: a pilot randomised controlled trial using standardised patients

BACKGROUND

Due to the difficulty of studying incentives in practice, there is limited empirical evidence of the full-impact pay-for-performance (P4P) incentive systems.

OBJECTIVE

To evaluate the impact of P4P in a controlled, simulated environment.

DESIGN

We employed a simulation-based randomised controlled trial with three standardised patients to assess advanced practice providers' performance. Each patient reflected one of the following: (A) indicated for P4P screenings, (B) too young for P4P screenings, or (C) indicated for P4P screenings, but screenings are unrelated to the reason for the visit. Indication was determined by the 2016 Centers for Medicare and Medicaid Services quality measures.

INTERVENTION

The P4P group was paid $150 and received a bonus of $10 for meeting each of five outcome measures (breast cancer, colorectal cancer, pneumococcal, tobacco use and depression screenings) for each of the three cases (max $300). The control group received $200.

SETTING

Learning resource centre.

PARTICIPANTS

35 advanced practice primary care providers (physician assistants and nurse practitioners) and 105 standardised patient encounters.

MEASUREMENTS

Adherence to incentivised outcome measures, interpersonal communication skills, standards of care, and misuse.

RESULTS

patient was more likely to be prescribed screenings not indicated, but highlighted by P4P: breast cancer screening (47% P4P vs 0% control, p<0.01) and colorectal cancer screening (24% P4P vs 0% control, p=0.03). The P4P group over-reported completion of incentivised measures resulting in overpayment (average of $9.02 per patient).

LIMITATIONS

A small sample size and limited variability in patient panel limit the generalisability of findings.

CONCLUSIONS

Our findings caution the adoption of P4P by highlighting the unintended consequences of the incentive system.

Reliability and validity of the simulation learning effectiveness inventory

BACKGROUND

Instruments developed to measure simulation learning outcomes need evidence of their reliability and validity for rigorous research. The purpose of this paper is to report psychometric properties of the English version of the Simulation Learning Effectiveness Inventory.

METHODS

Psychometric properties of the English version of the Simulation Learning Effectiveness Inventory, which included internal consistency reliability and construct validity with factor analysis, were examined in a sample of 132 undergraduate nursing students.

RESULTS

Cronbach's alpha coefficients were >0.70 for all subscales. There was evidence of convergent, discriminant, and known-group validity. The factor analysis resulted in some items being associated with different subscales than in the original Chinese version.

CONCLUSIONS

The English version of the Simulation Learning Effectiveness Inventory has evidence of reliability and validity. Additional psychometric studies may result in changes in some of the subscales.

Cross-cultural adaptation and validation of questionnaire for knowledge, attitudes, and behavioral intentions in Italian nurses with regard to provision of artificial nutrition and hydration to patients terminally ill with cancer

OBJECTIVES

Decision-making on artificial nutrition and hydration for patients terminally ill with cancer can be influenced by nurses' knowledge, attitudes, and behavioral intentions. A comprehensive 57-item questionnaire including six sections on the knowledge, attitudes, and behavioral intentions in providing artificial nutrition and hydration to patients terminally ill with cancer has been developed and used in Taiwan. However, the questionnaire needs further psychometric testing and adaptation for other cultures. This study aimed to cross-culturally adapt the questionnaire within the Italian cultural context and test its psychometric properties.

METHODS

The questionnaire was translated into Italian and cross-culturally adapted per the recommendations by Beaton. A panel of 10 experts assessed content validity. A multicenter cross-sectional study was conducted with 411 nurses to test its psychometric properties. Dimensionality and construct validity were assessed through exploratory and confirmatory factor analyses. Reliability was estimated by composite ω and traditional methods, such as the Kuder Richardson formula-20 and Cronbach's α coefficients.

RESULTS

The overall content validity index was 0.85. A confirmatory factor analysis was conducted for the knowledge section and the four attitudes sections. A preliminary analysis for the behavioral intentions section yielded non acceptable results. The internal consistency of the scales was adequate (range, 0.64-0.93).

CONCLUSIONS

This study constituted a notable advancement in the psychometric testing of the tool, and provides evidence that the Italian version of the questionnaire has acceptable psychometric characteristics for the sections on knowledge and attitudes.

Effects of high-fidelity simulation based on life-threatening clinical condition scenarios on learning outcomes of undergraduate and postgraduate nursing students: a systematic review and meta-analysis

OBJECTIVE

The purpose was to analyse the effectiveness of high-fidelity patient simulation (HFPS) based on life-threatening clinical condition scenarios on undergraduate and postgraduate nursing students' learning outcomes.

DESIGN

A systematic review and meta-analysis were conducted based on the Cochrane Handbook for Systematic Reviews of Interventions and its reporting was checked against the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist.

DATA SOURCES

PubMed, Scopus, CINAHL with Full Text, Wiley Online Library and Web of Science were searched until July 2017. Author contact, reference and citation lists were checked to obtain additional references.

STUDY SELECTION

To be included, available full-texts had to be published in English, French, Spanish or Italian and (a) involved undergraduate or postgraduate nursing students performing HFPS based on life-threatening clinical condition scenarios, (b) contained control groups not tested on the HFPS before the intervention, (c) contained data measuring learning outcomes such as performance, knowledge, self-confidence, self-efficacy or satisfaction measured just after the simulation session and (d) reported data for meta-analytic synthesis.

REVIEW METHOD

Three independent raters screened the retrieved studies using a coding protocol to extract data in accordance with inclusion criteria.

SYNTHESIS METHOD

For each study, outcome data were synthesised using meta-analytic procedures based on random-effect model and computing effect sizes by Cohen's d with a 95% CI.

RESULTS

Thirty-three studies were included. HFPS sessions showed significantly larger effects sizes for knowledge (d=0.49, 95% CI [0.17 to 0.81]) and performance (d=0.50, 95% CI [0.19 to 0.81]) when compared with any other teaching method. Significant heterogeneity among studies was detected.

CONCLUSIONS

Compared with other teaching methods, HFPS revealed higher effects sizes on nursing students' knowledge and performance. Further studies are required to explore its effectiveness in improving nursing students' competence and patient outcomes.