Time-to-Task Validation in Sequential Simulations as a Component of Nursing Student Competency

BACKGROUND

Time goals have not been tested as a component of nursing competence.

METHODS

Using a known-groups approach, a multisite study was performed on prelicensure nursing students (novices) and experienced nurses (experts) who participated in 2 simulations of patient deterioration with time goals. Video capture was achieved with an eye tracker.

RESULTS

While all novices did not achieve the time goals in both simulations, there was a significant improvement between the 2 simulations (P = .04). Experts were more successful in achieving the time-to-task goals in both simulations (44% and 95%, respectively). Statistical significance was found between the novice and expert groups in achieving the time goal for both simulations (P < .01).

CONCLUSIONS

This study was able to differentiate the expert group as being more proficient in a timed assessment, although the novices showed improvement between 2 simulations. This illustrates an opportunity for skill development among novices in managing patient deterioration amenable to time as an objective assessment.

Comparing formative and summative simulation-based assessment in undergraduate nursing students: nursing competency acquisition and clinical simulation satisfaction

BACKGROUND

Formative and summative evaluation are widely employed in simulated-based assessment. The aims of our study were to evaluate the acquisition of nursing competencies through clinical simulation in undergraduate nursing students and to compare their satisfaction with this methodology using these two evaluation strategies.

METHODS

Two hundred eighteen undergraduate nursing students participated in a cross-sectional study, using a mixed-method. MAES© (self-learning methodology in simulated environments) sessions were developed to assess students by formative evaluation. Objective Structured Clinical Examination sessions were conducted to assess students by summative evaluation. Simulated scenarios recreated clinical cases of critical patients. Students´ performance in all simulated scenarios were assessed using checklists. A validated questionnaire was used to evaluate satisfaction with clinical simulation. Quantitative data were analysed using the IBM SPSS Statistics version 24.0 software, whereas qualitative data were analysed using the ATLAS-ti version 8.0 software.

RESULTS

Most nursing students showed adequate clinical competence. Satisfaction with clinical simulation was higher when students were assessed using formative evaluation. The main students' complaints with summative evaluation were related to reduced time for performing simulated scenarios and increased anxiety during their clinical performance.

CONCLUSION

The best solution to reduce students' complaints with summative evaluation is to orient them to the simulated environment. It should be recommended to combine both evaluation strategies in simulated-based assessment, providing students feedback in summative evaluation, as well as evaluating their achievement of learning outcomes in formative evaluation.

An integrative review of simulation, senior practicum and readiness for practice

Readiness for practice (RFP) is essential as new nurses will practice in a nursing shortage climate, with high acuity patients and complex technology. Nurse educators are challenged to foster readiness for practice.

AIM

The aim of this integrative review was to explore the impact of simulation and senior practicum on graduating senior nursing students' readiness for practice.

BACKGROUND

Readiness for practice has been a point of concern for healthcare for quite some time. This has led to a need for better understanding for what RFP means including their perceptions of various stakeholders. Further, it is important to address what teaching and learning strategies can be implemented to assist in ensuring the graduating senior nursing student is ready for practice on graduation and for their first professional practice. Readiness for practice is defined as the ability to safely and competently care for patients by synthesizing theory, skills, attitudes and values in applying clinical reasoning in practice settings.

METHOD

An integrative review identified 48 published papers on simulation and senior practicum that met the inclusion criteria.

RESULTS

Themes identified from the senior practicum/preceptor literature included clinical reasoning, skills, barriers to the senior practicum and transition. Themes identified regarding simulation as a strategy included preparation, competence development, clinical reasoning and the number of human patient simulators used.

CONCLUSION

Although the evidence is not conclusive, simulation and senior practicums have the promise of preparing graduating nursing students for their first professional practice. However, nurse educators must ensure simulations are well-organized with clear objectives and that preceptors are supported in teaching and assessing students.

Reliability of simulation-based assessment for practicing physicians: performance is context-specific

INTRODUCTION

Even physicians who routinely work in complex, dynamic practices may be unprepared to optimally manage challenging critical events. High-fidelity simulation can realistically mimic critical clinically relevant events, however the reliability and validity of simulation-based assessment scores for practicing physicians has not been established.

METHODS

Standardised complex simulation scenarios were developed and administered to board-certified, practicing anesthesiologists who volunteered to participate in an assessment study during formative maintenance of certification activities. A subset of the study population agreed to participate as the primary responder in a second scenario for this study. The physicians were assessed independently by trained raters on both teamwork/behavioural and technical performance measures. Analysis using Generalisability and Decision studies were completed for the two scenarios with two raters.

RESULTS

The behavioural score was not more reliable than the technical score. With two raters > 20 scenarios would be required to achieve a reliability estimate of 0.7. Increasing the number of raters for a given scenario would have little effect on reliability.

CONCLUSIONS

The performance of practicing physicians on simulated critical events may be highly context-specific. Realistic simulation-based assessment for practicing physicians is resource-intensive and may be best-suited for individualized formative feedback. More importantly, aggregate data from a population of participants may have an even higher impact if used to identify skill or knowledge gaps to be addressed by training programs and inform continuing education improvements across the profession.

High-Fidelity Virtual Objective Structured Clinical Examinations with Standardized Patients in Nursing Students: An Innovative Proposal during the COVID-19 Pandemic

In response to the cancellation of in-person objective structured clinical examinations (OSCEs) prompted by confinement due to the COVID-19 pandemic, we designed a solution to adapt our traditional OSCEs to this new reality in nursing education. We implemented an innovative teaching proposal based on high-fidelity virtual OSCEs with standardized patients. The purposes of our study were to describe this innovative teaching proposal and compare nursing competence acquisition in final year nursing students through virtual and in-person OSCE modalities. The study included 234 undergraduate students: 123 students were assessed through high-fidelity virtual OSCEs during May 2020, whereas 111 students were assessed through in-person OSCEs during May 2019. The structure of OSCEs, including its stations, clinical simulated scenarios, and checklists, was the same in both OSCE modalities. The effect size of the differences among the competence categories of checklists, including their total scores, was small. Regarding our virtual OSCEs was similarly successful to in-person OSCEs, this online format was found to be useful, feasible, and cost-saving when in-person OSCE was not possible. Therefore, high-fidelity virtual OSCEs with standardized patients could be considered as another choice of OSCE not only in the current COVID-19 pandemic but could also be extended to normal situations, even post-pandemic.

The Effect of Evaluator Training on Inter- and Intrarater Reliability in High-Stakes Assessment in Simulation

AIM

The aim of this study was to evaluate the effectiveness of a training intervention in achieving inter- and intrarater reliability among faculty raters conducting high-stakes assessment of clinical performance in simulation.

BACKGROUND

High-stakes assessment of simulation performance is being adopted in nursing education. However, limited research exists to guide best practices in training raters, which is essential to ensure fair and defensible assessment.

METHOD

A nationwide sample of 75 prelicensure RN program faculty participated in an experimental, randomized, controlled study.

RESULTS

Participants completing a training intervention achieved higher inter- and intrarater reliability than control group participants when using a checklist evaluation tool. Mixed results were achieved by participants when completing a global competency assessment.

CONCLUSION

The training intervention was effective in helping participants to achieve a shared mental model for use of a checklist, but more time may be necessary to achieve consistent global competence decisions.

Waging the War on Clinical Grade Inflation: The Ongoing Quest

This study examined the presence of grade inflation in clinical courses 9 years after implementing strategies to improve grading precision. A comparison of clinical grades for cohort I (1997-2002) with cohort II (2009-2016) showed statistically lower grades in 2 courses (Adult 1 and Maternity) for cohort II. Suggestions for changing the way clinical experiences are planned, executed, and evaluated are provided.

Establishing Inter- and Intrarater Reliability for High-Stakes Testing Using Simulation

AIM

This article reports one method to develop a standardized training method to establish the inter- and intrarater reliability of a group of raters for high-stakes testing.

BACKGROUND

Simulation is used increasingly for high-stakes testing, but without research into the development of inter- and intrarater reliability for raters.

METHOD

Eleven raters were trained using a standardized methodology. Raters scored 28 student videos over a six-week period. Raters then rescored all videos over a two-day period to establish both intra- and interrater reliability.

RESULTS

One rater demonstrated poor intrarater reliability; a second rater failed all students. Kappa statistics improved from the moderate to substantial agreement range with the exclusion of the two outlier raters' scores.

CONCLUSION

There may be faculty who, for different reasons, should not be included in high-stakes testing evaluations. All faculty are content experts, but not all are expert evaluators.