Recommendations After an Objective Structured Clinical Examination Program Evaluation

BACKGROUND

The use of objective structured clinical examinations (OSCEs) is a well-established method of clinical assessment in the medical profession, and these are currently being integrated into advanced practice nursing curricula.

METHOD

A program assessment was conducted at a current family nurse practitioner track within a southwestern U.S. university using the OSCE evaluative strategy. The logic model from the Centers for Disease Control and Prevention framework was incorporated. Both focus group and the de-identified faculty-scored student rubrics during the fall, spring, and summer semesters of one cohort of students (N = 73) were examined.

RESULTS

Strengths and challenges were found in the areas of student preparation, scoring/grading with variations of inter-rater reliability, and debriefing.

CONCLUSION

Establishing a specific OSCE center with a dedicated director could support adequate attainment of formative and summative competencies for nurse practitioners and provide faculty education and standardized student preparation, while adhering to the multiple guidelines and standards related to nurse practitioner competency evaluation with simulation. [J Contin Educ Nurs. 2023;54(4):169-175.].

Evaluation of undergraduate students’ nursing assessment and communication skills through an objective structured clinical examination within a high-fidelity simulation using a student-simulated patient†

Objective

To evaluate nursing students’ assessment and communication skills through an objective structured clinical examination (OSCE) within a high-fidelity simulation using a student-simulated patient (SSP) and analyze the validity, reliability, degree of difficulty, and discriminability of the OSCE.

Methods

A descriptive study was conducted at one nursing school in Macao. All scenarios were designed using a high-fidelity simulator combined with SSPs. A convenience sample of 54 baccalaureate nursing students completed the OSCE. The nursing assessment OSCE checklist (NAOC) and the communication evaluation rubric (CER) were used as observational measurements with three-point Likert scales (2 = Achieved, 1 = Partly achieved, 0 = Not completed/Incorrect).

Results

Difficulty coefficients of the exam were 0.63 for nursing assessment skills and 0.56 for communication skills. The discrimination index of the majority items of the NAOC (86.4%) was >0.20, showing a better to good discriminability. The items of the CER had satisfactory indexes of item discrimination (from 0.38 to 0.84). Students received high scores in conducting blood oxygen saturation and cardiac and lung auscultation but low scores in vomiting and diarrhea assessment. Students presented good communication skills in eye contacting and listening, but culture assessment needs to be improved. The students with experiences in simulation or simulated patient (SP) interactions had better assessment and communication skills than students without those experiences. There was a positive relationship between nursing assessment and communication skills (r = 0.67, P = 0.000).

Conclusions

SSPs were involved in enhancing the realism of interactions in simulated scenarios. Nursing students can conduct nursing assessments specific to patient conditions, explain the conditions to the patient, and ensure that the patient remains informed at all times of the precautions to be taken. However, students’ cultural awareness and some communication skills need further training. With moderate difficulty and high discrimination index, OSCE showed satisfactory reliability and validity.

Development and Evaluation of an Algorithm-Corresponding Instrument for Nursing Simulation

BACKGROUND

This study aimed to develop and validate an assessment instrument for students engaging with an algorithm-based simulation scenario addressing emergency measures for high-risk newborns with apnea in a neonatal intensive care unit.

METHOD

The study was conducted in two phases of development and evaluation of the algorithm-corresponding instrument. One hundred sixty-nine senior nursing students from two universities in South Korea were evaluated using the developed instrument.

RESULTS

The developed and validated instrument consisted of three dimensions (assessment points, nursing skills, and communication) measured through 13 items. The exploratory factor analysis revealed three factors of the instrument, and the confirmatory factor analysis demonstrated a better model fit for a three-factor instrument model than for other models.

CONCLUSION

The developed algorithm-corresponding assessment instrument is suitable for assessing the clinical decision-making ability of nursing students in a simulation scenario. [J Nurs Educ. 2020;59(11):617-626.].

Exploring User Needs in the Development of a Virtual Reality-Based Advanced Life Support Training Platform: Exploratory Usability Study

BACKGROUND

Traditional methods of delivering Advanced Life Support (ALS) training and reaccreditation are resource-intensive and costly. Interactive simulations and gameplay using virtual reality (VR) technology can complement traditional training processes as a cost-effective, engaging, and flexible training tool.

OBJECTIVE

This exploratory study aimed to determine the specific user needs of clinicians engaging with a new interactive VR ALS simulation (ALS-SimVR) application to inform the ongoing development of such training platforms.

METHODS

Semistructured interviews were conducted with experienced clinicians (n=10, median age=40.9 years) following a single playthrough of the application. All clinicians have been directly involved in the delivery of ALS training in both clinical and educational settings (median years of ALS experience=12.4; all had minimal or no VR experience). Interviews were supplemented with an assessment of usability (using heuristic evaluation) and presence.

RESULTS

The ALS-SimVR training app was well received. Thematic analysis of the interviews revealed five main areas of user needs that can inform future design efforts for creating engaging VR training apps: affordances, agency, diverse input modalities, mental models, and advanced roles.

CONCLUSIONS

This study was conducted to identify the needs of clinicians engaging with ALS-SimVR. However, our findings revealed broader design considerations that will be crucial in guiding future work in this area. Although aligning the training scenarios with accepted teaching algorithms is important, our findings reveal that improving user experience and engagement requires careful attention to technology-specific issues such as input modalities.

The emerging literature for the triad of health informatics, healthcare quality and safety, and healthcare simulation

The areas of health informatics, healthcare quality and safety, and healthcare simulation are often thought of as separate domains. The purpose of this position paper is to report on the interdependence that is emerging as an important triad across the healthcare/health system continuum. A qualitative review of 24 studies suggests the interdependence of health informatics, healthcare quality and safety, and healthcare simulation reaches much broader than traditional utilisation of simulation. We suggest ways that organisations can take advantage of the interdependence of this triad across a broader variety of healthcare environments, including teamwork, communication, and complex system relationships. In conclusion, the reviewed 24 studies suggest that the research in the triad focuses on simulation education and computerised simulation, and when coupled with health informatics, bears greater strength on quality improvement or patient safety.

Validation of time to task performance assessment method in simulation: A comparative design study

BACKGROUND

There is a lack of objective and valid measures for assessing nursing clinical competence which could adversely impact patient safety. Therefore, we evaluated an objective assessment of clinical competence, Time to Task (ability to perform specific, critical nursing care activities within 5 min), and compared it to two subjective measures, (Lasater Clinical Judgement Rubric [LCJR] and common "pass/fail" assessment).

DESIGN/METHODS

Using a prospective, "Known Groups" (Expert vs. Novice nurses) comparative design, Expert nurses (ICU nurses with >5 years of ICU experience) and Novice nurses (senior prelicensure nursing students) participated individually in a simulation of a patient in decompensated heart failure. Fourteen nursing instructors or preceptors, blinded to group assignment, reviewed 28 simulation videos (15 Expert and 13 Novice) and scored them using the LCJR and pass/fail assessments. Time to Task assessment was scored based on time thresholds for specific nursing actions prospectively set by an expert clinical panel. Statistical analysis consisted of Medians Test and sensitivity and specificity analyses.

RESULTS

The LCJR total score was significantly different between Experts and Novices (p < 0.01) and revealed adequate sensitivity (ability to correctly identify "Expert" nurses; 0.72) but had a low specificity (ability to correctly identify "Novice" nurses; 0.40). For the subjective measure 'pass/fail', sensitivity was high (0.90) but specificity was low (0.47). The Time to Task measure had statistical significance between Expert and Novice groups (p < 0.01) and sensitivity (0.80) and specificity (0.85) were good.

CONCLUSION

Commonly used subjective measures of clinical nursing competence have difficulties with achieving acceptable specificity. However, an objective measure, Time to Task, had good sensitivity and specificity in differentiating between groups. While more than one assessment instrument should be used to determine nurse competency, an objective measure, such as Time to Task, warrants further study.