Evaluation of Undergraduate Students Using Objective Structured Clinical Evaluation

Feasibility and reliability of the pandemic-adapted online-onsite hybrid graduation OSCE in Japan

Objective structured clinical examination (OSCE) is widely used to assess medical students' clinical skills. Virtual OSCEs were used in place of in-person OSCEs during the COVID-19 pandemic; however, their reliability is yet to be robustly analyzed. By applying generalizability (G) theory, this study aimed to evaluate the reliability of a hybrid OSCE, which admixed in-person and online methods, and gain insights into improving OSCEs' reliability. During the 2020-2021 hybrid OSCEs, one examinee, one rater, and a vinyl mannequin for physical examination participated onsite, and a standardized simulated patient (SP) for medical interviewing and another rater joined online in one virtual breakout room on an audiovisual conferencing system. G-coefficients and 95% confidence intervals of the borderline score, namely border zone (BZ), under the standard 6-station, 2-rater, and 6-item setting were calculated. G-coefficients of in-person (2017-2019) and hybrid OSCEs (2020-2021) under the standard setting were estimated to be 0.624, 0.770, 0.782, 0.759, and 0.823, respectively. The BZ scores were estimated to be 2.43-3.57, 2.55-3.45, 2.59-3.41, 2.59-3.41, and 2.51-3.49, respectively, in the score range from 1 to 6. Although hybrid OSCEs showed reliability comparable to in-person OSCEs, they need further improvement as a very high-stakes examination. In addition to increasing clinical vignettes, having more proficient online/on-demand raters and/or online SPs for medical interviews could improve the reliability of OSCEs. Reliability can also be ensured through supplementary examination and by increasing the number of online raters for a small number of students within the BZs.

The effect of objective structured clinical examinations for nursing students

Twenty core nursing skills have been identified by the Korean Accreditation Board of Nursing Education. Proficiency in these skills is essential for all nursing professions, and many educational strategies exist to develop these skills in nursing students, including the Objective Structured Clinical Examination (OSCE). To date, no study on the effects of the OSCE on nursing education has been published. Therefore, we evaluated the effects of the OSCE on the core nursing skills of 207 pre-licensure nursing students in Korea. We measured the nursing students' confidence, skills, and knowledge acquisition and retention. A one-way analysis of variance and Fisher's least significant difference were used for data analysis. Among the four nursing areas (fall, transfusion, pre-operative, and post-operative), students demonstrated the highest confidence level scores in pre-operative nursing. On the OSCE, students scored the highest in transfusion nursing. Significant differences were found between prior knowledge, knowledge acquisition, and retention. Our findings confirm that the OSCE, after lectures and core nursing skill practice, improved the retention of nursing students' knowledge. Therefore, this program can positively influence nursing students' knowledge level, and implementing the OSCE can strengthen students' clinical competency.

Has the OSCE Met Its Final Demise? Rebalancing Clinical Assessment Approaches in the Peri-Pandemic World

The Objective Structured Clinical Examination (OSCE) has been traditionally viewed as a highly valued tool for assessing clinical competence in health professions education. However, as the OSCE typically consists of a large-scale, face-to-face assessment activity, it has been variably criticized over recent years due to the extensive resourcing and relative expense required for delivery. Importantly, due to COVID-pandemic conditions and necessary health guidelines in 2020 and 2021, logistical issues inherent with OSCE delivery were exacerbated for many institutions across the globe. As a result, alternative clinical assessment strategies were employed to gather assessment datapoints to guide decision-making regarding student progression. Now, as communities learn to “live with COVID”, health professions educators have the opportunity to consider what weight should be placed on the OSCE as a tool for clinical assessment in the peri-pandemic world. In order to elucidate this timely clinical assessment issue, this qualitative study utilized focus group discussions to explore the perceptions of 23 clinical assessment stakeholders (examiners, students, simulated patients and administrators) in relation to the future role of the traditional OSCE. Thematic analysis of the FG transcripts revealed four major themes in relation to participants' views on the future of the OSCE vis-a-vis other clinical assessments in this peri-pandemic climate. The identified themes are (a) enduring value of the OSCE; (b) OSCE tensions; (c) educational impact; and (d) the importance of programs of assessment. It is clear that the OSCE continues to play a role in clinical assessments due to its perceived fairness, standardization and ability to yield robust results. However, recent experiences have resulted in a diminishing and refining of its role alongside workplace-based assessments in the new, peri-pandemic programs of assessment. Future programs of assessment should consider the strategic positioning of the OSCE within the context of utilizing a range of tools when determining students' clinical competence.

An Innovative Evaluation in Fundamental Nursing Curriculum for Novice Nursing Students: An Observational Research

BACKGROUND

<Fundamental Nursing> is the first specialized curriculum for nursing students, laying a foundation for nursing education. <Fundamental Nursing> includes many important basic nursing skills, which suggests the traditional examination could not evaluate how well students learn. Since the OSCE (objective structured clinical examination) developed, it has been used among newly graduated nurses, postgraduate nursing students and achieved some success. OSCE might be a better way to evaluate and reflect students' fundamental nursing skills and comprehensive abilities, although few studies have put it into practice.

OBJECTIVES

To build an examination model relating to <Fundamental Nursing> based on OSCE, and put this examination model into practice, and then evaluate the reliability and validity of this examination model.

DESIGN

This study is to design an examination model based on literature and teaching purposes of <Fundamental Nursing>. A quantitative descriptive survey methodology was used to obtain the students' outcome and the views of the students on the process and outcome.

SETTINGS

This study was conducted in the nursing school of one comprehensive university in Beijing, China.

PARTICIPANTS

99 undergraduate nursing students in second year participated in this study.

METHODS

The OSCE consisted of 4 interactive simulated patient stations, 2 operating stations without standard patients, 2 theoretical stations. We designed 4 standardized process-based case scenarios and corresponding scripts pertinent to each station. 4 standard patients and 8 teachers of <Fundamental Nursing> were recruited, trained and evaluated before the implementation of objective structured clinical examination.

RESULTS

The average score of overall fundamental nursing skills was 90.51 ± 4.00. The Cronbach alpha was 0.665, demonstrating the OSCE is reliable. The content validity of each station varies from 0.90-1, and the criterion validity was promising (R = 0.661, p = 0.000). Also, most students were in favor of OSCE to evaluate in <Fundamental Nursing>.

CONCLUSIONS

The OSCE is a reliable and valid assessment tool in <Fundamental Nursing>. Meanwhile, it could inspire students' critical thinking and promote students' communication skills. Through successive application of OSCE in nursing education from the very beginning, it might also have some potential effect on the clinical performance.

Debriefing to Improve Student Ability to Assess and Plan for the Care of Persons With Disability

INTRODUCTION

Although recent literature suggests that students should be trained in the care of persons with disability (PWDs) as a form of cultural sensitivity (CS), healthcare professionals may receive limited experience during their formal training. After pharmacy students in 2 previous years of testing failed to adequately assess and plan for the care of a standardized patient's chief complaint and disability in an Objective Structured Clinical Examination (OSCE), the investigators added debriefing to the OSCE to determine if it would improve student's ability to assess and plan for the care of PWD.

METHODS

Two sequentially enrolled second-year pharmacy school student cohorts participated in this study (control n = 90; intervention n = 82). During the OSCE, students interviewed and examined a standardized patient with a simulated physical disability and other chronic disease states. Students were then instructed to develop a care plan considering the patient's disability and other disease states. The intervention cohort received debriefing; the control did not. Students documented the care plan in a subjective, objective, assessment, and plan (SOAP) note. Investigators assessed SOAP note score (general ability of students to write a SOAP note) and CS score (specific ability to care for PWD) to determine the effectiveness of the debriefing.

RESULTS

The intervention group showed a significantly higher percent mean CS score than the control group (93.6% ± 19% and 61.1% ± 30.7%, respectively, P < 0.001), translating to a mean of 56.2/60 points earned for the intervention group and 36.7/60 points earned for the control group. Scores ranged from 0 to 60 points for both intervention and control groups. Students in the intervention group had an absolute improvement in pass rates (those students scoring ≥70% on the OSCE) of 59.4% with 92.7% of the students passing in the intervention group versus 33.3% of the students passing in the control group (P < 0.001). The overall SOAP note scores were no different between the 2 cohorts (P = 0.353).

CONCLUSIONS

Debriefing added to an OSCE improved students' performance in developing care plans for disabled patients. Ideally, longitudinal studies should be completed to determine if these skills transfer from debriefings to clinical practice. Development of effective training and assessment methods is essential for students to obtain adequate skills and knowledge to care for persons with disabilities.

Dental students' perception of the Objective Structured Clinical Examination (OSCE): The Taibah University experience, Almadinah Almunawwarah, KSA

Objective

The Objective Structured Clinical Examination (OSCE) is an assessment tool used to objectively evaluate clinical competence in medical schools and, more recently, in the nursing profession as well. However, few studies have been conducted to elicit the views of dental students in KSA regarding the OSCE. The present study was designed to explore, evaluate, and analyse students' perceptions of the OSCE and to explore the strengths and weaknesses of this assessment tool as perceived by dental students.

Methods

This was a cross-sectional analytical observational study using a survey design. It took place in the College of Dentistry, Taibah University, KSA. The study sample consisted of third, fourth, and fifth-year students who took the OSCE assessment during their studies.

Results

Of 138 invited students, 119 responded (response rate of 86.2%). Almost half the students (47.1%) agreed that the OSCE exam was fair, while 19.3% agreed that a broad range of clinical skills were included in the OSCE exam. A low percentage of students believed to a great extent that the scores were standardised; however, the practicality and usefulness of the OSCE was questioned. Students perceived that issues pertaining to personality, ethnicity, and gender would not impact the scores.

Conclusion

The findings of this study indicated that the OSCE is a meaningful and fair assessment tool for clinical skills.

Towards the implementation of OSCE in undergraduate nursing curriculum: A qualitative study

BACKGROUND

Despite the fact that Objective Structured Clinical Examination is considered to be an efficient assessment method, their implementation in the undergraduate nursing curriculum encounters difficulties. However, the initiative of the European Higher Education Area to promote competency-based assessment may represent an opportunity to introduce this instrument in undergraduate nursing curriculum.

OBJECTIVE

To explore the perception of nursing faculty members regarding the implementation of the OSCE as an assessment tool in Catalan Nursing Schools.

DESIGN/PARTICIPANTS/SETTING

In this qualitative study, fifteen teachers participated in semi-structured interviews in eight Catalan Nursing Schools.

METHODS

Semi-structured interviews were conducted. A thematic content analysis was used to identify major themes in the interview data and collaborative analysis was undertaken to ensure rigorous results.

RESULTS

The relevant aspects that are emphasized by teachers included the consideration of the dual purpose of the OSCE via its formative and evaluative facets by enhancing the feedback received by students about their performance on the OSCE. The OSCE should be administered towards the end of the degree program and should complement other methods of assessment. Despite its high cost, the OSCE was deemed to be efficient as it enables student competencies to be assessed with objective criteria, which is a difficult task with other instruments. OSCE implementation is feasible with the institutional support of and collaborative work between schools.

CONCLUSIONS

The implementation of the OSCE in the Catalan undergraduate nursing degree programs is feasible if the project receives the support of all involved parties and if creative strategies are determined to reduce economic costs and optimize resources. With adequate feedback, the OSCE is an assessment tool that can provide high-impact training to students.

Assessing Nursing Students' Clinical Competencies Using a Problem-Focused Objective Structured Clinical Examination

The objectives of this study were to develop, implement, and evaluate an innovative modified Objective Structured Clinical Examination (OSCE) model, and to compare students' performance of different clinical skills as assessed by standardized patients and OSCE examiners. Data were obtained from final year undergraduate students undergoing the modified OSCE as a graduation examination. Seventy-seven students rotated through four stations (nine substations). Standardized patients scored students higher than examiners in history taking (9.14 ± 0.92 vs. 8.42 ± 0.85), response to emergency event (8.88 ± 1.12 vs. 7.62 ± 1.54), executive medical orders (8.77 ± 0.96 vs. 8.25 ± 1.43), technical operation (18.21 ± 1.26 vs. 16.91 ± 1.35), nursing evaluation (4.53 ± 0.28 vs. 4.29 ± 0.52), and health education stations (13.79 ± 1.31 vs. 11.93 ± 2.25; p < .01). In addition, the results indicated that the difference between standardized patient and examiner scores for physical examination skills was nonsignificant (8.70 ± 1.18 vs. 8.80 ± 1.27; p > .05). The modified, problem-focused, and nursing process-driven OSCE model effectively assessed nursing students' clinical competencies, and clinical and critical thinking.

Improving and validating children's nurses communication skills with standardized patients in end of life care

Children's nurse education is experiencing increases in recruitment targets at the same time that clinical placements are decreasing. With regard to end-of-life care, it is has become a challenge to ensure that all students come into contact with a satisfactory range of experience as part of the requirement for competency at the point of registration. The aim of our study was to find out if students at the end of their course were able to use communication skills acquired in their three years of training and adapt and transfer them to a specific palliative care context even if they had never worked in that area of care. Focus groups were conducted after the simulations which explored the students' experiences of being involved in the scenarios. Four themes emerged that students identified either inhibited or enabled their communication skills, which included anxiety and fear, the need for professional props, the experience of it being real and feeling empowered.

Doing the Right Thing at the Right Time: Assessing Responses to Patient Deterioration in Electronic Simulation Scenarios Using Course-of-Action Analysis

International studies indicate that the recognition and management of deteriorating patients in hospitals are poor and that patient assessment is often inadequate. Face-to-face simulation programs have been shown to have an impact on educational and clinical outcomes; however, little is known about performance in contemporary healthcare e-simulation approaches. Using data from an open-access Web-based patient deterioration program (FIRSTACTWeb), the performance of 367 Australian nursing students in identification of treatment priorities and clinical actions was analyzed using a military model of Course of Action Simulation Analysis. Participants' performance in the whole program demonstrated a significant improvement in knowledge and skills (P ≤ .001) with high levels of participant satisfaction. Course of Action Simulation Analysis modeling identified three key participant groupings within which only 18% took the "best course of action" (the right actions and timing), with most (70%) completing the right actions but in the wrong order. The remaining 12% produced incomplete assessments and actions in an incorrect sequence. Contemporary approaches such as e-simulation do enhance educational outcomes. Measurement of performance when combined with Course of Action Simulation Analysis becomes a useful tool in the description of outcomes, an understanding of decision making, and the prediction of future events.

Undergraduate nursing students' subjective attitudes to curriculum for Simulation-based objective structured clinical examination

BACKGROUND

The objective structured clinical examination (OSCE) is an effective educational method to evaluate the clinical competencies of students.

OBJECTIVE

To identify undergraduate nursing students' subjective attitudes to a curriculum for simulation-based OSCE (C-SOSCE).

DESIGN

Q methodology was used since it integrates the use of quantitative and qualitative methods.

PARTICIPANTS

A convenience P-sample consisting of 44 third-year nursing students.

METHODS

Forty selected Q-statements were obtained from each participant and were classified into a normal distribution by using a 9-point bipolar scale. The collected data were analyzed using the pc-QUANL program.

RESULTS

Three discrete factors emerged: Factor I (C-SOSCE stimulates self-study; self-directed learning perspectives), Factor II (C-SOSCE promotes self-confidence; practical learning perspectives), and Factor III (C-SOSCE needs proficient personnel; constructive learning perspectives).

CONCLUSIONS

Subjective positions of the three factors that were identified can be applied during the planning stages of an effective C-SOSCE. These findings establish a baseline that will lead to further research.

Have OSCEs come of age in nursing education?

This article is intended to contribute to the current debate as to whether the objective structured clinical examination (OSCE) should become a standard assessment tool for undergraduate nursing education as they currently are for medicine. The authors describe how one UK university developed an OSCE for a nursing undergraduate programme with the aim of emphasising the need for nursing students to be competent in clinical skills and offering a means of standardising the assessment of these skills. There has been an increasing number of research studies carried out in this area at international level and this article's main contribution to the literature is the description of the Angoff standard-setting procedure that was used to calibrate the OSCE at this University and which makes it the first nursing OSCE in the UK to incorporate a scientific standard-setting procedure.

Virtual reality simulation: using three-dimensional technology to teach nursing students

The use of computerized technology is rapidly growing in the classroom and in healthcare. An emerging computer technology strategy for nursing education is the use of virtual reality simulation. This computer-based three-dimensional educational tool simulates real-life patient experiences in a risk-free environment, allows for repeated practice sessions, requires clinical decision making, exposes students to diverse patient conditions, provides immediate feedback, and is portable. The purpose of this article was to review the importance of virtual reality simulation as a computerized teaching strategy. In addition, a project to explore readiness of nursing faculty at one major Midwestern university for the use of virtual reality simulation as a computerized teaching strategy is described where faculty thought virtual reality simulation would increase students' knowledge of an intravenous line insertion procedure. Faculty who practiced intravenous catheter insertion via virtual reality simulation expressed a wide range of learning experiences from using virtual reality simulation that is congruent with the literature regarding the barriers to student learning. Innovative teaching strategies, such as virtual reality simulation, address barriers of increasing patient acuity, high student-to-faculty ratio, patient safety concerns from faculty, and student anxiety and can offer rapid feedback to students.

Staff and students' perceptions and experiences of teaching and assessment in Clinical Skills Laboratories: interview findings from a multiple case study

BACKGROUND

The Clinical Skills Laboratory has become an essential structure in nurse education and several benefits of its use have been identified. However, the literature identifies the need to examine the transferability of skills learned there into the reality of practice.

OBJECTIVE

This research explored the role of the Clinical Skills Laboratory in preparing nursing students for the real world of practice. This paper focuses specifically on the perceptions of the teaching and assessment strategies employed there.

DESIGN

Qualitative multiple case study design.

SETTING

Five case study sites.

PARTICIPANTS

Interviewees (n=58) included academic staff, clinical staff and nursing students.

METHODS

Semi-structured interviews.

RESULTS

The Clinical Skills Laboratory can provide a pathway to practice and its authenticity is significant. Teaching strategies need to incorporate communication as well as psychomotor skills. Including audio-visual recording into assessment strategies is beneficial. Effective relationships between education institutions and clinical settings are needed to enhance the transferability of the skills learned.

CONCLUSIONS

The Clinical Skills Laboratory should provide an authentic learning environment, with the appropriate use of teaching strategies. It is crucial that effective links between educators and clinical staff are established and maintained.

Identifying strengths and weaknesses in the utilization of Objective Structured Clinical Examination (OSCE) in a nursing program

Objective Structured Clinical Examination (OSCE) is used in nursing to assess students' transfer of classroom and laboratory learning experiences into simulated clinical practice. OSCE is a performance-based exam in which students are observed demonstrating a multitude of clinical behaviors. The purpose of this study was to identify strengths and weaknesses in the utilization of OSCE in this nursing program with 60 full-time students ages 21 to 23. An evaluation methodology was used for this study. Interviews were conducted with two groups: faculty facilitators of OSCE and standardized patients (SPs). Areas of focus were: data collection of students' performance, SP selection and training, and modification of the Nursing Interview Interaction Scale (NIIS). It was found that with appropriate SP selection and training, utilization of appropriate tools, and good data collection, OSCE can offer a valid and reliable means of testing nursing students' clinical competencies.

Use of anecdotal notes by clinical nursing faculty: a descriptive study

Although the use of anecdotal notes by faculty to document clinical performance is thought to be a common practice, no empirical study of this evaluation tool has been conducted. To investigate the frequency and pattern of use, a faculty questionnaire was developed using the Context, Input, Process, Product (CIPP) evaluation model as a framework. The model was adapted to focus on clinical nursing education. Sixty-four nursing faculty from six schools participated in the regional study. A descriptive design was used to collect quantitative data from clinical faculty. Findings indicated that 97% of clinical faculty use anecdotal notes during the student evaluation process, and the majority of faculty do so on a weekly basis. Based on faculty feedback and the CIPP evaluation model, a clinical nursing faculty tool was developed after study completion to support clinical faculty in note use.

Personal protective equipment in health care: can online infection control courses transfer knowledge and improve proper selection and use?

We used observational evaluation to assess the ability of an online learning course to effectively transfer knowledge on personal protective equipment (PPE) selection and removal. During orientations for new hospital staff, 117 participants applied either airborne, droplet, or contact precautions in mock scenarios. Postcourse, all 3 scenarios demonstrated improvement in PPE sequence scores (P = .001); moreover, hand hygiene also was more frequent during both donning and doffing of PPE (P < .001).