Objective structured clinical examination (OSCE): review of literature and implications for nursing education

The use of the objective structured clinical examination to evaluate paediatric cardiopulmonary resuscitation skills in medical students and measures to improve training

BACKGROUND

The main objectives of the study were to analyse the use of the Objective Structured Clinical Examination (OSCE) to evaluate the skills of medical students in paediatric basic life support (PBLS), to compare two resuscitation training models and to evaluate the measures to improve the teaching program.

METHODS

Comparative, prospective, observation study with intervention in two hospitals, one undergoing a PILS course (Paediatric Immediate Life Support) and another PBLS. The study was performed over three phases. 1º. PBLS OSCE in 2022 three months after the resuscitation training 2. Measures to improve the training program in 2023 3. PBLS OSCE in 2023. Overall results were analysed and comparison between both sites and those for 2022 and 2023 were made.

RESULTS

A total of 210 and 182 students took part in the OSCE in 2022 and 2023, respectively. The overall mean score out of 100 was 83.2 (19), 77.8 (19.8) in 2022 and 89.5 (15.9) and 2023, P < 0.001. Overall cardiopulmonary resuscitation (CPR) effectiveness was adequate in 79.4% and 84.6% of students in 2022 and 2023, respectively. The results of hospital students undergoing a PILS course were better (86.4 (16.6) than those undergoing a PBLS. 80.2 (20.6) p < 0.001. The results from both hospitals improved significantly in 2023.

CONCLUSIONS

The OSCE is a valid instrument to evaluate PBLS skills in medical students and to compare different training methods and program improvements. Medical students who receive a PILS attain better PBLS skills than those who undergo a PBLS course.

Knowledge Mapping and Global Trends in the Field of the Objective Structured Clinical Examination: Bibliometric and Visual Analysis (2004-2023)

BACKGROUND

The Objective Structured Clinical Examination (OSCE) is a pivotal tool for assessing health care professionals and plays an integral role in medical education.

OBJECTIVE

This study aims to map the bibliometric landscape of OSCE research, highlighting trends and key influencers.

METHODS

A comprehensive literature search was conducted for materials related to OSCE from January 2004 to December 2023, using the Web of Science Core Collection database. Bibliometric analysis and visualization were performed with VOSviewer and CiteSpace software tools.

RESULTS

Our analysis indicates a consistent increase in OSCE-related publications over the study period, with a notable surge after 2019, culminating in a peak of activity in 2021. The United States emerged as a significant contributor, responsible for 30.86% (1626/5268) of total publications and amassing 44,051 citations. Coauthorship network analysis highlighted robust collaborations, particularly between the United States and the United Kingdom. Leading journals in this domain-BMC Medical Education, Medical Education, Academic Medicine, and Medical Teacher-featured the highest volume of papers, while The Lancet garnered substantial citations, reflecting its high impact factor (to be verified for accuracy). Prominent authors in the field include Sondra Zabar, Debra Pugh, Timothy J Wood, and Susan Humphrey-Murto, with Ronaldo M Harden, Brian D Hodges, and George E Miller being the most cited. The analysis of key research terms revealed a focus on "education," "performance," "competence," and "skills," indicating these are central themes in OSCE research.

CONCLUSIONS

The study underscores a dynamic expansion in OSCE research and international collaboration, spotlighting influential countries, institutions, authors, and journals. These elements are instrumental in steering the evolution of medical education assessment practices and suggest a trajectory for future research endeavors. Future work should consider the implications of these findings for medical education and the potential areas for further investigation, particularly in underrepresented regions or emerging competencies in health care training.

Language tests and tests of competence for registration purposes - a scoping review of the experiences of overseas nurses

BACKGROUND

The overseas applicant's capability of practising safely and effectively is proven through the tests of competence which consist of computer-based tests and the Objective Structured Clinical Examination (OSCE). All prospective applicants to the Nursing and Midwiferey Council (NMC) register must be able to demonstrate that their skills, knowledge and behaviours are at the level required to meet the NMC preregistration nursing or midwifery standards for the United Kingdom (UK).

AIM

The aim of this review is to explore the challenges faced whilst undertaking these tests of competence, the OSCE, by overseas educated nurses who aspire for Nursing and Midwifery Council (NMC) registration in the UK.

METHODS

A scoping review using the Arksey and O'Malley (2005) framework was conducted to explore and produce a profile of the existing literature on the registration requirements of the NMC. A search of CINAHL, Medline and Scopus resulted in 150 records, which were then screened against the inclusion criteria - English Language, publication between 2015 onwards and discussed the language tests/competency tests required for gaining entry to the NMC register. A total of nine articles met the criteria and are included in this scoping review. The PRISMA-ScR framework is used to present the review.

RESULTS

There was a paucity of studies that addressed the experience of overseas nurses who faced the OSCE. An interpretative stance was adopted to formulate the themes which were: competence/practice disparity, arbitrary issues for failing, failure to capture the digital health agenda, financial implications, and consequences of failing the OSCE. The results raise concern whether the nurses from overseas are held to higher standards than those trained in the UK and whether the assessment process is realistic and not pedantic.

CONCLUSIONS

This scoping review demonstrates there is a lack of robust research evaluating the effectiveness of tests of competencies. The review indicates there is no due acknowledgement of the previous skills and knowledge of the overseas nurses. Future research should focus on exploring the feasibility of tests of competence and its role in the integration of the nursing workforce.

Entrustable professional activities-based objective structured clinical examinations in a pharmacy curriculum

BACKGROUND

The integration of Objective Structured Clinical Examinations (OSCEs) within the professional pharmacy program, contributes to assessing the readiness of pharmacy students for Advanced Pharmacy Practice Experiences (APPEs) and real-world practice.

METHODS

In a study conducted at an Accreditation Council for Pharmacy Education (ACPE)-accredited Doctor of Pharmacy professional degree program, 69 students in their second professional year (P2) were engaged in OSCEs. These comprised 3 stations: best possible medication history, patient education, and healthcare provider communication. These stations were aligned with Entrustable Professional Activities (EPAs) and Ability Statements (AS). The assessment aimed to evaluate pharmacy students' competencies in key areas such as ethical and legal behaviors, general communication skills, and interprofessional collaboration.

RESULTS

The formulation of the OSCE stations highlighted the importance of aligning the learning objectives of the different stations with EPAs and AS. The evaluation of students' ethical and legal behaviors, the interprofessional general communication, and collaboration showed average scores of 82.6%, 88.3%, 89.3%, respectively. Student performance on communication-related statements exceeded 80% in all 3 stations. A significant difference (p < 0.0001) was found between the scores of the observer and the SP evaluator in stations 1 and 2 while comparable results (p = 0.426) were shown between the observer and the HCP evaluator in station 3. Additionally, a discrepancy among the observers' assessments was detected across the 3 stations. The study shed light on challenges encountered during OSCEs implementation, including faculty involvement, resource constraints, and the necessity for consistent evaluation criteria.

CONCLUSIONS

This study highlights the importance of refining OSCEs to align with EPAs and AS, ensuring a reliable assessment of pharmacy students' clinical competencies and their preparedness for professional practice. It emphasizes the ongoing efforts needed to enhance the structure, content, and delivery of OSCEs in pharmacy education. The findings serve as a catalyst for addressing identified challenges and advancing the effectiveness of OSCEs in accurately evaluating students' clinical readiness.

Review of Objective Structured Clinical Examination Practices Within Pharmacy Programs Throughout the United States

OBJECTIVE

To update the description of current objective structured clinical examination (OSCE) practices within pharmacy schools in the United States and identify barriers to OSCE implementation and expansion.

METHODS

A survey was deployed to all accredited Doctor of Pharmacy programs within the United States. The survey was designed to collect information regarding the curricular mapping of OSCEs, OSCE design, OSCE delivery, assessment of OSCE performance, and barriers to OSCE implementation and expansion.

RESULTS

Of the 135 US-accredited programs identified, 109 (81%) programs completed the survey. In total, 93 (85%) programs reported using OSCEs to assess students; however, implementation throughout the curriculum and current practices varied by institution. Most programs place OSCEs within specific courses (96%), with the applied skills coursework being the most used (80%). The most common number of OSCEs that occur throughout a curriculum is 6 (18%), however, 20 (22%) programs execute 10 or more OSCEs throughout their curriculum. Forty (43%) programs use OSCEs as high-stakes assessments where poor performance could prevent a student from progressing to advanced pharmacy practice experiences. Of the responding programs, over half (56%) use teaching objective structured examinations to enhance learning. Common barriers to OSCE expansion exist and are related to resource utilization.

CONCLUSION

Significant expansion of OSCE development and implementation has occurred over the last decade. There is substantial variability in implementation and utilization among programs. Although standards of best practice for OSCEs exist for other health professions, best practices in pharmacy education have not been widely accepted or adopted.

The cognitive processes employed by undergraduate nursing OSCE assessors: A qualitative research study

OBJECTIVES

The Objective Structured Clinical Examination (OSCE) is an assessment format commonly used to assess undergraduate nursing students. However, in spite of its prominence, relatively little research has been conducted into how OSCE assessors form judgements about student performances, and whether divergent processes of judgement formation have the potential to negatively impact the inter-rater reliability (IRR) of awarded scores. This qualitative study aimed to uncover the cognitive processes which assessors employ when assessing OSCE performances.

DESIGN, SETTING, PARTICIPANTS

In order to investigate this, a convenience, purposive sample of 12 assessors watched four videos of students completing single-station OSCEs: two videos of blood pressure measurement, and two of naso-gastric tube insertion.

METHODS

Assessors were asked to "think aloud" while watching the videos, and also participated in a semi-structured interview about their assessment practices.

RESULTS

Thematic analysis of the qualitative data revealed three themes: observation, processing, and integration. Within each theme, a number of sub-themes were identified, which explain the cognitive mechanisms used by assessors when watching, judging and grading student performances.

CONCLUSIONS

Notably, the presence of these mechanisms was not uniform across the sample, indicating that assessors utilise different approaches when viewing and interpreting the same performances. This has the potential to threaten the IRR of awarded scores, and thus the validity of decisions made on the basis of those scores.

An analysis of the effectiveness of reflective learning through watching videos recorded with smart glasses-With multiple views (student, patient, and overall) in radiography education

The Objective Structured Clinical Examination (OSCE) is designed to assess medical students' skills and attitude competencies before clinical practice. However, no method of reflective learning using video-based content has been used in OSCE education. This study aimed to confirm whether using smart glasses-based educational content is effective for OSCE reflective learning using multiple views (patient, student, and overall). This educational intervention study included a control group exposed to the traditional learning method and an intervention group exposed to a learning method incorporating smart glasses. Participants were 117 (72 in the control group and 45 in the intervention group) third-year radiological technology students scheduled to take the OSCE and 70 (37 in the control group and 33 in the intervention group) who met the eligibility criteria. Mock OSCEs were administered before and after the educational intervention (traditional and smart glasses-based education) to investigate changes in scores. After the educational intervention, a self-reported comprehension survey and a questionnaire were administered on the effectiveness of the video-based content from different views for student reflective learning. Unexpectedly, the OSCE evaluation score after the preliminary investigation significantly increased for the smart glasses control group (0.36±0.1) compared to the intervention group (0.06±0.1) setting up the radiographic conditions (x-ray center and detector center; p = 0.042). The intervention group's lower score in the mock OSCEs may have been due to the discomfort of wearing the smart glasses to perform the radiography procedure and their unfamiliarity with the smart glasses, which may have affected their concentration. The findings suggest that smart glasses-based education for OSCEs can be improved (e.g., being easy to handle and use and trouble-free).

Evaluation of an OSCE's implementation and a two-step approach for a theoretical and practical training program in Obstetrics and Gynecology

Objective structured clinical examination (OSCE) is a well-known assessment method to evaluate clinical skills and competence in healthcare. Following the recently reformed National Competence-Based Catalog of Learning Objectives in Medicine, the implementation of this assessment method in the training program for medical students is now obligatory in Germany. This major change requires a reorganization not only of the training programs but also of the students themselves and the way they learn. We performed a poll evaluating the students' opinions regarding these major changes and the implementation of the OSCE with a new training program. To implement this assessment method and to evaluate the OSCE, Kern's six-step approach comprising (1) problem identification and general needs assessment, (2) needs assessment of the targeted learners, (3) goals and objectives, (4) educational strategies, (5) implementation, and (6) evaluation and feedback was applied. To evaluate and gather feedback, a poll was used to analyze the student's opinions regarding OSCE in gynecology and obstetrics and OSCE in general, in addition to the regular analysis of the students' results. To reform the educational strategy, a two-step approach was developed: First, the students completed the regular training program and a written examination, and second, they participated in a 1-week clerkship, in small group teaching, and in the OSCE. The OSCE stations were developed primarily based on the National Competence-Based Catalog and the German Catalog of Learning Objectives in Medicine, as well as on the feedback of experts reflecting their expectations for physicians beginning their careers. The students performed well in the OSCE and gave positive feedback regarding this examination method. Furthermore, they welcomed the upcoming changes by considering OSCE a valuable assessment tool, and they showed appreciation for the two-step approach by supporting the combination of an OSCE and a written examination. Thus, this article presents the implementation of an OSCE and a strategy for the adaptation of the curriculum to fulfill the new OSCE requirements and-to our knowledge-reveals students' primary opinions regarding the changes in their medical training program for the first time.

Applying a video recording, video-based rating method in OSCEs

INTRODUCTION

Objective structured clinical examination (OSCE) results could be affected by low homogeneity of examiners, non-retrospectiveness of test results, and examiner-cohort effect. In China, many students participate in medical qualification examinations, and this issue is particularly significant. This study aimed to develop a video recording, video-based rating method and compare the reliability of video and on-site ratings to enhance the quality assurance of OSCEs.

METHODS

The subjects of this study were clinical students one year after graduation participating in the clinical skills portion of the National Medical Licensing Examination. The participants were from four cities in Jiangsu province. Participants were randomly allocated to on-site and video rating groups to evaluate the rating methods consistency. We verified the reliability of recording equipment and evaluability of video recording. Moreover, we compared the consistency and equivalence of the two rating methods and analyzed the impact of video recording on scores.

RESULTS

The reliability of recording equipment and evaluability of video recording were high. Evaluation consistency between experts and examiners was acceptable, and there was no difference in evaluation results (P = 0.61). There was good consistency between video and on-site rating; however, a difference between the two rating methods was detected. The scores of video-based rating group students were lower than those of all students (P < 0.00).

CONCLUSIONS

Video-based rating could be reliable and offer advantages over on-site rating. The video recording, video-based rating method could provide greater content validity based on its traceability and the ability to view details. Video recording, video-based rating offers a promising mthod for improving the effectiveness and fairness of OSCEs.

A comparison of the American Board of Anesthesiology's in-person and virtual objective structured clinical examinations

BACKGROUND

The American Board of Anesthesiology's Objective Structured Clinical Examination (OSCE), as a component of its initial certification process, had been administered in-person in a dedicated assessment center since its launch in 2018 until March 2020. Due to the COVID-19 pandemic, a virtual format of the exam was piloted in December 2020 and was administered in 2021. This study aimed to compare candidate performance, examiner grading severity, and scenario difficulty between these two formats of the OSCE.

METHODS

The Many-Facet Rasch Model was utilized to estimate candidate performance, examiner grading severity, and scenario difficulty for the in-person and virtual OSCEs separately. The virtual OSCE was equated to the in-person OSCE by common examiners and common scenarios. Independent-samples t-test was used to compare candidate performance, and partially overlapping samples t-tests were applied to compare examiner grading severity and scenario difficulty between the in-person and virtual OSCEs.

RESULTS

The in-person (n = 3235) and virtual (n = 2934) first-time candidates were comparable in age, sex, race/ethnicity, and whether U.S. medical school graduates. The virtual scenarios (n = 35, mean [0.21] ± SD [0.38] in logits) were more difficult than the in-person scenarios (n = 93, 0.00 ± 0.69, Welch's partially overlapping samples t-test, p = 0.01); there were no statistically significant differences in examiner severity (n = 390, -0.01 ± 0.82 vs. n = 304, -0.02 ± 0.93, Welch's partially overlapping samples t-test, p = 0.81) or candidate performance (2.19 ± 0.93 vs. 2.18 ± 0.92, Welch's independent samples t-test, p = 0.83) between the in-person and virtual OSCEs.

CONCLUSIONS

Our retrospective analyses of first-time OSCEs found comparable candidate performance and examiner grading severity between the in-person and virtual formats, despite the virtual scenarios being more difficult than the in-person scenarios. These results provided assurance that the virtual OSCE functioned reasonably well in a high-stakes setting.

Designing and implementing an OMMID midwifery professional competence mixed test: A multimethod, multiphasic study

The assessment of professional competence is essential to ensure the achievement of competence standards. The goal of this study is to design and implement a professional competence test model to make the test unified, comprehensive, and fair. This research was a multimethod, multiphasic study. The first qualitative phase of the nominal group technique was conducted to design the test model with specific guidelines. The second phase was a mixed-method parallel field trial conducted on 161 senior midwifery bachelor students in universities of Iran. The test was conducted following the traditional method in the control group and OMMID model in the intervention group. This model consists of three parts: the MCQs (multiple choice questions), OSCE (objective structured clinical examination), and clinically-oriented tests, which included Mini-CEX (mini-clinical evaluation exercise) and DOPS (direct observation of procedural skills). Data were collected using questionnaires and interviews, and the results were assessed qualitatively and quantitatively. The OMMID model and the associated guideline were designed. Qualitative data analysis resulted in six main themes, including organizing, structure, tension due to change, fairness, unification, and outcome. The OMMID model did not increase students' anxiety and stress and did not decrease their satisfaction. The merits of this model include having a centralized guideline, using multiple evaluation methods, comprehensive evaluation of necessary skills, promotion of fairness, and increased student satisfaction.

Measuring the Quality of the OSCE in a Chiropractic Programme: A Review of Metrics and Recommendations

OBJECTIVES

The Objective Structured Clinical Examination (OSCE) is a commonly used assessment of clinical skill, and ensuring the quality and reliability in OSCEs is a complex, and on-going process. This paper discusses scoring schemas and reviews checklists and global rating scales (GRS) for marking. Also detailed are post-examination quality assurance metrics tailored to smaller cohorts, with an illustrative dataset.

METHODS

A deidentified OSCE dataset, from stations with both a checklist and GRS, of 24 examinees from a 2021 cohort was assessed using the following metrics: Cut-scores or pass-rates, number-of-failures, R2, inter-grade discrimination, and between-group-variation. The results were used to inform a set of implementable recommendations to improve future OSCEs.

RESULTS

For most stations the calculated cut-score calculated was higher than the traditional pass of 50% (58.9.8-68.4%). Number-of-failures were low for traditional pass rates and cuts-scores (0.00-16.7%), excepting Lab Analysis where number-of-failures was 50.0%. R2 values ranged from 0.67-0.97, but proportion of total variance was high (67.3-95.9). These data suggest there were potential missed teaching concepts, that station marking was open to examiner interpretation, and there were inconsistencies in examiner marking.Recommendations included increasing checklist detail and using a weighted marking scale, separating some stations into dichotomous and key-feature checklists, using GRSs specific to each station, and reviewing all future OSCEs with the metrics described to guide refinements.

CONCLUSIONS

The analysis used revealed several potential issues with the OSCE assessment. These findings informed recommendations to improve the quality of future examinations.

Comparability of Objective Structured Clinical Examinations (OSCEs) and Written Tests for Assessing Medical School Students' Competencies: A Scoping Review

Objective Structured Clinical Examinations (OSCEs) and written tests are commonly used to assess health professional students, but it remains unclear whether the additional human resources and expenses required for OSCEs, both in-person and online, are worthwhile for assessing competencies. This scoping review summarized literature identified by searching MEDLINE and EMBASE comparing 1) OSCEs and written tests and 2) in-person and online OSCEs, for assessing health professional trainees' competencies. For Q1, 21 studies satisfied inclusion criteria. The most examined health profession was medical trainees (19, 90.5%), the comparison was most frequently OSCEs versus multiple-choice questions (MCQs) (18, 85.7%), and 18 (87.5%) examined the same competency domain. Most (77.5%) total score correlation coefficients between testing methods were weak (r < 0.40). For Q2, 13 articles were included. In-person and online OSCEs were most used for medical trainees (9, 69.2%), checklists were the most prevalent evaluation scheme (7, 63.6%), and 14/17 overall score comparisons were not statistically significantly different. Generally low correlations exist between MCQ and OSCE scores, providing insufficient evidence as to whether OSCEs provide sufficient value to be worth their additional cost. Online OSCEs may be a viable alternative to in-person OSCEs for certain competencies where technical challenges can be met.

Effectiveness of a virtual program for OSCE preparation during COVID-19: a descriptive and repeated cross-sectional study among nursing students

BACKGROUND

Despite the prevalence of distance learning during COVID-19, conducting clinical training for nursing students remains challenging. In compliance with social-distancing restrictions, a Zoom-based virtual OSCE preparation program for nursing students was designed, and it included clinical skills. The aims of this study were to assess nursing students' satisfaction with a virtual program for Objective Structured Clinical Examination (OSCE) preparation, and to evaluate its learning outcomes measured by OSCE scores as compared to those of in-person preparation programs.

METHODS

A descriptive and repeated cross-sectional study was designed. Students' satisfaction with the virtual program was based on a post-course survey and personal reflections. OSCE scores of graduates of the virtual program (n = 82) tested in 2021 were compared to those of 337 graduates of in-person programs tested in 2017-2020.

RESULTS

A post-program survey revealed that 88% of the students in 2021 were satisfied with the virtual program and felt it prepared them properly for the OSCE (26% agree and 62% strongly agree). No significant differences were found between OSCE scores following the virtual program conducted in 2021 and scores following in-person programs conducted in 2017-2020.

CONCLUSIONS

This study suggests that nursing education can benefit from integrating virtual programs which incorporate clinical practices into the curricula, without harming student competency. The study results may address the problem of maintaining clinical practices in a time of limited accessibility, and in settings of low resources. It is important to expand the investigation to long-term impact of virtual training programs on nursing students' competencies.

The effect of objective structured clinical examinations for nursing students

Twenty core nursing skills have been identified by the Korean Accreditation Board of Nursing Education. Proficiency in these skills is essential for all nursing professions, and many educational strategies exist to develop these skills in nursing students, including the Objective Structured Clinical Examination (OSCE). To date, no study on the effects of the OSCE on nursing education has been published. Therefore, we evaluated the effects of the OSCE on the core nursing skills of 207 pre-licensure nursing students in Korea. We measured the nursing students' confidence, skills, and knowledge acquisition and retention. A one-way analysis of variance and Fisher's least significant difference were used for data analysis. Among the four nursing areas (fall, transfusion, pre-operative, and post-operative), students demonstrated the highest confidence level scores in pre-operative nursing. On the OSCE, students scored the highest in transfusion nursing. Significant differences were found between prior knowledge, knowledge acquisition, and retention. Our findings confirm that the OSCE, after lectures and core nursing skill practice, improved the retention of nursing students' knowledge. Therefore, this program can positively influence nursing students' knowledge level, and implementing the OSCE can strengthen students' clinical competency.

Recommendations After an Objective Structured Clinical Examination Program Evaluation

BACKGROUND

The use of objective structured clinical examinations (OSCEs) is a well-established method of clinical assessment in the medical profession, and these are currently being integrated into advanced practice nursing curricula.

METHOD

A program assessment was conducted at a current family nurse practitioner track within a southwestern U.S. university using the OSCE evaluative strategy. The logic model from the Centers for Disease Control and Prevention framework was incorporated. Both focus group and the de-identified faculty-scored student rubrics during the fall, spring, and summer semesters of one cohort of students (N = 73) were examined.

RESULTS

Strengths and challenges were found in the areas of student preparation, scoring/grading with variations of inter-rater reliability, and debriefing.

CONCLUSION

Establishing a specific OSCE center with a dedicated director could support adequate attainment of formative and summative competencies for nurse practitioners and provide faculty education and standardized student preparation, while adhering to the multiple guidelines and standards related to nurse practitioner competency evaluation with simulation. [J Contin Educ Nurs. 2023;54(4):169-175.].

Using cultural historical activity theory to reflect on the sociocultural complexities in OSCE examiners' judgements

Examiners' judgements play a critical role in competency-based assessments such as objective structured clinical examinations (OSCEs). The standardised nature of OSCEs and their alignment with regulatory accountability assure their wide use as high-stakes assessment in medical education. Research into examiner behaviours has predominantly explored the desirable psychometric characteristics of OSCEs, or investigated examiners' judgements from a cognitive rather than a sociocultural perspective. This study applies cultural historical activity theory (CHAT) to address this gap in exploring examiners' judgements in a high-stakes OSCE. Based on the idea that OSCE examiners' judgements are socially constructed and mediated by their clinical roles, the objective was to explore the sociocultural factors that influenced examiners' judgements of student competence and use the findings to inform examiner training to enhance assessment practice. Seventeen semi-structured interviews were conducted with examiners who assessed medical student competence in progressing to the next stage of training in a large-scale OSCE at one Australian university. The initial thematic analysis provided a basis for applying CHAT iteratively to explore the sociocultural factors and, specifically, the contradictions created by interactions between different elements such as examiners and rules, thus highlighting the factors influencing examiners' judgements. The findings indicated four key factors that influenced examiners' judgements: examiners' contrasting beliefs about the purpose of the OSCE; their varying perceptions of the marking criteria; divergent expectations of student competence; and idiosyncratic judgement practices. These factors were interrelated with the activity systems of the medical school's assessment practices and the examiners' clinical work contexts. Contradictions were identified through the guiding principles of multi-voicedness and historicity. The exploration of the sociocultural factors that may influence the consistency of examiners' judgements was facilitated by applying CHAT as an analytical framework. Reflecting upon these factors at organisational and system levels generated insights for creating fit-for-purpose examiner training to enhance assessment practice.

Using objective structured clinical examination (OSCE) for circulating and scrub skills of first-semester operating room technology students: Is it valid and reliable?

Objectives

This study aimed to determine the validity and reliability of an Objective Structured Clinical Examination (OSCE) developed for the summative assessment of the practical credit of the course titled "Principles and Techniques of Circulating and Scrub Persons' Practice."

Methods

This applied descriptive study was conducted on 63 first-semester undergraduate Operating Room (OR) technology students. Totally, 17 circulating and scrub skills were incorporated in four 7-min OSCE stations. The content validity of the examination was assessed by an expert panel (n = 14). Additionally, its concurrent validity was evaluated by Intra-Class Correlation (ICC) through measuring the correlation between the total score of OSCE and grade point average, grades of the practical and theoretical credits of the course that were obtained by multiple-choice tests, and the total grade of the course (achieved by the grades of the practical and theoretical credits tests). The Inter-Rater Reliability (IRR) was also calculated by the correlation between the scores recorded by two examiners at each OSCE station using ICC. Meanwhile, the internal consistency across stations (item-total correlation) and across items within a station was determined by Cronbach's alpha (α).

Results

The OSCE was deemed to fulfill the content validity criteria. The total score of OSCE showed a significant correlation with the concurrent validity criteria (ICC = 0.66-0.85, p < 0.001). Besides, the IRR was significant for the total score of OSCE as well as for the score of each station (ICC = 0.90-0.98, p < 0.001). Likewise, a good level of internal consistency was found across stations (α = 0.63-0.93) and across items within a station (α = 0.78-0.92).

Conclusion

The developed OSCE had appropriate validity and reliability. Thus, it can be used to assess the circulating and scrub skills of first-semester OR technology students.

[Ecos among nursing students]

Objective, structured clinical examinations provide an adaptive, harmonized and egalitarian device for standardized assessment of health care students. This method is structured around several thematic stations, with a rhythmic and timed passage. All future professionals in the field, including nursing students, could benefit from this method.

Dental hygiene educators' knowledge and implementation of objective structured clinical examination in United States dental hygiene programs

PURPOSE

In 2021, the American Dental Association (ADA) announced the intention to develop a standardized dental hygiene licensure objective structured clinical examination (DHLOSCE). The purpose of this study was to measure the United States (US) dental hygiene (DH) educators' foundational knowledge of OSCE development and delivery in light of the impending development of the DHLOSCE by the ADA's Testing Services.

METHODS

The study was determined to be exempt from Institutional Review Board oversight. A 21-question survey was developed, pilot tested, and electronically disseminated through Qualtrics. The survey recruitment was emailed to the directors of all entry-level DH education programs in the US (n = 328), asking them to participate in the survey and to forward it to the clinical faculty in their institutions. Descriptive and inferential statistics were utilized to analyze the data.

RESULTS

There were 143 study participants, for a completion rate of 45%. Over two-thirds of respondents (64%) were unaware of the plans to develop the DHLOSCE, while 13% reported utilizing OSCEs to meet accreditation standards. Only 3% reported receiving a formal education in OSCE development compared to 29% who learned through a colleague or peer. Nearly half reported a lack of faculty experience as a barrier to OSCE implementation. Over three-quarters, 76% reported a lack of OCSE development committees within their program and only 14% had experience developing an OSCE station.

CONCLUSION

The study results suggest an urgent need for the development of OSCE training resources specific to DH education, as programs across the US prepare for the impending DHLOSCE.

Impact of high-fidelity simulation exposure of nursing students with their objective structured clinical examination: A quasi-experimental study

AIM

The study aimed to establish the impact of high-fidelity simulation (HFS) in the objective structured clinical examination (OSCE) of nursing students enrolled in four undergraduate courses (medical-surgical, critical-care, maternal-health and paediatric nursing).

DESIGN

This quasi-experimental research study was performed during the midterm and final OSCEs of nursing students at the institution, and their OSCE performance was assessed.

METHODS

The students were divided into two: those who were exposed to HFS in addition to their clinical training and the other group who underwent clinical training without HFS exposure.

RESULTS

The combined mean midterm and final OSCE results of the group of nursing students with HFS exposure and those without HFS exposure were 92.58 and 82.66, respectively, with a mean between-group difference of 9.92% (p < .01). Our findings reveal that the HFS exposure in addition to clinical training enhanced the students' OSCE performance.

A novel simulation-based approach to training for recruitment of older adults to clinical trials

BACKGROUND

The need to engage adults, age 65 and older, in clinical trials of conditions typical in older populations, (e.g. hypertension, diabetes mellitus, Alzheimer's disease and related dementia) is exponentially increasing. Older adults have been markedly underrepresented in clinical trials, often exacerbated by exclusionary study criteria as well as functional dependencies that preclude participation. Such dependencies may further exacerbate communication challenges. Consequently, the evidence of what works in subject recruitment is less generalizable to older populations, even more so for those from racial and ethnic minority and low-income communities.

METHODS

To support capacity of research staff, we developed a virtual, three station simulation (Group Objective Structured Clinical Experience-GOSCE) to teach research staff communication skills. This 2-h course included a discussion of challenges in recruiting older adults; skills practice with Standardized Participants (SPs) and faculty observer who provided immediate feedback; and debrief to highlight best practices. Each learner had opportunities for active learning and observational learning. Learners completed a retrospective pre-post survey about the experience. SP completed an 11-item communication checklist evaluating the learner on a series of established behaviorally anchored communication skills (29).

RESULTS

In the research staff survey, 92% reported the overall activity taught them something new; 98% reported it provided valuable feedback; 100% said they would like to participate again. In the SP evaluation there was significant variation: the percent well-done of items by case ranged from 25-85%.

CONCLUSIONS

Results from this pilot suggest that GOSCEs are a (1) acceptable; (2) low cost; and (3) differentiating mechanism for training and assessing research staff in communication skills and structural competency necessary for participant research recruitment.

Midwifery students' experiences of objective structured clinical examinations: A qualitative evidence synthesis

OBJECTIVES

To identify and synthesise the evidence of midwifery students' views and experiences of Objective Structured Clinical Examinations (OSCEs).

DESIGN

A qualitative evidence synthesis was undertaken using Thomas and Harden's synthesis methods and reported according to the ENTEREQ guidelines.

DATA SOURCES

PubMED, CINAHL Plus, Embase, ERIC, PsycINFO, Social Science Citation Index (via Web of Science) and Scopus were searched from inception to 19th February 2020 and an updated search was performed on 12th May 2021 to identify qualitative and mixed method studies, reporting qualitative data of undergraduate midwifery students' views and experiences of OSCEs.

REVIEW METHODS

Studies were independently screened by title and abstract according to a predetermined inclusion and exclusion criteria. Full text review was then conducted independently by two authors. Studies providing qualitative data on undergraduate midwifery students' views and experiences of OSCEs during their midwifery registration education programmes were included. Quality appraisal was performed using the Critical Appraisal Skills Programme (CASP) tool. Data synthesis was performed using Thomas and Harden's thematic synthesis method.

RESULTS

Eleven studies were identified and included. Thematic synthesis identified 13 descriptive themes which translated into four analytical themes that represented midwifery students' views and experiences of OSCEs: the preparation phase, the assessment phase, the feedback phase and the translation phase.

CONCLUSIONS

This synthesis offers insight into undergraduate midwifery students' views and experiences of OSCEs and how they are experienced across four phases. This is determined to be a useful structure to guide midwifery educators and researchers in their consideration of the student experience of OSCEs in the future. Midwifery students in this synthesis reported high levels of nervousness, anxiety, and stress during the OSCE process. The findings reveal that there are a number of controllable factors impacting on midwifery students' feelings of safety in the assessment environment, and that these should be addressed to maximise the advantages that the OSCE has to offer in terms of learning and preparation for clinical practice.

Reported outcomes and characteristics of objective structured clinical examinations in advanced practice nursing education: a scoping review protocol

OBJECTIVE

The objective of this scoping review is to identify the outcomes reported from objective structured clinical examinations in advanced practice registered nurse education and to describe their characteristics.

INTRODUCTION

Graduate medical education has used objective structured clinical examinations to assess student health professionals since 1975. Nursing programs have adopted objective structured clinical examinations to assess evidence-based practice and clinical competency of advance practice nursing students, but little is known about the outcomes that are being reported.

INCLUSION CRITERIA

This review will consider evidence on the outcomes and other characteristics of objective structured clinical examinations for students enrolled in graduate-level advanced practice nurse education programs (ie, nurse practitioners, clinical nurse specialists, nurse midwives, and nurse anesthetists). All types of evidence will be considered, including quantitative and qualitative studies, systematic reviews, as well as text and opinion papers.

METHODS

This review will be conducted in accordance with the JBI methodology for scoping reviews. A literature search will be conducted in PubMed, CINAHL, MEDLINE, Nursing and Allied Health, Scopus, and Google Scholar. Sources that are published in English or that could be translated into English will be considered for inclusion. No date limit will be applied. Study selection and data extraction will be conducted by three independent reviewers. Any disagreements will be resolved through discussion. The scoping review findings will be presented in a narrative style and reported using diagrams and extraction tables.

A video-based situational judgement test of medical students' communication competence in patient encounters: Development and first evaluation

OBJECTIVE

We developed and evaluated the Video-Based Assessment of Medical Communication Competence (VA-MeCo), a construct-driven situational judgement test measuring medical students' communication competence in patient encounters.

METHODS

In the construction phase, we conducted two expert studies (n_panel1 = 6, n_panel2 = 13) to ensure curricular and content validity and sufficient expert agreement on the answer key. In the evaluation phase, we conducted a cognitive pre-test (n = 12) and a pilot study (n = 117) with medical students to evaluate test usability and acceptance, item statistics and test reliability depending on the applied scoring method (raw consensus vs. pairwise comparison scoring).

RESULTS

The results of the expert interviews indicated good curricular and content validity. Expert agreement on the answer key was high (ICCs> .86). The pilot study showed favourable usability and acceptance by students. Irrespective of the scoring method, reliability for the complete test (Cronbach's α >.93) and its subscales (α >.83) was high.

CONCLUSION

There is promising evidence that medical communication competence can be validly and reliably measured using a construct-driven and video-based situational judgement test.

PRACTICE IMPLICATIONS

Video-based SJTs allow efficient online assessment of medical communication competence and are well accepted by students and educators.

Implementierung des Objective Structured Clinical Examination (OSCE) im Masterstudiengang Pflegewissenschaft – Pilotprojekt zur Prüfung der ethischen Kompetenz in Pflegepraxis und -forschung

Die Vermittlung ethischer Kompetenzen ist wesentlicher Bestandteil in berufsqualifizierenden und postgraduellen Studiengängen. Dabei werden praktisch-ethische Problemlösungskompetenzen, je nach Studiengang aber auch die forschungsethische Betrachtung von Studien thematisiert. Die Überprüfung dieser ethischen Kompetenzen stellt sich als Herausforderung dar. Das schriftliche oder mündliche Abfragen von Lehrinhalten greift zu kurz, da somit lediglich Wissen, jedoch nicht Fertigkeiten oder gar Haltung erfasst werden können.

Bei der Reakkreditierung des Masterstudiengangs Pflegewissenschaft an der Universität Witten/Herdecke wurde dieser Herausforderung mit der Umsetzung eines für die postgraduelle Ausbildung innovativen Prüfungsformats begegnet. Für das Modul „Ethik in der Pflegepraxis und -forschung“ wurde ein Format ausgearbeitet, welches sich an den „Objective Structured Clinical Examinations“ orientiert und somit die erworbenen Fertigkeiten und Haltungen der Studierenden mit einbezieht. Der Beitrag zeigt auf, wie das OSCE-Prüfungsformat (kurz: OSCE) konzipiert und eine Adaption erfolgt ist.

Assessment of an intensive education program for pharmacists on treatment of tobacco use disorder using an objective structured clinical examination: a randomized controlled trial

BACKGROUND

Tobacco use is one of the major public health threats globally. Community pharmacists are uniquely positioned to offer tobacco cessation services owing to their easy accessibility by the public. To prepare Qatar community pharmacists to develop the competencies and skills required to offer smoking cessation services, an intensive tobacco control education program was designed and implemented. The study aimed to assess the impact of the tobacco education program on the pharmacists' skills and competence.

METHODS

A random sample of community pharmacists in Qatar was chosen for participation in the program. Consenting participants were randomly assigned to either intervention or control groups. The intervention group received an intensive education program on treatment of tobacco-use disorder, while a short didactic session on a non-tobacco-related topic was delivered to the control group. The pharmacists' tobacco cessation skills and competencies were assessed using an Objective Structured Clinical Examination (OSCE).

RESULTS

A total of 54 and 32 community pharmacists in the intervention group and the control group, respectively, completed the OSCE. The intensive tobacco education group achieved significantly higher total scores than the control group in all the OSCE cases. Specifically, the mean total scores for the intervention group were 15.2, 15.3, 14.2, 14.6, 16.3, and 15.2 compared to 8.8, 6.2, 7.7, 9.2, 8.3, and 11.3 for the control group (p < 0.001) for cases one to six respectively.

CONCLUSION

The study demonstrated that an intensive tobacco cessation education program can improve pharmacists' tobacco cessation skills and increase their tobacco cessation counseling abilities.

TRIAL REGISTRATION

Clinical Trials NCT03518476 ( https://clinicaltrials.gov/ct2/show/NCT03518476 ) Registration date: May 8, 2018.

Interrater reliability in the assessment of physiotherapy students

BACKGROUND

Reliable and objective assessment of psychomotor skills in physiotherapy students' education is essential for direct feedback and skill improvement. The aim of this study is to determine the interrater reliability in the assessment process of physiotherapy students and to analyse the assessment behaviour of the examiners.

METHODS

Physiotherapy teachers from two different schools assessed students from two different schools performing proprioceptive neuromuscular facilitation (PNF) patterns. An evaluation sheet with a 6-point rating scale and 20 evaluation criteria including an overall rating was used for assessment. The interrater reliability was determined calculating an intraclass-correlation coefficient (ICC) and Krippendorff's alpha. The assessment behaviour of the examiners was further analysed calculating the location parameters and showing the item response distribution over item in form of a Likert plot.

RESULTS

The ICC estimates were mostly below 0.4, indicating poor interrater reliability. This was confirmed by Krippendorff's alpha. The examiners showed a certain central tendency and intergroup bias.

DISCUSSION AND CONCLUSION

The interrater reliability in this assessment format was rather low. No difference between the two physiotherapy schools concerning the interrater reliability could be identified. Despite certain limitations of this study, there is a definite need for improvement of the assessment process in physiotherapy education to provide the students with reliable and objective feedback and ensure a certain level of professional competence in the students.

TRIAL REGISTRATION

The study was approved by the ethics committee of the Medical Faculty RWTH Aachen University (EK 340/16).

Quality management of objective structured clinical examinations: An integrative literature review

BACKGROUND

Objective Structured Clinical Examinations are widely adopted as a clinical assessment method in nurse education. Quality is an integral aspect of the design and implementation of OSCEs, facilitating their credibility and rigour. However, there is a dearth of literature regarding the management of the quality of Objective Structured Clinical Examinations.

OBJECTIVE

To summarise existing literature regarding best practices on the management of the quality of Objective Structured Clinical Examinations in health science education.

DESIGN

An integrative literature review.

DATA SOURCES

EBSCOhost, including CINAHL, eBook Collection, E-journals, ERIC, Health Source-Consumer Edition, Health Source-Nursing/Academic Edition, Humanities International Complete and MEDLINE, together with Cochrane Online, PubMed, Taylor & Francis Online and ScienceDirect, were searched, followed by hand searching of references as well as a manual search for grey literature, using Google.

REVIEW METHODS

Using predetermined inclusion and exclusion criteria, research documents (experimental, non-experimental, descriptive and qualitative studies), non-research documents (editorials, opinion letters), including grey literature, published between January 2010 and March 2021.

RESULTS

A total of thirteen (n = 13) studies were included in the review. A total of 22 quality measures were identified, which should be applied in the preparation and planning, implementation and evaluation phases of Objective Structured Clinical Examinations. The preparation and planning phase is crucial for the quality of Objective Structured Clinical Examinations, as more than half (n = 12) of the 22 quality measures were identified in this phase.

CONCLUSIONS

There is limited available recorded evidence that defines and describes the management of the quality of Objective Structured Clinical Examinations. Future research should focus on the context-specific use of psychometric analysis to ensure generation of valid, objective and consistent assessment during Objective Structured Clinical Examinations. More rigorous large-scale studies (such as quantitative or randomised controlled trials) regarding management of variances amongst examiner scores, impact of moderation on Objective Structured Clinical Examinations, and virtual Objective Structured Clinical Examinations should be conducted.

Objective structured clinical examination vs traditional clinical examination to evaluate students' clinical competence: A systematic review of nursing faculty and students' perceptions and experiences

BACKGROUND

Assessment of clinical competence of nursing students is an essential requirement in professional nursing education. This article summarizes the current published evidence indicating the nursing faculty and students' perceptions and experiences on benefits of OSCE over TCE.

METHODS

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations. Electronic databases including Scopus, Medline, Science Direct, CINAHL, EBSCO, PsychINFO, and Pubmed Central were used to identify relevant articles. The studies published between January 1, 2010 and December 31, 2020 and fourteen full-text articles that met all the inclusion criteria were included in the review.

RESULTS

The review identified five themes namely: a) Student's perception of OSCE; b) student's satisfaction regarding OSCE as an examination approach; c) students' perception of TCE as an examination approach; d) student's level of stress and anxiety towards OSCE vs TCE; and e) faculty member's perception and experience of OSCE.

CONCLUSION

We conclude that OSCE is a more credible assessment format to evaluate the clinical competence of undergraduate nursing students compared to the TCE method.

A competency roadmap for advanced practice nursing education using PRIME-NP

BACKGROUND

Clinical competency validation is essential for nurse practitioner (NP) education and public accountability. While there has been robust discussion around what constitutes clinical competency and assessment, clear and consistent definitions and measurements remain elusive.

PURPOSE

This article describes the PRIME-NP clinical competency model that is scalable, reproducible and accurately documents NP student competency across clinical courses.

METHODS

To develop the model, work in 5 discrete domains was necessary: (a) model development, (b) assessment tool to be used in Objective Structured Clinical Exams (OSCE), (c) rubrics to accompany the OSCE exam, (d) faculty education, and (e) evaluating the model use.

FINDINGS

Faculty and student outcomes reveal that the model and assessment tool acceptability and effectiveness of the model, especially for early identification for at risk students.

CONCLUSION

The PRIME-NP offered faculty the opportunity to identify at-risk students, identify a more nuanced remediation plan, and assess student competency in simulated environments.

Effectiveness of applying clinical simulation scenarios and integrating information technology in medical-surgical nursing and critical nursing courses

BACKGROUND

To determine the impact of combining clinical simulation scenario training and Information Technology Integrated Instruction (ITII) on the teaching of nursing skills.

METHODS

120 4th-year students in a nursing program who were enrolled in medical and surgical nursing courses. 61 received innovative instruction (experimental group) and 59 received conventional instruction (control group). The ADDIE model, systematic method of course development that includes analysis, design, development, implementation, and evaluation,was used to build simulation teaching and clinical scenarios and to create and modify objective structure clinical examination (OSCE) scenario checklists for acute myocardial infarction (AMI) care, basic life support and operation of automated external defibrillator (BLS), and subdural hemorrhage (SDH) care. The modified OSCE checklists were assessed for reliability, consistency, and validity. The innovative training included flipped classrooms, clinical simulation scenarios, ITII and blended learning formats.

RESULTS

The reliability and validity of the OSCE checklists developed in this study were acceptable and comparable or higher than checklists in past studies and could be utilized as an OSCE performance tool. Students in innovative instruction obtained significantly better OSCE performance, lab scores and improvements from the previous year's grades. Significant differences were found in situational awareness (SA). No strong correlations were found between OSCE scores and clinical internship scores, and no significant differences were found between the groups in overall clinical internship performance.

CONCLUSIONS

Innovative instruction showed better performance than conventional methods in summative evaluation of knowledge components, OSCE formative evaluation and clinical nursing internship scores, as well as improved situational awareness in nursing students.

Challenges of objective structured clinical examination in undergraduate nursing curriculum: Experiences of faculties and students

BACKGROUND

Assessment of clinical competence of nursing students is necessary to ensure safe practice and the realization of professional clinical standards. An Objective Structured Clinical Examination (OSCE) is commonly used internationally in nursing education programs to assess clinical competence, but is a new process in Iranian nursing curricula.

OBJECTIVE

The aim of this study was to explore and describe challenges associated with OSCE implementation based on the experiences of faculty members and nursing students, with the objective of further improving the assessment of clinical competence in nursing education.

DESIGN

This study used a qualitative approach that included thematic analysis of the transcribed interviews.

SETTING

Faculty of Nursing and Midwifery, Tabriz University of Medical Sciences (TBZMED), Tabriz, Iran.

PARTICIPANTS

Purposive sampling was used to select eighteen faculty members and fifteen students from those interested. These participants represent a reasonable cross-section of those who had participated in the OSCE.

METHODS

A pre-developed interview guide was used to inform semi-structured in-depth interviews. These were recorded and analyzed using the 6-phase thematic analysis method as described by Braun and Clarke (2013). Themes were independently verified.

RESULTS

Challenges identified by participants around OSCEs as part of undergraduate nursing education were extracted from the interview data. Two main themes were evident from the data: 1) shortcomings in executive and technical infrastructure, 2) shortcomings in educational infrastructure. These themes reflected both student and faculty experiences. These also aligned with themes that commonly emerge in related literature.

CONCLUSION

The study findings illustrate several key challenges associated with organizing and implementing OSCEs and so provides unique insights into the development of strategies to implementing and promoting OSCEs in nursing education. We recommend that managers and authorities in nursing education focus on these challenges and explore processes to successfully introduce this exam for assessment of nursing student capacity.

Materials matter: Understanding the importance of sociomaterial assemblages for OSCE candidate performance

INTRODUCTION

The OSCE is a sociomaterial assemblage-a meshing together of human and material components producing multiple effects. Materials matter because they shape candidate performance, with potentially calamitous career consequences if materials influence performance unjustly. Although the OSCE literature refers to materials, few papers study the sociomateriality of OSCEs. Therefore, we explored OSCE stakeholders' talk about sociomaterial assemblages to better understand their importance for candidate performance.

METHODS

We conducted 15 focus groups with OSCE candidates (n = 42), examiners (n = 20) and simulated patients (n = 17) after an Australian postgraduate nursing OSCE. Sociomateriality informed our team-based framework analysis of data.

RESULTS

Participants identified a multiplicity of OSCE materials (objects, technologies and spaces) thought to matter for candidate performance. Candidates' unfamiliarity with materials and missing or malfunctioning materials were reported to yield numerous negative impacts (eg cognitive overload, negative affect, time-wasting), thereby adversely affecting candidate performance. Both examiners and candidates made micro-adjustments to sociomaterial assemblages during the OSCE in order to make it work (eg candidates saying what they would do rather than doing it). Sometimes, such tinkering extended so far that sociomaterial assemblages were ruptured (eg examiners ignoring rubrics to help pass candidates), potentially influencing OSCE standardisation.

DISCUSSION

Our novel empirical study extends previous conceptual work by illustrating wide-ranging sociomaterial assemblages influencing OSCE candidate performance. Further research is now needed employing sociomaterial approaches to further elucidate sociomaterial entanglements in diverse OSCEs. We encourage OSCE stakeholders to become more attuned to the productive nature of materials within all stages of OSCE design and implementation.

The objective structured clinical examination as an assessment strategy for clinical competence in novice nursing practitioners in Taiwan

Background

The conventional written tests and professional assessment have limitation in fair judgement of clinical competence. Because the examiners may not have total objectivity and may lack standardization throughout the assessment process. We sought to design a valid method of competence assessment in medical and nursing specialties. This work was aimed to develop an Objective Structured Clinical Exam (OSCE) to evaluate novice nursing practitioners’ clinical competency, work stress, professional confidence, and career satisfaction.

Methods

A Quasi-experimental study (pre-post). Fifty-five novice nursing practitioners received the OSCE three-months following their graduation, which consisted of four stations: history taking, physical examination, problem-directed management, interpersonal communication, and the required techniques of related procedures. The examiners had to complete an assessment checklist, and the participants had to complete a pre-post questionnaire (modified from a Nursing Competency Questionnaire, a Stress scale, and Satisfaction with Learning scale).

Results

Among the novice nursing practitioners, 41 of them (74.5 %) passed the exam with a mean score of 61.38 ± 8.34. There was a significantly higher passing rate among nurses who were working in medical-surgical wards (85.7 %) and the intensive care unit-emergency department (77.8 %) compared to novice nursing practitioners working in other units. All the novice nursing practitioners at Station A had poor performance in assessing patients with a fever. OSCE performance was more associated with educational attainment and work unit, rather than the gender. Finally, the participants showed statistically significant increases in their clinical competency, confidence in their professional competence, satisfaction with the clinical practice, and decreased work stress after the OSCE.

Conclusions

We found that the OSCE process had a positive educational effect, in providing a meaningful and accurate assessment of the competence of novice nursing practitioners. An appropriate OSCE program is vital for novice nursing practitioners, educators, and administrators. The effective application of OSCEs can help novice nursing practitioners gain confidence in their clinical skills.

The use and effect of OSCES in post-registration nurses: An integrative review

BACKGROUND

Objective structured clinical examinations (OSCEs) have been used in the undergraduate medical setting since the 1970s, however little is known about its use and effectiveness in post-registration nurse education.

OBJECTIVES

The purpose of this review was to critically analyse studies using OSCEs in post-registration nurse education and to explore the use and effectiveness of OSCEs in this cohort.

DESIGN

This review was conducted using the Whittemore and Knafl's framework for integrated reviews.

DATA SOURCES

Using the search terms OSCE OR OSCA OR objective structured clinical examination AND nursing NOT undergraduate, a comprehensive review was conducted using CINHAL and MEDLINE.

METHODS

The initial search located 173 studies. After screening and checking eligibility 19 studies were critically appraised. The final number of studies included in this review was 16.

RESULTS

Three themes were generated: (i) Application of OSCEs in post-registration level nursing (ii) OSCEs more than an exam: Learning enhancements and (iii) Participant perceived impact of OSCEs.

CONCLUSION

OSCEs for post-registration nurses have evolved from the original OSCE design and have been adapted to a wide range of clinical settings and specialities. OSCEs were recognized as a valuable learning and assessment tool across the world for post-registration nurses. OSCEs offered strengths in terms of learner satisfaction and increased self-efficacy for this cohort. There was limited evidence supporting the effects of OSCEs within post-registration nursing education in comparison with other educational tools. Further research is needed to explore whether the knowledge gained in OSCEs is translated into clinical practice and whether simulation-based education is more effective in achieving enhanced knowledge compared to traditional-based education. Future research is required using RCT methods to compare the impact of OSCE to traditional-based education.

Evaluation of an international program for nurse practitioner students

BACKGROUND

There are few opportunities for nurse practitioner (NP) students to develop relationships and exchange ideas with international counterparts. In addition, discovering differences in instructional pedagogy between programs is beneficial for every student.

PURPOSE

To facilitate an international understanding of health care services and compare curricula of advanced practice programs through an international experience for first year NP students from the United Kingdom and the United States.

METHODOLOGICAL ORIENTATION

The study used a qualitative analysis of responses from specified questions provided to both cohorts of students before and after the experience.

SAMPLE

A virtual global exchange between 23 advanced practice students and their professors in the spring of 2018 culminated in a trip by the UK students and faculty to the United States in April of that year.

CONCLUSIONS

The UK and US NP students positively evaluated the international encounter and pointed out specific educational activities that were most helpful to their training.

IMPLICATIONS FOR PRACTICE

International experiences offer NP students the opportunity to experience health care delivery in an alternative cultural milieu.

Design of a nursing objective structured clinical examination of a first-year clinical practice program

OBJECTIVE

The aim of the present study was to design a content-valid nursing objective structured clinical examination attending a first-year clinical nursing practice program.

METHOD

The examination was designed following a procedure based on the consensus of experts which was comprised of three phases: selection of the activities in which students should be competent according to the learning outcomes of the course, clinical case design, and integration of the clinical cases designed into the stations of the test.

RESULTS

Of the 44 surveys submitted for the design of the stations, 37 were answered, of which 31 respondents met the inclusion criteria of the panel of experts. The activities on which the experts reached the highest degrees of consensus were: basic physical assessment and monitoring of vital signs, assessment of hygiene and skin status, ability to develop care plans, management of safety principles in administration of medication and administration of oral medication. Based on the selected activities, the experts developed 20 clinical cases, from which a four-station nursing objective structured clinical examination was designed.

CONCLUSION

The structured methodology based on the design of experts enabled the design of a content-valid objective structured clinical examination appropriate for the evaluation of the learning outcomes achieved by the students attending a clinical practice program.

Introduction of the Objective Structured Clinical Examination in Speech and Language Therapy Education: Student Perspectives

BACKGROUND/AIMS

The Objective Structured Clinical Examination (OSCE) has an established history of assessing clinical competence for medical and allied healthcare professionals. No research has investigated the use of the OSCE within speech and language therapy undergraduate education. This study aimed to evaluate undergraduate speech and language therapy students' perceptions of the OSCE and to determine if perceptions differed depending on stage of undergraduate education.

METHODS

An online survey was distributed to second- and final-year students in a 4-year undergraduate speech and language therapy university programme after completing an OSCE. Quantitative survey data were analysed descriptively and statistically using Fisher's exact tests. Thematic analysis was used to analyse narrative survey comments.

RESULTS

The response rate was 80%. The students perceived the OSCE to be a fair (95%) and meaningful way (97%) of assessing clinical competencies, which provided them with greater confidence for practice (91%). The OSCE was deemed to be less stressful than a written exam (65%). No significant differences were observed between second- and fourth-year students' perceptions. Five major themes included clarity of expectations, time allocated, consistency between examiners, influence on student learning, and personal impact.

DISCUSSION

The survey findings suggest that undergraduate speech and language therapy students across different stages of education and different clinical contexts perceive the OSCE as a fair and meaningful assessment approach. This study provides initial evidence that the OSCE is suitable for assessing clinical competencies specific to the profession of speech and language therapist.

Inter-rater reliability in clinical assessments: do examiner pairings influence candidate ratings?

BACKGROUND

The reliability of clinical assessments is known to vary considerably with inter-rater reliability a key contributor. Many of the mechanisms that contribute to inter-rater reliability however remain largely unexplained and unclear. While research in other fields suggests personality of raters can impact ratings, studies looking at personality factors in clinical assessments are few. Many schools use the approach of pairing examiners in clinical assessments and asking them to come to an agreed score. Little is known however, about what occurs when these paired examiners interact to generate a score. Could personality factors have an impact?

METHODS

A fully-crossed design was employed with each participant examiner observing and scoring. A quasi-experimental research design used candidate's observed scores in a mock clinical assessment as the dependent variable. The independent variables were examiner numbers, demographics and personality with data collected by questionnaire. A purposeful sample of doctors who examine in the Final Medical examination at our institution was recruited.

RESULTS

Variability between scores given by examiner pairs (N = 6) was less than the variability with individual examiners (N = 12). 75% of examiners (N = 9) scored below average for neuroticism and 75% also scored high or very high for extroversion. Two-thirds scored high or very high for conscientiousness. The higher an examiner's personality score for extroversion, the lower the amount of change in his/her score when paired up with a co-examiner; reflecting possibly a more dominant role in the process of reaching a consensus score.

CONCLUSIONS

The reliability of clinical assessments using paired examiners is comparable to assessments with single examiners. Personality factors, such as extroversion, may influence the magnitude of change in score an individual examiner agrees to when paired up with another examiner. Further studies on personality factors and examiner behaviour are needed to test associations and determine if personality testing has a role in reducing examiner variability.

Multiple Speed Assessments

Abstract. This paper presents Multiple Speed Assessments as an umbrella term to encompass a variety of approaches that include multiple (e.g., 20), short (e.g., 3 min), and often integrated interpersonal simulations to elicit overt behavior in a standardized way across participants. Multiple Speed Assessments can be used to get insight into the behavioral repertoire of a target person in situations sampled from a predefined target domain and their intraindividual variability across these situations. This paper outlines the characteristics and theoretical basis of Multiple Speed Assessments. We also discuss various already existing examples of Multiple Speed Assessments (Objective Structured Clinical Examinations, Multiple Mini-Interviews, and constructed response multimedia tests) and provide an overview of design variations. Finally, we present current research evidence and future research directions related to Multiple Speed Assessments. Although we present Multiple Speed Assessments in the context of personnel selection, it can also be used for assessment in the educational, personality, or clinical psychology field.

Are workplace-based assessment methods (DOPS and Mini-CEX) effective in nursing students' clinical skills? A single-blind randomized, parallel group, controlled trial

Background: Evaluation of clinical skills is critically important for nursing students. However, the quality of evaluation tools is poor.Objectives: To evaluate the effectiveness of Direct Observation of Procedural Skills (DOPS) and Mini-Clinical Evaluation Exercise (Mini-CEX) on clinical skills of nursing students.Methods: This study was conducted among 108 senior nursing students. Mini-CEX and DOPS were utilized to evaluate clinical skills in the intervention group.Results: The mean of students' scores in all of the five procedures was significantly higher in the intervention group compared to control group.. Students' scores for the procedures significantly raised through the first stage of DOPS and Mini-CEX to the third stage.Conclusions: Utilization of DOPS and Mini-CEX for evaluation of clinical skills in nursing students effectively enhance their learning ability. Implementing of such assessment methods lead to promoting clinical skills of students which eventually help them to provide high quality care for their patients.

The usefulness and acceptance of the OSCE in nursing schools

This qualitative study explores the usefulness and acceptance attributed by students and faculty members to an Objective Structured Clinical Evaluation (OSCE) administered to nursing undergraduates in Catalonia (Spain) for 10 years. Seventy undergraduate nursing students and twelve faculty members participated in the study. The data collection techniques included an open-ended questionnaire, a student focus group, and individualized faculty interviews. The students experienced the OSCE positively as a learning event that offered an opportunity for feedback that could help them master the required competencies. The OSCE increased students' responsibility by presenting them with a set of challenges that they had to tackle individually. Moreover, it reaffirmed their confidence in situations that closely resembled professional practice. Faculty members valued the ability of the OSCE to integrate and assess competencies, its objectivity, and the indirect information it provided on the effectiveness of the curriculum. The educational impact attributed to the OSCE and its acceptance among faculty and students suggest that it would be useful to re-implement it in the Bachelor's of Nursing in Catalan universities. Our findings may be of use to other nursing programs considering how to assess competency-based education, especially in the context of the European Higher Education Area.

The development of a six-station OSCE for evaluating the clinical competency of the student nurses before graduation: A validity and reliability analysis

BACKGROUND

Assessing clinical competency in undergraduate nursing students is essential before they enter clinical practice. The objective structured clinical examination (OSCE) is widely used and is part of clinical competency assessment. Aim The aim of this study was to develop and validate a six-station OSCE for evaluating the clinical competency of the student nurses before graduation.

METHODS

One-group longitudinal observational study. An OSCE of 6 stations of 10 min each was adopted. Blueprint setting, checklist development, and standard patient and examiner training were conducted. The mean, standard deviation, item scale (item-total correlation) and comparisons of extreme groups for item analysis. The inter-coder of examiners and inter-case of students were used to test reliability. Content Validity and concurrent validity were used to evaluate validity of OSCE.

RESULTS

One hundred nursing students participated in this study. The mean OSCE score of students who passed the registered nurses' examination was significantly higher than that of students who failed. Significant differences in OSCE score between students who remain versus left clinical job were observed at 3, 6 and 9 months after graduation.

CONCLUSION

The OSCE is predictable method for assessing undergraduate students' first year clinical retention.