Identifying strengths and weaknesses in the utilization of Objective Structured Clinical Examination (OSCE) in a nursing program

Meeting the AACN's Essentials for Competency-Based Education With Hands-Free Video-Recorded Skills Assessment

BACKGROUND

In the American Association of Colleges of Nursing's updated Essentials, competency-based education (CBE) is the expected standard for nursing curricula. CBE approaches include observation, practice repetition, demonstration, and video supervision guided by a framework of competency, feedback, and self-reflection.

METHOD

The promotion of skills competency among nursing students was facilitated by self-regulated learning using video skills-based competency testing. Video surveillance enabled students to practice and self-test their competency skills.

RESULTS

Self-videotaping by students provided a comfortable learning environment for repeated practice and fostered clinical judgment and accountability. Students reported increased confidence, decreased anxiety, and improved performance. Students developed skills and attitudes regarding self-reflection and self-remediation. Additionally, faculty reported improved visibility, a more efficient workload, and greater achievement of student outcomes.

CONCLUSION

Integrating CBE and incorporating video technology during skills testing enhances the opportunity for students to demonstrate skills competency and gain comfort in their own skills acquisition. [J Nurs Educ. 2023;62(12):733-736.].

A systematic review of effective quality feedback measurement tools used in clinical skills assessment

BACKGROUND

Objective Structured Clinical Examination (OSCE) is a valid tool to assess the clinical skills of medical students. Feedback after OSCE is essential for student improvement and safe clinical practice. Many examiners do not provide helpful or insightful feedback in the text space provided after OSCE stations, which may adversely affect learning outcomes. The aim of this systematic review was to identify the best determinants for quality written feedback in the field of medicine.   Methods: PubMed, Medline, Embase, CINHAL, Scopus, and Web of Science were searched for relevant literature up to February 2021. We included studies that described the quality of good/effective feedback in clinical skills assessment in the field of medicine. Four independent reviewers extracted determinants used to assess the quality of written feedback. The percentage agreement and kappa coefficients were calculated for each determinant. The ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) tool was used to assess the risk of bias.

RESULTS

14 studies were included in this systematic review. 10 determinants were identified for assessing feedback. The determinants with the highest agreement among reviewers were specific, described gap, balanced, constructive and behavioural; with kappa values of 0.79, 0.45, 0.33, 0.33 and 0.26 respectively. All other determinants had low agreement (kappa values below 0.22) indicating that even though they have been used in the literature, they might not be applicable for good quality feedback. The risk of bias was low or moderate overall.

CONCLUSIONS

This work suggests that good quality written feedback should be specific, balanced, and constructive in nature, and should describe the gap in student learning as well as observed behavioural actions in the exams.  Integrating these determinants in OSCE assessment will help guide and support educators for providing effective feedback for the learner.

Recommendations After an Objective Structured Clinical Examination Program Evaluation

BACKGROUND

The use of objective structured clinical examinations (OSCEs) is a well-established method of clinical assessment in the medical profession, and these are currently being integrated into advanced practice nursing curricula.

METHOD

A program assessment was conducted at a current family nurse practitioner track within a southwestern U.S. university using the OSCE evaluative strategy. The logic model from the Centers for Disease Control and Prevention framework was incorporated. Both focus group and the de-identified faculty-scored student rubrics during the fall, spring, and summer semesters of one cohort of students (N = 73) were examined.

RESULTS

Strengths and challenges were found in the areas of student preparation, scoring/grading with variations of inter-rater reliability, and debriefing.

CONCLUSION

Establishing a specific OSCE center with a dedicated director could support adequate attainment of formative and summative competencies for nurse practitioners and provide faculty education and standardized student preparation, while adhering to the multiple guidelines and standards related to nurse practitioner competency evaluation with simulation. [J Contin Educ Nurs. 2023;54(4):169-175.].

A systematic review of effective quality feedback measurement tools used in clinical skills assessment

Background: Objective Structured Clinical Examination (OSCE) is a valid tool to assess the clinical skills of medical students. Feedback after OSCE is essential for student improvement and safe clinical practice. Many examiners do not provide helpful or insightful feedback in the text space provided after OSCE stations, which may adversely affect learning outcomes. The aim of this systematic review was to identify the best determinants for quality written feedback in the field of medicine.   Methods: PubMed, Medline, Embase, CINHAL, Scopus, and Web of Science were searched for relevant literature up to February 2021. We included studies that described the quality of good/effective feedback in clinical skills assessment in the field of medicine. Four independent reviewers extracted determinants used to assess the quality of written feedback. The percentage agreement and kappa coefficients were calculated for each determinant. The ROBINS-I (Risk Of Bias In Non-randomized Studies of Interventions) tool was used to assess the risk of bias. Results: 14 studies were included in this systematic review. 10 determinants were identified for assessing feedback. The determinants with the highest agreement among reviewers were specific, described gap, balanced, constructive and behavioural; with kappa values of 0.79, 0.45, 0.33, 0.33 and 0.26 respectively. All other determinants had low agreement (kappa values below 0.22) indicating that even though they have been used in the literature, they might not be applicable for good quality feedback. The risk of bias was low or moderate overall. Conclusions: This work suggests that good quality written feedback should be specific, balanced, and constructive in nature, and should describe the gap in student learning as well as observed behavioural actions in the exams.  Integrating these determinants in OSCE assessment will help guide and support educators for providing effective feedback for the learner.

Objective structured clinical examination vs traditional clinical examination to evaluate students' clinical competence: A systematic review of nursing faculty and students' perceptions and experiences

BACKGROUND

Assessment of clinical competence of nursing students is an essential requirement in professional nursing education. This article summarizes the current published evidence indicating the nursing faculty and students' perceptions and experiences on benefits of OSCE over TCE.

METHODS

A systematic review was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) recommendations. Electronic databases including Scopus, Medline, Science Direct, CINAHL, EBSCO, PsychINFO, and Pubmed Central were used to identify relevant articles. The studies published between January 1, 2010 and December 31, 2020 and fourteen full-text articles that met all the inclusion criteria were included in the review.

RESULTS

The review identified five themes namely: a) Student's perception of OSCE; b) student's satisfaction regarding OSCE as an examination approach; c) students' perception of TCE as an examination approach; d) student's level of stress and anxiety towards OSCE vs TCE; and e) faculty member's perception and experience of OSCE.

CONCLUSION

We conclude that OSCE is a more credible assessment format to evaluate the clinical competence of undergraduate nursing students compared to the TCE method.

Design of a nursing objective structured clinical examination of a first-year clinical practice program

OBJECTIVE

The aim of the present study was to design a content-valid nursing objective structured clinical examination attending a first-year clinical nursing practice program.

METHOD

The examination was designed following a procedure based on the consensus of experts which was comprised of three phases: selection of the activities in which students should be competent according to the learning outcomes of the course, clinical case design, and integration of the clinical cases designed into the stations of the test.

RESULTS

Of the 44 surveys submitted for the design of the stations, 37 were answered, of which 31 respondents met the inclusion criteria of the panel of experts. The activities on which the experts reached the highest degrees of consensus were: basic physical assessment and monitoring of vital signs, assessment of hygiene and skin status, ability to develop care plans, management of safety principles in administration of medication and administration of oral medication. Based on the selected activities, the experts developed 20 clinical cases, from which a four-station nursing objective structured clinical examination was designed.

CONCLUSION

The structured methodology based on the design of experts enabled the design of a content-valid objective structured clinical examination appropriate for the evaluation of the learning outcomes achieved by the students attending a clinical practice program.

The usefulness and acceptance of the OSCE in nursing schools

This qualitative study explores the usefulness and acceptance attributed by students and faculty members to an Objective Structured Clinical Evaluation (OSCE) administered to nursing undergraduates in Catalonia (Spain) for 10 years. Seventy undergraduate nursing students and twelve faculty members participated in the study. The data collection techniques included an open-ended questionnaire, a student focus group, and individualized faculty interviews. The students experienced the OSCE positively as a learning event that offered an opportunity for feedback that could help them master the required competencies. The OSCE increased students' responsibility by presenting them with a set of challenges that they had to tackle individually. Moreover, it reaffirmed their confidence in situations that closely resembled professional practice. Faculty members valued the ability of the OSCE to integrate and assess competencies, its objectivity, and the indirect information it provided on the effectiveness of the curriculum. The educational impact attributed to the OSCE and its acceptance among faculty and students suggest that it would be useful to re-implement it in the Bachelor's of Nursing in Catalan universities. Our findings may be of use to other nursing programs considering how to assess competency-based education, especially in the context of the European Higher Education Area.

Objective structured clinical exam: how clinical nurse educators can support internationally educated nurses

Internationally educated nurses (IENs) require robust teaching programmes to support them through the objective structured clinical exam (OSCE) process to gain registration with the Nursing and Midwifery Council. There should be measures in place to support these nurses from a clinical and pastoral perspective. It is also important to learn from IENs, because moving to a different country is an emotional and life-changing experience that affects nurses and their families. Preparation is central to completing the programme successfully, alongside having the resources required in terms of time and a dedicated facility for practise. This article describes the development and implementation of an OSCE preparation teaching programme to support IENs, discusses the challenges for teachers and participants, and shows how having a dedicated facility and programme lead has led to excellent results.

A Systematic Review of the Use of Standardized Patients as a Simulation Modality in Nursing Education

AIM

The objective of the study was to search, extract, appraise, and synthesize studies using standardized patients (SPs) in nursing academia to determine how this modality of simulation is being used.

BACKGROUND

SPs are a common simulation modality used in nursing education.

METHOD

This review was conducted using the Preferred Reporting Items for Systematic Reviews and Meta-analyses. Five databases were searched as well as keywords to retrieve nonindexed citations for the period January 2011 to September 2016. The inclusion criteria included nurses, a simulated experience with SPs, and original research published in English.

RESULTS

Sixty-five studies were identified and analyzed.

CONCLUSION

More randomized controlled trials and studies with power analyses and validated measurement instruments are needed. Studies that compare SPs to high-fidelity simulators are also desired to determine optimal student learning outcomes and standardize best practices in simulation.

Experiences of using an OSCE protocol in clinical examinations of nursing students - A comparison of student and faculty assessments

Peer-assessment in nursing education using an OSCE protocol is an increasing educational activity that complements that of teachers. However, little is known about how students' and teachers' assessments correspond. The study aimed to compare OSCE assessments made by student examiners and faculty examiners during examinations of clinical skills in undergraduate nursing education. Four cohorts of third-year nursing students participated between 2014 and 2016. The students underwent a clinical examination of the management of central venous catheters and totally implantable venous access devices. Students who performed the examinations were observed both by a faculty examiner and student examiner. Both observers used the same OSCE protocol for the assessment but independently. The OSCE protocols from both faculty and student examiners were reviewed and compared. Total agreement between the student and faculty examiner was reached in 127 of 135 (94%) paired protocols. The level of agreement was substantial with a kappa value of 0.79 (95% CI 0.65-0.93). The conclusion was that the level of agreement between student and faculty examiners was high when using an OSCE protocol in clinical examinations of two different clinical skill tasks. The structured checklist (OSCE protocol) was easy to use for the student examiners despite the lack of experience or training in advance.

Mitigating the Challenges of Objective Structured Clinical Examination (OSCE) in Nursing Education: A Phenomenological Research Study

BACKGROUND

Objective evaluation of clinical competencies is a key component of undergraduate nursing education programs. Objective Structured Clinical Examination (OSCE), widely used in nursing education internationally has limited utilization in undergraduate nursing programs in the United States. The efficacy of OSCE as a valuable method of objective evaluation of students' clinical competencies is not so much the problem for the nursing faculty; overcoming the impediments of its implementation is a much greater challenge.

OBJECTIVE

To explore the strategies for mitigating the challenges of OSCE in baccalaureate nursing education program.

DESIGN/PARTICIPANTS/SETTING

In this descriptive phenomenological research study, 10 undergraduate nursing faculty participated in semi-structured interviews at a public university in the North-Eastern region of United States.

METHOD

Observations and semi-structured interviews were conducted. All interviews were recorded and transcribed. Colaizzi seven steps analysis framework was used to identify major themes in the collected data. The clear, detailed, and sequential procedural processes in both data collection and analysis ensured rigorous and trustworthy results.

RESULTS

Five mitigating OSCE strategies utilized by the faculty included: administrative and technical supports, use of clinical instructors during evaluation, faculty OSCE education, limitation of validation to the required skills essential for professional practice, and collaboration among all course faculty members.

CONCLUSION

Utilization of OSCE as an objective evaluation method is feasible in United States undergraduate nursing education programs if creative strategies are determined to mitigate its challenges.

The Barriers and Challenges of Applying New Strategies in the Clinical Evaluation of Nursing Students from the Viewpoints of Clinical Teachers

BACKGROUND

Numerous studies have emphasized the use of new approaches in clinical evaluation. However, there are some challenges and barriers to applying these new approaches. The aim of the present study was to investigate the barriers and challenges of applying new strategies in the clinical evaluation of nursing students from the viewpoints of clinical teachers.

MATERIALS AND METHODS

This cross-sectional study was conducted among 151 clinical teachers. A researcher-made questionnaire was used to collect data. The questionnaire was validated using library references and a variety of texts, as well as thorough consulting with 15 clinical teachers. The questionnaire's reliability was approved with a Cronbach's alpha of 78%. Data analysis was conducted using Pearson's correlation coefficient, one-way analysis of variance (ANOVA), and descriptive statistics in SPSS software.

RESULTS

The highest score was related to the "students and clinical environment" domain [24.05 (8.10)], and the lowest to the "facilities" domain [13.31 (1.50)]. One-way ANOVA results showed a significant relationship between the mean scores of academic degree and the two domains of "tests" (F = 9.66, p < 0.001) and "facilities" (F = 8.26, p < 0.001).

CONCLUSIONS

The implementation of a new approach for evaluating clinical training requires infrastructure and overcoming executive obstacles. Educating students and clinical teachers on new evaluation methods requires their familiarity with the implementation process as well as encouragement and support by their educational institution and administrators.

Waging the War on Clinical Grade Inflation: The Ongoing Quest

This study examined the presence of grade inflation in clinical courses 9 years after implementing strategies to improve grading precision. A comparison of clinical grades for cohort I (1997-2002) with cohort II (2009-2016) showed statistically lower grades in 2 courses (Adult 1 and Maternity) for cohort II. Suggestions for changing the way clinical experiences are planned, executed, and evaluated are provided.

Towards the implementation of OSCE in undergraduate nursing curriculum: A qualitative study

BACKGROUND

Despite the fact that Objective Structured Clinical Examination is considered to be an efficient assessment method, their implementation in the undergraduate nursing curriculum encounters difficulties. However, the initiative of the European Higher Education Area to promote competency-based assessment may represent an opportunity to introduce this instrument in undergraduate nursing curriculum.

OBJECTIVE

To explore the perception of nursing faculty members regarding the implementation of the OSCE as an assessment tool in Catalan Nursing Schools.

DESIGN/PARTICIPANTS/SETTING

In this qualitative study, fifteen teachers participated in semi-structured interviews in eight Catalan Nursing Schools.

METHODS

Semi-structured interviews were conducted. A thematic content analysis was used to identify major themes in the interview data and collaborative analysis was undertaken to ensure rigorous results.

RESULTS

The relevant aspects that are emphasized by teachers included the consideration of the dual purpose of the OSCE via its formative and evaluative facets by enhancing the feedback received by students about their performance on the OSCE. The OSCE should be administered towards the end of the degree program and should complement other methods of assessment. Despite its high cost, the OSCE was deemed to be efficient as it enables student competencies to be assessed with objective criteria, which is a difficult task with other instruments. OSCE implementation is feasible with the institutional support of and collaborative work between schools.

CONCLUSIONS

The implementation of the OSCE in the Catalan undergraduate nursing degree programs is feasible if the project receives the support of all involved parties and if creative strategies are determined to reduce economic costs and optimize resources. With adequate feedback, the OSCE is an assessment tool that can provide high-impact training to students.

Assessing Nursing Students' Clinical Competencies Using a Problem-Focused Objective Structured Clinical Examination

The objectives of this study were to develop, implement, and evaluate an innovative modified Objective Structured Clinical Examination (OSCE) model, and to compare students' performance of different clinical skills as assessed by standardized patients and OSCE examiners. Data were obtained from final year undergraduate students undergoing the modified OSCE as a graduation examination. Seventy-seven students rotated through four stations (nine substations). Standardized patients scored students higher than examiners in history taking (9.14 ± 0.92 vs. 8.42 ± 0.85), response to emergency event (8.88 ± 1.12 vs. 7.62 ± 1.54), executive medical orders (8.77 ± 0.96 vs. 8.25 ± 1.43), technical operation (18.21 ± 1.26 vs. 16.91 ± 1.35), nursing evaluation (4.53 ± 0.28 vs. 4.29 ± 0.52), and health education stations (13.79 ± 1.31 vs. 11.93 ± 2.25; p < .01). In addition, the results indicated that the difference between standardized patient and examiner scores for physical examination skills was nonsignificant (8.70 ± 1.18 vs. 8.80 ± 1.27; p > .05). The modified, problem-focused, and nursing process-driven OSCE model effectively assessed nursing students' clinical competencies, and clinical and critical thinking.

Ensino do manejo da via aérea com máscara laríngea: estudo randomizado controlado

RESUMO Objetivo: ensinar manejo da via aérea com máscara laríngea a estudantes de enfermagem mediante aula expositivo-dialogada acompanhada de atividade prática em laboratório ou exclusivamente aula simulada. Método: ensaio clínico randomizado controlado. População: bacharelandos oitavo semestre. Amostra: 17 estudantes randomizados em grupo intervenção (GI: aula simulada) ou controle (GC: aula expositivo-dialogada e atividade prática em laboratório). Elaborados e validados instrumentos: avaliação escrita, cenário de simulação, avaliação clínica objetiva estruturada (checklist). Coletou-se dados em workshop. Aplicaram-se teste escrito e avaliação clínica estruturada em cenário de simulação filmada e avaliada por três experts. Resultados: idade 24,4±4,2 anos. Acertos GC: pré-teste 66±10%; pós-teste 84±8%. GI: pré-teste 65±5%; pós-teste 86±11%. Cenário: GC 78±5.2%; GI 84±8.9%. Conclusão: estratégias proporcionaram aquisição de conhecimento, habilidades e tomada de decisão, indispensáveis para atingir objetivos do cenário. Houve incorporação de conhecimento em manejo da via aérea com máscara laríngea, evidenciado pelo incremento dos escores no teste escrito e cenário.

How to maintain equity and objectivity in assessing the communication skills in a large group of student nurses during a long examination session, using the Objective Structured Clinical Examination (OSCE)

BACKGROUND

While development, testing, and innovation of the Objective Structured Clinical Examination (OSCE) are common in the international literature, studies from the United States of America (USA), Australia, and the United Kingdom (UK) predominate. There is little known about OSCE use in European countries, such as Italy, where other than cost analysis, there is little reporting of OSCE use or validation.

OBJECTIVES

This paper reports on one Italian initiative, which evaluated the equity and objectivity of the OSCE method of assessing communication skills.

DESIGN

An OSCE method was used to assess the communication skills of first-year students of the Degree Course in Nursing. A method of simulation was implemented through role-playing with standardized patients. An observational method was used to collect data.

PARTICIPANTS AND SETTINGS

Four hundred and twenty-one first-year undergraduate nursing students at one university site in Italy took part.

METHODS

Ten examination sessions were carried out. The students' performances were assessed by two examiners who used a structured observation grid and conducted their assessment separately. A situation simulated by four nurses with experience as actors was used as the topic for the students' examination.

RESULTS

Calculation of the daily rate of students who passed the examination revealed a random distribution over time. The nonparametric correlation indexes referring to the assessments and to the scores assigned by the two examiners proved statistically significant (P≤0.001).

CONCLUSIONS

The study confirmed the validity of the OSCE method in ensuring equity and objectivity of communication skills assessment in a large population of nursing students for the purpose of certification throughout the duration of the examination. This has important implications for nurse education and practice as the extent to which OSCE approaches, while deemed objective, are culturally sensitive or valid and reliable across cultures is not clear. This is something that requires further research and examination in this field.

Changes in Student Performance and Confidence with a Standardized Patient and Standardized Colleague Interprofessional Activity

OBJECTIVE

To assess the impact of a standardized patient and standardized colleague interprofessional activity on student performance and perceived confidence in communicating with patients and physicians.

METHODS

Students in the third professional year were presented with a practice and final activity including a standardized patient interaction, SOAP note preparation, and standardized colleague interaction. Student performance was measured by assessment rubrics on practice and final activities. Students' perceived confidence was measured via presurvey and postsurvey.

RESULTS

Students performed significantly better from the practice to the final activity with regard to communicating with patients, SOAP note, and the overall activity with a mean difference (95% CI) of 9.2 (6.9-11.5), 3.6 (1.3-5.8), and 3.9 (2.0-5.7), respectively. There was a positive significant change from presurvey to postsurvey in students' confidence talking to patients and physicians on majority of questions.

CONCLUSION

This study demonstrates that active learning and integrated assessments improve overall student performance. Integration of interprofessional education also has positive effects on students' perceived confidence.

Have OSCEs come of age in nursing education?

This article is intended to contribute to the current debate as to whether the objective structured clinical examination (OSCE) should become a standard assessment tool for undergraduate nursing education as they currently are for medicine. The authors describe how one UK university developed an OSCE for a nursing undergraduate programme with the aim of emphasising the need for nursing students to be competent in clinical skills and offering a means of standardising the assessment of these skills. There has been an increasing number of research studies carried out in this area at international level and this article's main contribution to the literature is the description of the Angoff standard-setting procedure that was used to calibrate the OSCE at this University and which makes it the first nursing OSCE in the UK to incorporate a scientific standard-setting procedure.

Evaluating Objective Structured Clinical Assessment (OSCA) in undergraduate nursing

The aims of this mixed methods study were to gain insight into how individual assessors determine an Objective Structured Clinical Assessment (OSCA) result for undergraduate nursing students and identify whether individual assessor perceptions and professional characteristics have an impact on students' results. Results from 25 participants showed that although less than half (44%) of the participants were teaching in the course that they were assessing, the participants were highly experienced clinicians and nearly three-quarters (72%) had completed formal teaching qualifications. There were wide variations in pass rates (16.7-90%) between assessors. The widest disparity was observed between assessors with and those without critical care experience (66% versus 39%), as well as assessors who were teaching the course and those who were not (68% versus 49%). Qualitative analysis revealed three dominant themes within participants' transcripts. The themes focused on determining student safety, and the use of personal perceptions and clinical experience to determine competency. Findings indicate that assessors' individual perceptions and clinical experience have the potential to influence and determine undergraduate nursing students OSCA results. Development of criteria standards and objective assessment may be enhanced by greater involvement of assessors and thorough education and training within the context of student assessments.

Vascular Nursing in Greece

Although peripheral arterial disease is prevalent in the primary care setting, insufficient vascular education among nurses and physicians coupled with certain economic constraints undermines treatment efficacy. Moreover, the burden of advanced venous pathology such as posthrombotic syndrome, venous ulcers, and lymphedema remains suboptimally treated. This article advocates the development of a vascular nursing specialty as a means to improving vascular care especially nowadays, when health care providers dictate comprehensive and cost-effective nursing practice and patient management. It also presents the first attempt to organize a Vascular Nursing Educational Session in Greece.