Virtual Evaluation of Clinical Competence in Nurse Practitioner Students

The design, implementation, and evaluation of a blended (in-person and virtual) Clinical Competency Examination for final-year nursing students

INTRODUCTION

Studies have reported different results of evaluation methods of clinical competency tests. Therefore, this study aimed to design, implement, and evaluate a blended (in-person and virtual) Competency Examination for final-year Nursing Students.

METHODS

This interventional study was conducted in two semesters of 2020-2021 using an educational action research method in the nursing and midwifery faculty. Thirteen faculty members and 84 final-year nursing students were included in the study using a census method. Eight programs and related activities were designed and conducted during the examination process. Students completed the Spielberger Anxiety Inventory before the examination, and both faculty members and students completed the Acceptance and Satisfaction questionnaire.

FINDINGS

The results of the analysis of focused group discussions and reflections indicated that the virtual CCE was not capable of adequately assessing clinical skills. Therefore, it was decided that the CCE for final-year nursing students would be conducted using a blended method. The activities required for performing the examination were designed and implemented based on action plans. Anxiety and satisfaction were also evaluated as outcomes of the study. There was no statistically significant difference in overt, covert, and overall anxiety scores between the in-person and virtual sections of the examination (p > 0.05). The mean (SD) acceptance and satisfaction scores for students in virtual, in-person, and blended sections were 25.49 (4.73), 27.60 (4.70), and 25.57 (4.97), respectively, out of 30 points, in which there was a significant increase in the in-person section compared to the other sections. (p = 0.008). The mean acceptance and satisfaction scores for faculty members were 30.31 (4.47) in the virtual, 29.86 (3.94) in the in-person, and 30.00 (4.16) out of 33 in the blended, and there was no significant difference between the three sections (p = 0.864).

CONCLUSION

Evaluating nursing students' clinical competency using a blended method was implemented and solved the problem of students' graduation. Therefore, it is suggested that the blended method be used instead of traditional in-person or entirely virtual exams in epidemics or based on conditions, facilities, and human resources. Also, the use of patient simulation, virtual reality, and the development of necessary virtual and in-person training infrastructure for students is recommended for future research. Furthermore, considering that the acceptance of traditional in-person exams among students is higher, it is necessary to develop virtual teaching strategies.

Effects of virtual reality OSCE on nursing students' education: a study protocol for systematic review and meta-analysis

INTRODUCTION

Virtual objective structured clinical examination (OSCE) has been shown to influence the performance of nursing students. However, its specific effects, particularly students' competence, stress, anxiety, confidence, satisfaction with virtual reality OSCE and examiners' satisfaction, remain unclear.

METHOD AND ANALYSIS

This study aims to assess the effects of virtual reality OSCE on nursing students' education. The study follows the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocol guidelines. A literature search is performed on electronic databases, namely, PubMed, Web of Science, CINAHL, EBSCO, EMBASE and the Cochrane Library. The inclusion criteria adhere to the PICOS principle, encompassing nursing students, including those studying in school and those engaged in hospital internship. This review includes studies on the use of virtual reality OSCE as an assessment tool, compared with traditional clinical examinations, such as in-person OSCE. The outcome assessments encompass (1) competence, (2) stress, (3) anxiety, (4) confidence, (5) student satisfaction with virtual reality OSCE and (6) examiners' satisfaction. These studies are designed as randomised controlled trials (RCTs) or quasi-experimental research. The search time is from the inception of each database to 30 June 2023, without language restriction. Studies for inclusion are screened by two reviewers for data extraction dependently. Any dispute is resolved through discussion. Unresolved disputes are decided by consulting a third author. For the risk of bias (ROB) assessment, the Cochrane ROB tool for RCTs and the risk of bias in non-randomised studies of intervention tool are used. Moreover, RevMan V.5.3 is used for meta-analysis.

ETHICS AND DISSEMINATION

This study protocol does not include any clinical research and thus does not require ethical approval. Research findings are published in a peer-reviewed journal.

PROSPERO REGISTRATION NUMBER

CRD42023437685.

Designing a touchless physical examination for a virtual Objective Structured Clinical Examination

Purpose

Given the COVID-19 pandemic, many Objective Structured Clinical Examinations (OSCEs) have been adapted to virtual formats without addressing whether physical examination maneuvers can or should be assessed virtually. In response, we developed a novel touchless physical examination station for a virtual OSCE and gathered validity evidence for its use.

Methods

We used a touchless physical examination OSCE station pilot-tested in a virtual OSCE in which Internal Medicine residents had to verbalize their approach to the physical examination, interpret images and videos of findings provided upon request, and make a diagnosis. We explored differences in performance by training year using ANOVA. In addition, we analyzed data using elements of Bloom's taxonomy of learning, i.e. knowledge, understanding, and synthesis.

Results

Sixty-seven residents (PGY1-3) participated in the OSCE. Scores on the pilot station were significantly different between training levels (F=3.936, p = 0.024, ηp2 = 0.11). The pilot station-total correlation (STC) was r = 0.558, and the item-station correlations ranged from r = 0.115-0.571, with the most discriminating items being those that assessed application of knowledge (interpretation and synthesis) rather than recall.

Conclusion

This touchless physical examination station was feasible, had acceptable psychometric characteristics, and discriminated between residents at different levels of training.

Understanding the Effects of Health Care Distance Simulation: A Systematic Review

ABSTRACT

The use of distance simulation has rapidly expanded in recent years with the physical distance requirements of the COVID-19 pandemic. With this development, there has been a concurrent increase in research activities and publications on distance simulation. The authors conducted a systematic review of the peer-reviewed distance health care simulation literature. Data extraction and a risk-of-bias assessment were performed on selected articles. Review of the databases and gray literature reference lists identified 10,588 titles for review. Of those, 570 full-text articles were assessed, with 54 articles included in the final analysis. Most of these were published during the COVID-19 pandemic (2020-2022). None of the included studies examined an outcome higher than a Kirkpatrick level of 2. Most studies only examined low-level outcomes such as satisfaction with the simulation session. There was, however, a distinction in studies that were conducted in a learning environment where all participants were in different locations ("distance only") as compared with where some of the participants shared the same location ("mixed distance"). This review exclusively considered studies that focused solely on distance. More comparative studies exploring higher level outcomes are required to move the field forward.

Evaluation of nursing process competencies, nursing quality, and patient safety using virtual simulation with debriefing: A quasi-experimental study

Background

Investing in clinical education is important for adult urgent and emergency surgery and traumatology as it promotes registered nurses' competencies by providing professional development training to respond to urgent or emergency surgeries.

Objective

To examine registered nurses' self-assessment of the effects of virtual video simulation with an immediate debriefing approach on nursing process competencies, nursing care quality, incomplete care, and patient safety in surgical units.

Methods

This study used a quasi-experimental two-group pre- and post-test design. The study was conducted at two provincial hospitals in Cambodia. Participants included registered nurses employed in surgical units. The experimental group (n = 46) completed a virtual video simulation and immediate debriefing. The control group (n = 35) completed virtual training on the nursing process. Data were collected two months after a successful second-week follow-up using Competency of Nursing Process, Cambodian Nursing Care Quality, Care Left Undone, and Patient Safety scales. Wilcoxon signed-rank test and Mann-Whitney U test were used to evaluate the differences before and after the sessions. Generalized linear model was used to compare the differences between the two groups.

Results

The results showed statistically significant improvements in the experimental group on competency, nursing care quality, patient safety, and reducing care left undone after the intervention. However, the control group revealed statistically insignificant differences. In addition, the experimental group provided positive feedback, such as experiencing a real patient scenario, developing critical-thinking, improving communication skills, and having an opportunity to ask questions.

Conclusion

Our study showed that VVS and immediate debriefing have the potential to support in-service training of RNs from diverse backgrounds. Particularly, integrating virtual video simulation and immediate debriefing may to promote competency in the nursing process and improve care outcomes.

Comparability of Objective Structured Clinical Examinations (OSCEs) and Written Tests for Assessing Medical School Students' Competencies: A Scoping Review

Objective Structured Clinical Examinations (OSCEs) and written tests are commonly used to assess health professional students, but it remains unclear whether the additional human resources and expenses required for OSCEs, both in-person and online, are worthwhile for assessing competencies. This scoping review summarized literature identified by searching MEDLINE and EMBASE comparing 1) OSCEs and written tests and 2) in-person and online OSCEs, for assessing health professional trainees' competencies. For Q1, 21 studies satisfied inclusion criteria. The most examined health profession was medical trainees (19, 90.5%), the comparison was most frequently OSCEs versus multiple-choice questions (MCQs) (18, 85.7%), and 18 (87.5%) examined the same competency domain. Most (77.5%) total score correlation coefficients between testing methods were weak (r < 0.40). For Q2, 13 articles were included. In-person and online OSCEs were most used for medical trainees (9, 69.2%), checklists were the most prevalent evaluation scheme (7, 63.6%), and 14/17 overall score comparisons were not statistically significantly different. Generally low correlations exist between MCQ and OSCE scores, providing insufficient evidence as to whether OSCEs provide sufficient value to be worth their additional cost. Online OSCEs may be a viable alternative to in-person OSCEs for certain competencies where technical challenges can be met.

Effectiveness of a virtual program for OSCE preparation during COVID-19: a descriptive and repeated cross-sectional study among nursing students

BACKGROUND

Despite the prevalence of distance learning during COVID-19, conducting clinical training for nursing students remains challenging. In compliance with social-distancing restrictions, a Zoom-based virtual OSCE preparation program for nursing students was designed, and it included clinical skills. The aims of this study were to assess nursing students' satisfaction with a virtual program for Objective Structured Clinical Examination (OSCE) preparation, and to evaluate its learning outcomes measured by OSCE scores as compared to those of in-person preparation programs.

METHODS

A descriptive and repeated cross-sectional study was designed. Students' satisfaction with the virtual program was based on a post-course survey and personal reflections. OSCE scores of graduates of the virtual program (n = 82) tested in 2021 were compared to those of 337 graduates of in-person programs tested in 2017-2020.

RESULTS

A post-program survey revealed that 88% of the students in 2021 were satisfied with the virtual program and felt it prepared them properly for the OSCE (26% agree and 62% strongly agree). No significant differences were found between OSCE scores following the virtual program conducted in 2021 and scores following in-person programs conducted in 2017-2020.

CONCLUSIONS

This study suggests that nursing education can benefit from integrating virtual programs which incorporate clinical practices into the curricula, without harming student competency. The study results may address the problem of maintaining clinical practices in a time of limited accessibility, and in settings of low resources. It is important to expand the investigation to long-term impact of virtual training programs on nursing students' competencies.

Comparing satisfaction and outcomes in on-campus versus virtual education for nurse practitioner students

ABSTRACT

This article highlights the development and implementation of interactive training experiences for graduate nursing students as part of specialty training in endocrinology. Emphasis was placed on accomplishing the shift from on-campus to virtual training while maintaining fidelity and student satisfaction. A total of 106 graduate nursing students from five cohorts submitted evaluations. Student satisfaction remained high regardless of whether the content was delivered in person or virtually. Most students in the virtual cohorts evaluated the online training positively. Student presentation grades were highest with on-campus delivery. Transitioning in-person training to a virtual environment can be an effective method of delivering nurse practitioner education while promoting student satisfaction. Recommendations for optimizing hybrid learning experiences are offered based on adult learning principles.

Global Trends in Nursing-Related Research on COVID-19: A Bibliometric Analysis

Background

Coronavirus disease 2019 (COVID-19) has significantly impacted healthcare, especially the nursing field. This study aims to explore the current status and hot topics of nursing-related research on COVID-19 using bibliometric analysis.

Methods

Between 2019 and 2022, publications regarding nursing and COVID-19 were retrieved from the Web of Science core collection. We conducted an advanced search using the following search query string: TS = (“Novel coronavirus 2019” or “Coronavirus disease 2019” or “COVID 19” or “2019-nCOV” or “SARS-CoV-2” or “coronavirus-2”) and TS = (“nursing” or “nurse” or “nursing-care” or “midwife”). Bibliometric parameters were extracted, and Microsoft Excel 2010 and VOSviewer were utilized to identify the largest contributors, including prolific authors, institutions, countries, and journals. VOSviewer and CiteSpace were used to analyze the knowledge network, collaborative maps, hotspots, and trends in this field.

Results

A total of 5,267 papers were published between 2020 and 2022. The findings are as follows: the USA, China, and the UK are the top three prolific countries; the University of Toronto, the Harvard Medical School, the Johns Hopkins University, and the Huazhong University of Science &amp; Technology are the top four most productive institutions; Gravenstein, Stefan, and White, Elizabeth M. from Brown University (USA) are the most prolific authors; The International Journal of Environmental Research and Public Health is the most productive journal; “COVID-19,” “SARS-CoV-2,” “nurse,” “mental health,” “nursing home,” “nursing education,” “telemedicine,” “vaccine-related issues” are the central topics in the past 2 years.

Conclusion

Nursing-related research on COVID-19 has gained considerable attention worldwide. In 2020, the major hot topics included “SARS-CoV-2,” “knowledge,” “information teaching,” “mental health,” “psychological problems,” and “nursing home.” In 2021 and 2022, researchers were also interested in topics such as “nursing students,” “telemedicine,” and “vaccine-related issues,” which require further investigation.

Nursing Students' Dissatisfaction With Course Organization and Student Engagement in Remote Learning 1 Year Post-COVID-19 Restrictions

BACKGROUND

Undergraduate and doctoral nursing students enrolled in face-to-face (F2F) learning transitioned abruptly to remote learning in March 2020. Few studies have focused on these nursing students' satisfaction with remote learning a year after the unplanned transition.

PURPOSE

Undergraduate and doctoral students' satisfaction with remote and F2F learning regarding course organization and student engagement were examined.

METHODS

A cross-sectional descriptive study was conducted among 522 nursing students at a research intensive university in the eastern United States. Survey data were analyzed with an analysis of variance to compare students' remote and F2F learning satisfaction within the undergraduate and doctoral programs.

RESULTS

Results indicated that nursing students who enrolled in F2F learning preferred F2F to remote learning (P < .001). Differences in satisfaction existed among programs (P = .035) and among undergraduate class levels (P < .001).

CONCLUSION

It is essential to learn why nursing students were dissatisfied with remote learning to improve these types of learning experiences in the future.

Evaluation of the Utility of Online Objective Structured Clinical Examination Conducted During the COVID-19 Pandemic

Background

The COVID-19 pandemic led to profound restrictions on the face-to-face learning and assessment in all educational institutions, particularly the medical schools. The College of Medicine and Medical Sciences of the Arabian Gulf University (CMMS-AGU) conducted the final exams, both theoretical and clinical components, for its MD students online. This study was conducted to evaluate the utility of online clinical exams held at CMMS-AGU.

Methods

This is a cross-sectional, mixed method study that included samples from final year medical students, examiners, and heads of clinical departments. Data were collected through surveys, structured interviews, documents' review, and calculation of online examination's psychometrics. Descriptive statistics were used. Quantitative data were presented in the form of means and standard deviations. Responses of heads of clinical departments in the structured interview were transcribed and analyzed thematically based on three pre-established themes.

Results

Quantitative and qualitative data on the utility (validity, reliability, acceptability, educational impact, and cost and feasibility) of online objective structured clinical examination (OSCE) were collected. Content validity of the online clinical examination was established through high mean scores of content representativeness, which was confirmed by the heads of clinical departments regarding the proper coverage of clinical skills. Criterion validity was established through a high correlation between clinical and theoretical exam results (r = 0.75). Reliability of the exam was established through an acceptable Cronbach's alpha value (0.70 to 0.78) over the four days of the examinations. The examinations were perceived as highly acceptable by both students and examiners. High educational impact was inferred from students' responses and review of documents. The examination was found to be feasible and of reasonable cost.

Conclusion

Online OSCE might be a good alternative of conventional clinical assessments in times of crises and impossibility of having in-person contact between students, examiners, and patients. An important major drawback is still present in such initiatives, which is the inability to assess students' physical examination skills.