Using objective structured clinical examinations to meet clinical competence evaluation challenges with distance education students

Recommendations After an Objective Structured Clinical Examination Program Evaluation

BACKGROUND

The use of objective structured clinical examinations (OSCEs) is a well-established method of clinical assessment in the medical profession, and these are currently being integrated into advanced practice nursing curricula.

METHOD

A program assessment was conducted at a current family nurse practitioner track within a southwestern U.S. university using the OSCE evaluative strategy. The logic model from the Centers for Disease Control and Prevention framework was incorporated. Both focus group and the de-identified faculty-scored student rubrics during the fall, spring, and summer semesters of one cohort of students (N = 73) were examined.

RESULTS

Strengths and challenges were found in the areas of student preparation, scoring/grading with variations of inter-rater reliability, and debriefing.

CONCLUSION

Establishing a specific OSCE center with a dedicated director could support adequate attainment of formative and summative competencies for nurse practitioners and provide faculty education and standardized student preparation, while adhering to the multiple guidelines and standards related to nurse practitioner competency evaluation with simulation. [J Contin Educ Nurs. 2023;54(4):169-175.].

Psychometric properties of clinical competency assessment instruments for psychiatric nurses: A systematic review of literature

BACKGROUND

Facilitating the healing process of patients with psychiatric disorders depends on high-quality mental health care and expert psychiatric nurses. A valid tool is required to objectively evaluate the quality of performance and competency of psychiatric nurses. This systematic review aimed to investigate and critique the psychometric properties of some psychiatric nursing clinical competency assessment instruments with Consensus-based standards for the selection of health status measurement instruments (COSMIN) checklist and according to Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA).

MATERIALS AND METHODS

To retrieve published articles using Persian and English keywords "psychiatric nursing," "competence," "competency," "tool," "checklist," "scale," "questionnaire," "psychiatric mental health nurse," were searched in databases without time limitation. Then, psychometric properties of selected instruments were evaluated using the COSMIN checklist and reported according to the PRISMA statement.

RESULTS

Most of these tools did not entirely and desirably report psychometric properties. It is not designed as a special tool for postgraduate psychiatric nursing.

CONCLUSION

The methodology of existing instruments does not meet the COSMIN checklist criteria; therefore it needs to develop. To assess the competency of psychiatric nursing postgraduate, a tool tailored to the cultural and social context and with acceptable psychometric properties is necessary.

The validity and reliability of clinical judgement and decision-making skills assessment in nursing: A systematic literature review

OBJECTIVES

To appraise the validity and reliability of approaches to assessing the clinical decision-making skills of nurses, and use findings to inform the assessment of students as they transition to newly qualified nurses.

DESIGN

The preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines were used to conduct the review.

DATA SOURCES

Medline, CINAHL and the British Nursing Index were searched from inception to November 2019.

REVIEW METHODS

Studies were grouped according to their assessment approach following a competency framework with findings presented as a narrative synthesis.

RESULTS

38 articles were included in the review which assessed clinical decision-making in a variety of settings; clinical practice, simulation, written examinations and self-assessment. Multi-level rubric and checklist approaches demonstrated good validity and reliability in practice and simulation settings, and the former was effective at differentiating between students at different stages of their training. Written, case study examinations were also effective at assessing clinical decision-making, although an optimum structure for their presentation was not possible to discern. Students tended to score themselves more highly than faculty staff when undertaking rubric-based self-assessments.

CONCLUSIONS

Findings suggest that the best approach to assess clinical decision-making for final year students is to use several low-stakes, snap-shot summative assessments in practice environments, which are marked using a multi-level observational rubric. To assure reliability, it is recommended that a small team of expert practice assessors undergo regular training and peer review, have protected time to complete their assessor role and are appropriately supported.

Educational intervention to improve the competence of Mexican physicians to provide nutritional management for diabetes mellitus type 2

OBJECTIVE

To evaluate the effect of a participative educational intervention on the clinical competence of Mexican family physicians regarding the nutritional management of patients with diabetes mellitus Type 2.

MATERIALS AND METHODS

Quasi-experimental study with a before-and-after control group. Convenience sample included 60 family physicians distributed in two social security primary health care units, randomly selected: 30 in the "A" unit and 30 in the "B" unit. Unit "A" was assigned randomly as control group, and "B" unit as intervention group. The intervention consisted of a theoretical-practical course-workshop that lasted six months where real cases were discussed and solved. Clinical competence was evaluated by means of an instrument designed ex professo, with a maximum theoretical value of 100 and 94% reliability according to the Kuder-Richardson test. Medians of clinical competence were compared among groups, before and after intervention, using the Mann-Whitney U test, while frequencies distribution of clinical competence level were analyzed with the Kolmogorov-Smirnov test (p≤0.05).

RESULTS

Overall medians and intervals for unit "A" were 28 (9-45) pre-intervention and 34 (11-51) pos-intervention, with before-after difference p>0.05; for unit "B", values were 32 (12-50) pre-intervention, 61 (36-82) pos-intervention, and before-after difference p≤0.05. No significant differences were found among groups pre-intervention (p>0.05), although they were observed pos-intervention (p≤0.05).

CONCLUSIONS

The educational intervention evaluated proved to improve, in a statistically significant way, the overall and by dimensions clinical competence level of Mexican family physicians for nutritional management of patients with diabetes mellitus type 2.

Flipping the Objective Structured Clinical Examination: A Teaching Innovation in Graduate Nursing Education

Objective evaluation of distance-based family nurse practitioner (FNP) students can be challenging. One FNP program piloted a teaching innovation, the video-enhanced objective structured clinical examination (VE-OSCE) or "flip" of the traditional face-to-face OSCE, to assess student clinical performance in a controlled online environment using a teleconferencing platform. This project sought to assess the VE-OSCE design, implementation, and ability to identify FNP student learning needs.

Competencia clínica de médicos de seguridad social guatemaltecos para manejar hepatitis virales en atención primaria

Objetivo Medir la competencia clínica para el diagnóstico y manejo de hepatitis virales en médicos de primer nivel de atención a la salud.Metodología Se efectuó un estudio transversal en el que usando un instrumento previamente validado se midió la competencia y posteriormente se comparó entre médicos adscritos a diversas unidades médicas de atención primaria a la salud (UMAPS) del Instituto Guatemalteco de Seguridad Social (IGSS). La información fue analizada mediante estadística descriptiva e inferencial no paramétrica. Se evaluaron 104 médicos de 5 UMAPS del IGSS.Resultados Se encontró un nivel muy bajo de competencia clínica para el diagnóstico y tratamiento de las hepatitis virales, dentro de un intervalo de 9 a 62 puntos obtenidos en el instrumento que tiene un valor máximo teórico de 88, sin encontrar diferencias estadísticamente significativas entre UMAPS. Conclusiones Se requiere educación continua en los médicos de las UMAPS del IGSS para mejorar sus competencias en hepatitis virales.

Mental health and learning disability nursing students' perceptions of the usefulness of the objective structured clinical examination to assess their competence in medicine administration

The aim of this study was to evaluate mental health and learning disability nursing students' perceptions of the usefulness of the objective structured clinical examination (OSCE) in assessing their administration of medicine competence. Learning disability (n = 24) and mental health (n = 46) students from a single cohort were invited to evaluate their experience of the OSCE. A 10-item survey questionnaire was used, comprising open- and closed-response questions. Twelve (50%) learning disability and 32 (69.6%) mental health nursing students participated. The OSCE was rated highly compared to other theoretical assessments; it was also reported as clinically real and as a motivational learning strategy. However, it did not rate as well as clinical practice. Content analysis of written responses identified four themes: (i) benefits of the OSCE; (ii) suggestions to improve the OSCE; (iii) concern about the lack of clinical reality of the OSCE; and (iv) OSCE-induced stress. The themes, although repeating some of the positive statistical findings, showed that participants were critical of the university setting as a place to conduct clinical assessment, highlighted OSCE-related stress, and questioned the validity of the OSCE as a real-world assessment. The OSCE has an important role in the development of student nurses' administration of medicine skills. However, it might hinder their performance as a result of the stress of being assessed in a simulated environment.

A descriptive study of older bipolar disorder residents living in New York City's adult congregate facilities

OBJECTIVES

Much of the research on geriatric bipolar disorder is from outpatient populations or epidemiological surveys with small samples. In contrast, in this study a descriptive analysis was conducted of geriatric and younger adult residents with bipolar disorder or mania in non-clinical adult congregate facilities (ACFs) in the greater New York City region.

METHODS

A total of 2602 ACF residents were evaluated in 19 facilities, across multiple demographic and health domains. Within this sample, 200 residents had chart diagnoses of bipolar disorder or mania. Among these, 50 geriatric residents (age ≥ 60) were compared to 50 younger adult residents (age < 50) on a number of demographic and health measures.

RESULTS

  Based on chart diagnoses, the overall prevalence of bipolar disorder was 7.8%. Compared to other studies of outpatient, epidemiological, and census samples, both older and younger residents with bipolar disorder had higher rates of cognitive impairment, impairment in executive functioning, vision impairment, and proportion of residents who had never been married. The younger group also had higher rates of obesity and the elderly group had a greater proportion of residents without high-school education. Both age groups had rates of lithium or valproate use comparable to those found in outpatient studies. Comparing the two age groups, the elderly sample had lower overall cognitive and executive functioning, and was using a larger number of medication classes than the younger group. The elderly group also had a larger proportion of residents who were separated/divorced or widowed compared to the younger group, which had higher rates of never-married residents.

CONCLUSIONS

Overall, both age groups had relatively high rates of bipolar disorder, with significant cognitive impairment, medical burden, obesity, mental health service use, and lower education levels, as compared to outpatient, epidemiological, and census samples. Of note was the significant cognitive impairment across age groups.

The cost of the objective structured clinical examination on an Italian nursing bachelor's degree course

The OSCE (Objective Structured Clinical Examination) is considered the most valid and reliable method for assessing the clinical skills of students training for health professions, but its use is limited by the related high costs. We analyzed the cost retrospectively of using an OSCE designed for second-year students (2009) in our degree course, adopting the Reznick et al. guidelines (1993), which recommend assessing both high-end costs and low-end costs. The high-end costs adopting the OSCE amounted to € 145.23 per student, while the low-end costs were € 31.51 per student. Considering the economic crisis and the cost-containment measures applied also in nursing education, strategies for further reducing costs are discussed.

Feasibility and impact of telemonitor-based depression care management for geriatric homecare patients

OBJECTIVE

The objective of this study was to test the feasibility, acceptability, and preliminary clinical outcomes of a method to leverage existing home healthcare telemonitoring technology to deliver depression care management (DCM) to both Spanish- and English-speaking elderly homebound recipients of homecare services.

MATERIALS AND METHODS

Three stand-alone, nonprofit community homecare agencies located in New York, Vermont, and Miami participated in this study. Evidence-based DCM was adapted to the telemonitor platform by programming questions and educational information on depression symptoms, antidepressant adherence, and side effects. Recruited patients participated for a minimum of 3 weeks. Telehealth nurses were trained on DCM and received biweekly supervision. On-site trained research assistants conducted in-home research interviews on depression diagnosis and severity and patient satisfaction with the protocol.

RESULTS

An ethnically diverse sample of 48 English- and Spanish-only-speaking patients participated, along with seven telehealth nurses. Both patients and telehealth nurses reported high levels of protocol acceptance. Among 19 patients meeting diagnostic criteria for major depression, the mean depression severity was in the "markedly severe" range at baseline and in the "mild" range at follow-up.

CONCLUSIONS

Results of this pilot support the feasibility of using homecare's existing telemonitoring technology to deliver DCM to their elderly homebound patients. This was true for both English- and Spanish-speaking patients. Preliminary clinical outcomes suggest improvement in depression severity, although these findings require testing in a randomized clinical trial. Implications for the science and service of telehealth-based depression care for elderly patients are discussed.

Continuing the cultural competency journey through exploration of knowledge, attitudes, and skills with advanced practice psychiatric nursing students: an exemplar

Numerous training and education programs have evolved to address culturally competent health care delivery. This article describes an exemplar educational approach used to teach cultural competency to beginning graduate psychiatric mental health nursing students. Using interactive strategies delivered within the 4 phases of the curriculum, the approach has been shown to facilitate students' ongoing journey to cultural competence. Building on baccalaureate nursing competencies, the course addresses attitudes, knowledge, skills, and cultural humility to strengthen cultural self-assessment, cross-cultural clinical practice expertise, and the use of culturally appropriate research for graduate students.

National survey of psychiatric mental health advanced practice nursing: the adequacy of educational preparation: voices of our graduates

BACKGROUND

Little is known about the fit between practice and graduate education from the perspective of psychiatric mental health advanced practice registered nurses (PMH APRNs).

OBJECTIVE

With the purpose of addressing this gap in our knowledge, an e-mail survey that was sent to certified PMH APRNs included a query "Did your program prepare you for your role?" and an open-ended follow-up question: "Please explain how your graduate program did not prepare you adequately for your role?"

DESIGN

Descriptive survey.

RESULTS

Although 70% of respondents reported satisfaction with their educational programs, content analysis of the comments indicated that the most pressing concerns related to instructional content about psychopharmacology, the practice of prescribing and psychiatric diagnosing, and that there was a perception of a disconnect between the role they were prepared for and their role in the present-day world of work.

CONCLUSIONS

The findings reinforce the need for the PMH nursing specialty to further define the optimal methods for preparing APRNs for the full scope of psychiatric services they are licensed to provide as the size of the PMH APRN workforce increases.