Subjectivity and the valid assessment of pre-registration student nurse clinical learning outcomes: implications for mentors

Conformer-Based Dental AI Patient Clinical Diagnosis Simulation Using Korean Synthetic Data Generator for Multiple Standardized Patient Scenarios

The goal of clinical practice education is to develop the ability to apply theoretical knowledge in a clinical setting and to foster growth as a professional healthcare provider. One effective method of achieving this is through the utilization of Standardized Patients (SP) in education, which familiarizes students with real patient interviews and allows educators to assess their clinical performance skills. However, SP education faces challenges such as the cost of hiring actors and the shortage of professional educators to train them. In this paper, we aim to alleviate these issues by utilizing deep learning models to replace the actors. We employ the Conformer model for the implementation of the AI patient, and we develop a Korean SP scenario data generator to collect data for training responses to diagnostic questions. Our Korean SP scenario data generator is devised to generate SP scenarios based on the provided patient information, using pre-prepared questions and answers. In the AI patient training process, two types of data are employed: common data and personalized data. The common data are employed to develop natural general conversation skills, while personalized data, from the SP scenario, are utilized to learn specific clinical information relevant to a patient's role. Based on these data, to evaluate the learning efficiency of the Conformer structure, a comparison was conducted with the Transformer using the BLEU score and WER as evaluation metrics. Experimental results showed that the Conformer-based model demonstrated a 3.92% and 6.74% improvement in BLEU and WER performance compared to the Transformer-based model, respectively. The dental AI patient for SP simulation presented in this paper has the potential to be applied to other medical and nursing fields, provided that additional data collection processes are conducted.

Content Validation of the Quality and Safety Framed Clinical Evaluation for Nurse Practitioner Students

BACKGROUND

Clinical performance evaluation is a high-stakes assessment that requires a valid and reliable instrument. There appear to be no identified clinical evaluation instruments developed for nurse practitioner (NP) students that have undergone content validation.

PURPOSE

This study describes the process to develop, validate, and pilot a clinical evaluation instrument for NP students framed in the Quality and Safety Education of Nurses Competencies for advanced practice.

METHODS

Six expert nurse educators were recruited for the purpose of rating items for relevance for inclusion in the Quality and Safety Framed Clinical Evaluation Instrument for NP Students using the content validity index. The instrument was pilot tested in a primary care clinical course.

RESULTS

Two rounds of review yielded a 34-item content validated instrument.

CONCLUSIONS

Evidence suggests that the Quality and Safety Framed Clinical Evaluation for NP Students is a valid and reliable instrument to evaluate the clinical performance of NP students.

National survey: Developing a common approach to grading of practice in pre-registration midwifery

This paper presents the final phase of a national project exploring grading of practice in programmes leading to registration as a midwife in the United Kingdom. The aim was to develop a generic framework for grading practice, enhancing standardisation while enabling flexibility in application of current and new educational standards. A mixed method on-line survey considered existing practice assessment tools, factors contributing to robust and reliable assessment and perceptions of two assessment tools developed by the research team: a 'Lexicon Framework' and 'Rubric', which were tested through scenarios. Participants included 170 midwifery and nursing academics, clinicians and students, representing 20 universities in the UK. Seven key themes emerged, from which an 'Evidence Based Model for Professional Practice Assessment' was developed. The proposed tools were overall positively evaluated and demonstrated a good level of reliability. A national tool to standardise midwifery practice assessment is recommended, and scope for transferability of our tools to all midwifery programmes and to nursing was identified. Other recommendations include engagement of key stakeholders in development of practice assessment documentation, and maintaining the professional purpose of grading practice as central to the process. A set of key principles for assessing practice is presented.

Exploring mentors' interpretation of terminology and levels of competence when assessing nursing students: An integrative review

OBJECTIVES

The purpose of this integrative review is to evaluate the empirical and theoretical literature on the challenges mentors face in interpreting and assessing levels of competence of student nurses in clinical practice.

DESIGN

An integrative review of the literature.

DATA SOURCES

An extensive and systematic literature search was conducted covering the period 1986-September 2016 across twelve databases covering health and education related publications. Grey literature was searched from wide relevant sources.

REVIEW METHODS

Sources were eligible for review when they referred to mentor's interpretation or assessment of student nurses' level of competence in practice settings. Methodological rigor of the included studies was evaluated with the Mixed Methods Appraisal Tool.

RESULTS

After screening 1951 records by titles, abstracts and full text, 8 were selected for review. The methodological quality of the studies was moderate. The studies reported: Accurate and fair assessment of students is impeded by a lack of transparent and explicit criteria.

CONCLUSIONS

There is a need to establish a transparent and common language to distinguish between and facilitate interpretation of different levels of competence. Well-designed rubrics might offer the solution to the challenges faced in practice-based assessment and necessitates further research into their use.

Nursing students' and preceptors' experience of assessment during clinical practice: A multilevel repeated-interview study of student-preceptor dyads

Nursing students' learning process during clinical practice is dependent on the quality of their assessment, and the feedback that is given, however an issue that is very little examined. The aim of this study was to investigate student-preceptor dyads and their experiences of mid-course discussions and final assessment. Data were collected through repeated semi-structured individual interviews with 13 dyads close in time to the assessment discussions, and were analyzed by searching for similarities and differences in their experiences. Both students and preceptors shared the view that the preparations before the discussions caused uncertainty, but they described different reasons. Both students and preceptors meant that the assessment tool was used in a structured way. The feedback mediated to students emphasized what was important to concentrate on in the next part or next period, but less about the individual students' progress. The student-preceptor dyads did not share the view of what content had been in focus in the discussions and there were variations in the details. Conclusions can be drawn that one of the crucial elements of an assessment discussion, to give students constructive feedback in relation to learning outcomes, was not fulfilled.

Competence evaluation processes for nursing students abroad: Findings from an international case study

BACKGROUND

Assessing clinical competence in nursing students abroad is a challenge, and requires both methods and instruments capable of capturing the multidimensional nature of the clinical competences acquired.

OBJECTIVES

The aim of the study was to compare the clinical competence assessment processes and instruments adopted for nursing students during their clinical placement abroad.

DESIGN

A case study design was adopted in 2015.

SETTING AND PARTICIPANTS

A purposeful sample of eight nursing programmes located in seven countries (Belgium, Denmark, Greece, Norway, Poland, Portugal and Italy) were approached.

METHODS

Tools as instruments for evaluating competences developed in clinical training by international nursing students, and written procedures aimed at guiding the evaluation process, were scrutinised through a content analysis method.

FINDINGS

All clinical competence evaluation procedures and instruments used in the nursing programmes involved were provided in English. A final evaluation of the competences was expected by all nursing programmes at the end of the clinical placement, while only four provided an intermediate evaluation. Great variability emerged in the tools, with between five and 88 items included. Through content analysis, 196 items emerged, classified into 12 different core competence categories, the majority were categorised as 'Technical skills' (=60), 'Self-learning and critical thinking' (=27) and 'Nursing care process' (=25) competences. Little emphasis was given in the tools to competences involving 'Self-adaptation', 'Inter-professional skills', 'Clinical documentation', 'Managing nursing care', 'Patient communication', and 'Theory and practice integration'.

CONCLUSIONS

Institutions signing Bilateral Agreements should agree upon the competences expected from students during their clinical education abroad. The tools used in the process, as well as the role expected by the student, should also be agreed upon. Intercultural competences should be further addressed in the process of evaluation, in addition to adaptation to different settings. There is also a need to establish those competences achievable or not in the host country, aiming at increasing transparency in learning expectations and evaluation.

The perspectives of preceptors regarding clinical assessment for undergraduate nursing students

AIM

To explore the perspectives of preceptors about clinical assessment for undergraduate nursing students in transition to practice.

BACKGROUND

The assessment of clinical competence is a complex process due to the diverse nature of nursing practice. As such, nurse preceptors play a crucial role in clinical assessment for nursing students. Although preceptors may enjoy the process of clinical teaching and assessment, they face challenges that need to be addressed to facilitate students' clinical learning and assessment.

METHODS

An exploratory qualitative approach was adopted. A purposive sample of 17 preceptors from two tertiary hospitals in Singapore was recruited to participate in three focus group discussions from March to July 2014.

FINDINGS

Five themes emerged from the data analysis: the need for a valid and reliable clinical assessment tool; meaningful reflection and feedback; varied methods in clinical assessment; high level of commitment and struggles with dual roles and the need to enhance the support system for preceptors.

CONCLUSIONS

There is no doubt that a valid and reliable clinical assessment tool can serve as a guide for preceptors to enhance their clarity in assessment and feedback. It is essential for all stakeholders to be involved in the development of the clinical assessment tool. Workload, time, support system and formal educational programmes for preceptors influenced their preparation and self-confidence.

IMPLICATIONS FOR NURSING AND/OR HEALTH POLICY

The themes that emerged from the focus group discussions could assist in the development of a more reliable and valid clinical assessment tool to evaluate the clinical competence of nursing students in transition to practice. Clinical and educational institutions could collaborate closely to develop an educational programme and a preceptorship handbook, focusing on pedagogical contents for clinical education and assessment.

Preceptors' experiences of using a competence assessment tool to assess undergraduate nursing students

The purpose of this study was to explore Irish preceptors' experience of using a competence tool to assess undergraduate nursing students' clinical competence. This study used a mixed methods design. This study was conducted in two phases, the qualitative phase involved six focus group interviews to ascertain preceptors' experiences of using an assessment tool to assess clinical competence. The quantitative phase involved a descriptive survey measuring preceptors (N = 843) attitudes linked with the use of the assessment tool. The key themes that emerged from qualitative analysis were challenges of using the assessment tool, recognising competence and valuing adult learners. The challenges of using the tool included negotiating complex language and time constraints in completing assessments. Recognising competence revealed the use of intuition and subjectivity. While valuing adult learners acknowledged the reciprocal learning process between the preceptor and the learner. These findings reveal the inherent skills of preceptors to intuitively and subjectively recognise competence. The quantitative findings revealed merits and challenges for the preceptors using the assessment tool. In particular the complexity of the language was highlighted as an issue. A key recommendation from this research is the need to revise the assessment tool to support objective and subjective measurement of competence.

Learning and assessing competence in reflective practice: Student evaluation of the relative value of aspects of an integrated, interactive reflective practice syllabus

Although the literature is rich with information on the value of reflective practice, there is a paucity of information on techniques to assess whether professionals have learnt this essential skill. In this article, we describe the introduction and evaluation of an innovative, integrated, interactive approach to teaching and assessing competence in reflective practice using an online self directed learning package. As part of a new staff start up scholarship of teaching and learning grant project in an Australian university, we converted an existing one day reflective practice workshop for undergraduate nursing students to an interactive online learning package that could also be applicable for students in other health professions. The assessment of learning in the package was integrated with overall clinical competence assessment using Tanner's Clinical Judgement Model and Lasater's Clinical Judgement rubric to enable immediate online feedback to students on their progress. In this article, we focus on those aspects of the package that students evaluated as most beneficial to their learning, specifically immediate feedback by lecturers guided by Lasater's rubric.

Clinical practice models in nursing education: implication for students' mobility

BACKGROUND

In accordance with the process of nursing globalization, issues related to the increasing national and international mobility of student and qualified nurses are currently being debated. Identifying international differences and comparing similarities for mutual understanding, development and better harmonization of clinical training of undergraduate nursing students is recommended.

AIMS

The aim of the study was to describe and compare the nature of the nursing clinical practice education models adopted in different countries.

METHODS

A qualitative approach involving an expert panel of nurses was adopted. The Nominal Group Technique was employed to develop the initial research instrument for data collection. Eleven members of the UDINE-C network, representing institutions engaged in the process of professional nursing education and research (universities, high schools and clinical institutes), participated. Three data collection rounds were implemented. An analysis of the findings was performed, assuring rigour.

RESULTS

Differences and homogeneity are reported and discussed regarding: (a) the clinical learning requirements across countries; (b) the prerequisites and clinical learning process patterns; and (c) the progress and final evaluation of the competencies achieved.

CONCLUSIONS

A wider discussion is needed regarding nursing student exchange and internalization of clinical education in placements across European and non-European countries. A clear strategy for nursing education accreditation and harmonization of patterns of organization of clinical training at placements, as well as strategies of student assessment during this training, are recommended. There is also a need to develop international ethical guidelines for undergraduate nursing students gaining international experience.

Performance evaluation of nursing students following competency-based education

BACKGROUND

Competency-based education is known to improve the match between educational performance and employment opportunities.

OBJECTIVES

This study examined the effects of competency-based education on the learning outcomes of undergraduate nursing students.

DESIGN

The study used a quasi-experimental design.

PARTICIPANTS

A convenience sample of 312 second-year undergraduate nursing students from northern and southern Taiwan participated in the study.

METHODS

The experimental group (n=163) received competency-based education and the control group received traditional instruction (n=149) in a medical-surgical nursing course. Outcome measures included students' scores on the Objective Structured Clinical Examination, Self-Evaluated Core Competencies Scale, Metacognitive Inventory for Nursing Students questionnaire, and academic performance.

RESULTS

Students who received competency-based education had significantly higher academic performance in the medical-surgical nursing course and practicum than did the control group. Required core competencies and metacognitive abilities improved significantly in the competency-based education group as compared to the control group after adjusting for covariates.

CONCLUSIONS

Competency-based education is worth implementing and may close the gap between education and the ever-changing work environment.

Mentoring student nurses and the educational use of self: a hermeneutic phenomenological study

BACKGROUND

In the United Kingdom, pre-registration nurse education relies on workplace mentors to support and assess practice learning. Despite research to clarify expectations and develop support structures, mentors nevertheless report being overwhelmed by the responsibility of mentoring alongside their clinical work. Understanding of their lived experience appears limited.

OBJECTIVES

The aim of the study was to achieve a deeper understanding of the lived experience of mentoring, searching for insights into how mentors can be better prepared and supported.

DESIGN

The mentor lifeworld was explored utilizing a hermeneutic phenomenological methodology drawing on Heidegger.

SETTINGS AND PARTICIPANTS

Twelve mentors, who worked in a range of clinical settings in England were recruited via purposive and snowball sampling.

METHOD

Participants described their experiences of mentoring through in-depth interviews and event diaries which included 'rich pictures'. Analysis involved the application of four lifeworld existentials proposed by van Manen - temporality, spatiality, corporeality and relationality.

FINDINGS

The essence of being a mentor was 'the educational use of self'. Temporality featured in the past self and moving with daily/work rhythms. Spatiality evoked issues of proximity and accountability and the inner and outer spaces of patients' bodies. Mentor corporeality revealed using the body for teaching, and mentors revealed their relationality in providing a 'good educational experience' and sustaining their 'educational selves'.

CONCLUSIONS

'The educational use of self' offers insight into the lived experience of mentors, and exposes the potentially hidden elements of mentoring experience, which can inform mentor preparation and support.

Nursing students' and preceptors' perceptions of using a revised assessment form in clinical nursing education

Assessment of students' learning is a crucial question when great changes occur in the higher education sector. One such educational reform is the Bologna declaration, the requirements of which have resulted in significant modifications in documents as assessment forms for clinical education. The aim of this study was to investigate students' and preceptors' perceptions of using the revised version of an assessment form, the AssCE form. Using convenience sampling, a questionnaire survey was completed by 192 nursing students and 101 preceptors. Most of the participants found that the revised AssCE form was possible to use during different years of the programme, and factors in the AssCE form were possible to combine with learning outcomes in the course syllabus. Most participants perceived that the scale added to each factor facilitated the assessment dialogue and offered possibilities to illustrate the students' development during clinical periods. Findings also showed that students were most often prepared with self-assessment before the assessment discussions. More information about the use of the AssCE form, also in combination with learning outcomes in the course syllabus, may further support the use of the form and contribute to students' development during clinical practice.

Creating a brief rating scale for the assessment of learning disabilities using reliability and true score estimates of the scale's items based on the Rasch model

The purpose of the present studies was to provide the means to create brief versions of instruments that can aid the diagnosis and classification of students with learning disabilities and comorbid disorders (e.g., attention-deficit/hyperactivity disorder). A sample of 1,108 students with and without a diagnosis of learning disabilities took part in study 1. Using information from modern theory methods (i.e., the Rasch model), a scale was created that included fewer than one third of the original battery items designed to assess reading skills. This best item synthesis was then evaluated for its predictive and criterion validity with a valid external reading battery (study 2). Using a sample of 232 students with and without learning disabilities, results indicated that the brief version of the scale was equally effective as the original scale in predicting reading achievement. Analysis of the content of the brief scale indicated that the best item synthesis involved items from cognition, motivation, strategy use, and advanced reading skills. It is suggested that multiple psychometric criteria be employed in evaluating the psychometric adequacy of scales used for the assessment and identification of learning disabilities and comorbid disorders.

Developing an assessment tool for intended learning outcomes in clinical practice for nursing students

This report describes the development of a new criterion based reference tool to assess nursing knowledge and competence in clinical practice. Nursing education has changed from educating a profession, based on tested experience, to being based on a scientific approach and research based knowledge. Assessment should be capable of measuring whether intended learning outcomes have been reached or not, and if the aims of a course have been fulfilled in order to ensure safe and competent nursing care. The intended learning outcomes from a first year course syllabus were integrated and formed into a three-graded criterion-referenced assessment tool, Assessment of Clinical Education, (AClEd). The AClEd is to be seen as a template, and may be tailor-made in accordance to the objectives, level and criteria of a specific course. The tool showed validity in assessing nursing skills not only the nursing student's ability to perform a task but also, most importantly, the quality of nursing care.

Assessment of student nurses in practice: a comparison of theoretical and practical assessment results in England

This study was undertaken in response to concerns raised by Duffy (2003) that assessors of practice were reluctant to fail student nurses in assessments. This generated doubts about the fitness to practice of some registered nurses. An investigation was undertaken into whether quantitative evidence supported the view that pre-registration nurses rarely failed practical assessments. Comparative failure rates from theoretical and practical assessments were requested from all 52 universities in England that offered pre-registration nursing programmes. Responses were received from 27. Findings indicated that a very small proportion of students failed practical assessments; failure rates for theory outstripped practice by a ratio of 5:1. A quarter of universities failed no students in practice. Students were most likely to fail in year one and least likely in year three. This study supports the belief that assessors of practice are reluctant to fail student nurses. It raises a number of questions about the influence that the systems and practices of professional bodies and universities have on practical assessment. However it also indicates that some student nurses have failed practical assessments and that some universities do have systems in place to address this issue.

Facilitating pre-registration nurse learning: a mentor approach

The aim of this literature review is to highlight the need for effective learning and teaching facilitation in the clinical workplace, so that nurses are fit to practise on entry to the Nursing and Midwifery Council (NMC) register. The article focuses on the importance of the relationship between mentors and student nurses, and summarizes recommendations from the literature on this subject.