Exploring transformative learning when developing medical students' non-technical skills

Attitudes and Confidence in Communication Skills of Fourth-Year Medical Students After Online Small Group Discussion and Peer Role-Play: A Survey Study

OBJECTIVE

This study investigated the impact of online small group discussions and peer role-play on medical students' attitudes toward and confidence in communication skills.

METHODS

The study involved a 3-hour instructional design that integrated small group discussions and role-plays to enhance the communication abilities of fourth-year medical students. The data were obtained from students through a post-activity online survey. Likert scale responses were quantified as percentages. Students' narrative feedback on their learning achievements was subjected to content analysis. Codes and categories were agreed upon by investigators and reviewed by an external auditor.

RESULTS

A total of 151 medical students responded to the survey, representing a 47.94% response rate. The majority strongly agreed that good communication skills help physicians obtain reliable information (94.00%) and that practicing questioning skills is essential (92.1%). Content analysis revealed that nearly half (48.34%) considered that the learning activities fostered positive attitudes toward communication. These attitudinal improvements included increased empathy and emotional care (37.75%), recognition of communication as a trainable skill (10.57%), and recognition of the necessity of communication skills for physicians (8.61%). Additionally, most students (75.5%) reported increased confidence in their communication skills.

CONCLUSIONS

In alignment with transformative learning theory, online small group discussions combined with peer role-play may potentially enhance the attitudes and confidence of medical students in relation to communication skills.

Critical thinking pedagogical practices in medical education: a systematic review

Introduction

The development of critical thinking (CT) has been a universal goal in higher education. A systematic review of the literature was conducted to evaluate the effectiveness of currently used pedagogical practices to foster CT/ clinical reasoning (CR)/ clinical judgment (CJ) skills and/or dispositions in undergraduate medical students.

Methods

PubMed, Web of Science and Scopus databases were searched from January 2010 to April 2021 with a predefined Boolean expression.

Results

Of the 3221 articles originally identified, 33 articles were included by using PICOS methodology. From these, 21 (64%) reported CR pedagogical practices and 12 (36%) CT pedagogical practices.

Discussion

Overall, pedagogical practices such as cognitive/visual representation, simulation, literature exposure, test-enhancing and team-based learning, clinical case discussion, error-based learning, game-based learning seem to enhance CT/CR skills and/or dispositions. Further research is required to identify the optimal timing, duration and modality of pedagogical interventions for effectively foster CT/CR in medical education.

Handle with Care: Transformative Learning as Pedagogy in an Under-Resourced Health Care Context

Issue: A significant component of health professions education is focussed on students' exposure to the social determinants of health and the challenges that patients within the health care system face. An appropriate way to provide such exposure is through distributed clinical training. This usually entails students training in smaller groups along the continuum of care, away from tertiary academic hospitals. This also means students are away from their existing academic and social support systems. It is evident that knowledge and clinical skills alone are not sufficient to prepare students, they also need to be taught to critically reflect on how their own values and attitudes traverse their knowledge and skills to influence their practice as healthcare professionals. This process of critical reflection should aim to provide a transformative learning experience for students and requires active facilitation. In under-resourced health care contexts where clinicians responsible for student training are facing high patient load, lack of resources, inequitable health care services and high levels of burn-out, the facilitation of student learning may be compromised. Evidence: Clinical learning opportunities that are considered transformative, frequently challenge students' sense of self and sense of belonging. This experience can have detrimental effects if the processes of transformative learning pedagogy are not adequately facilitated. The provision of support staff, lecturers and clinical facilitators on the distributed training platform is challenged by the remote nature of some of the sites and the cost of recruiting and capacitating additional on-site staff. The potential for what has been termed "transformative trauma" and the subsequent halted transformative learning experience, has ethical implications in terms of student wellness and the educational responsibility institutions carry. Implications: The authors suggest considerations in facilitating an ethical transformative learning process. These include making the transformative learning pedagogy explicit to students and clinical facilitators and using the 'brave spaces' framework to help students with individuation and provide them with the tools to understand how emotion influences behavior. Strategies to improve relationship development and communities of support, as well as ideas for faculty development are offered.

Disorienting or Transforming? Using the Arts and Humanities to Foster Social Advocacy

Introduction

The arts and humanities (AH) have transformative potential in medical education. Research suggests that AH-based pedagogies may facilitate both personal and professional transformation in medical learners, which may then further enhance the teaching and learning of social advocacy skills. However, the potential for such curricula to advance social advocacy training remains under-explored. Therefore, we sought to identify how AH may facilitate transformative learning of social advocacy in medical education.

Methods

Building upon previous research, we conducted a critical narrative review seeking examples from the literature on how AH may promote transformative learning of social advocacy in North American medical education. Through a search of seven databases and MedEdPORTAL, we identified 11 articles and conducted both descriptive and interpretative analyses of their relation to key tenets of transformative learning, including: disorientation/dissonance, critical reflection, and discourse/dialogue.

Results

We found that AH are used in varied ways to foster transformative learning in social advocacy. However, most approaches emphasize their use to elicit disorientation and dissonance; there is less evidence in the literature regarding how they may be of potential utility when applied to disorienting dilemma, critical reflection, and discourse/dialogue.

Discussion

The tremendous potential of AH to foster transformative learning in social advocacy is constrained due to minimal attention to critical reflection and dialogue. Future research must consider how novel approaches that draw from AH may be used for more robust engagement with transformative learning tenets in medical education.

Narrative-based learning for person-centred healthcare: the Caring Stories learning framework

This paper describes the learning framework for an innovative narrative-based training platform for healthcare professionals based on older patients' narratives. The aim of Caring Stories is to place patients' desires and needs at the heart of healthcare and by doing so to promote person-centred care (PCC). It is argued that this narrative-based approach to training in healthcare education will provide professionals from different fields with competencies to better understand how to interpret the lifeworlds of older people, as well as facilitate better communication and navigation through increasingly complex care trajectories. The spiral learning framework supports narrative-based training to be accessible to a broad range of healthcare practitioners. We suggest this is a theoretically sophisticated methodology for training diverse healthcare professionals in PCC, alongside core tenets of narrative medicine, with applicability beyond the patient group it was designed for. The learning framework takes into account professionals' mindsets and draws on the epistemic tenets of pragmatism to support interprofessional education. Being informed by narrative pedagogy, narrative inquiry, and expansive learning and transformative learning theories, ensures that a robust pedagogical foundation underpins the learning framework. The paper sets out the conceptual ideas about narrative that we argue should be more widely understood in the broad body of work that draws on patient narratives in healthcare education, alongside the learning theories that best support this framing of narrative. We suggest that this conceptual framework has value with respect to helping to disseminate the ways in which narrative is most usefully conceptualised in healthcare education when we seek to foster routes to bring practitioners closer to the lifeworlds of their patients. This conceptual framework is therefore generic with respect to being a synthesis of the critical orientations to narrative that are important in healthcare education, then adaptable to different contexts with different patient narratives.

When I say… transformative learning

Transformation = change. Learning = change. So what do @VipsMDMEd & @APSawatskyMD mean when they say #TransformativeLearning?

Applied improvisation and transdisciplinary simulation: a necessity for any health curriculum?

From practising a procedure, such as a lumbar puncture, to explaining the aim and method and listening to concerns, the practice of health professionals requires a range of skills, often classified into technical and non-technical skills. Just as gestures and procedures can be taught, so can empathy and communication skills. This article introduces an innovative approach that unites both necessary types of skills. The specific framework of improvisational theatre ("improv") has widespread application, including the training of health professionals (health training improv). By sharing close contexts and skills, health training improv provides a valuable, safe, and effective learning environment that allows practitioners to practice exercises and situations that align with particular objectives. We created a transdisciplinary team to develop a programme of Health Professional Training Improv (HPTI), bringing together the fields of health, psychology, simulation, and arts. Since 2019, various health student groups (nurses, midwives, medical doctors, and speech therapists) have participated in a 16-h applied improv training workshop under the supervision of a professional improv facilitator. Additionally, drama students completed applied improv for health courses, which trained them to act as simulated patients, with a view to the implementation of transdisciplinary improv simulation sessions at SimUSanté (a multidisciplinary health simulation facility located in France). Students' feedback emphasized their interest in HPTI, the realism of the simulation sessions, and the skills they felt had improved. This feedback needs to be supplemented with quantitative data from standardised assessments. The development of this rich pedagogical and research framework, based on a transdisciplinary approach, has brought different fields together to prepare students for real patient encounters. It is essential to continue this training and conduct research to evaluate the curricula developed.

Beyond scientific and technical training: Assessing the relevance of empathy and assertiveness in future physiotherapists: A cross-sectional study

Background and Aims

Empathy and assertiveness are two essential social skills for a health professional such as a physiotherapist and are necessary for developing moral thinking. Previous studies show that the development of empathy and other social skills improves as students progress in their studies. However, other authors show deterioration of empathy as students progress in their studies and acquire clinical experience. Training in soft skills, such as assertiveness, among health science students will have an impact on the quality of patient care. Effective communication, conflict resolution and the ability to work as part of a team are competencies that have been put to one side as a result of the recent COVID-19 pandemic and it is important to resume training students in soft skills. The objective of this study is to investigate to determine the empathic and assertive state of physiotherapy university students.

Methods

A descriptive cross-sectional study of physiotherapy university students was conducted in the 2022/2023 academic year. The Interpersonal Reactivity Index (IRI) scales for empathy and the Rathus test for assertiveness (RAS) were used as study tools. Finally, 127 students participated in the study, 52.91% of the total population of physiotherapy students. The questionnaire was available for 4 weeks in November and December 2022.

Results

The empathetic and assertive development of the students was found to be acceptable. Significant differences were also observed according to the gender variable in the students, with female students presenting better results (p = 0.01). Students who are working or have clinical experience in other professions score lower on the empathy personal distress subscale (p < 0.001).

Conclusion

Future research should be considered to help improve clinical and professional expertise in physiotherapy students about empathic and assertive development. The findings provide new evidence on the levels of empathy and assertiveness in physiotherapy students.

A scoping review of emotions and related constructs in simulation-based education research articles

BACKGROUND

While acknowledgement of emotions' importance in simulation-based education is emerging, there are concerns regarding how education researchers understand the concept of emotions for them to deliberately incorporate emotionally charged scenarios into simulation-based education. This concern is highlighted especially in the context of medical education often lacking strong theoretical integration. To map out how current simulation-based education literature conceptualises emotion, we conducted a scoping review on how emotions and closely related constructs (e.g. stress, and emotional intelligence) are conceptualised in simulation-based education articles that feature medical students, residents, and fellows.

METHODS

The scoping review was based on articles published in the last decade identified through database searches (EMBASE and Medline) and hand-searched articles. Data extraction included the constructs featured in the articles, their definitions, instruments used, and the types of emotions captured. Only empirical articles were included (e.g. no review or opinion articles). Data were charted via descriptive analyses.

RESULTS

A total of 141 articles were reviewed. Stress was featured in 88 of the articles, while emotions and emotional intelligence were highlighted in 45 and 34 articles respectively. Conceptualisations of emotions lacked integration of theory. Measurements of emotions mostly relied on self-reports while stress was often measured via physiological and self-report measurements. Negative emotions such as anxiety were sometimes seen as interchangeable with the term stress. No inferences were made about specific emotions of participants from their emotional intelligence.

CONCLUSIONS

Our scoping review illustrates that learners in simulation-based education are most often anxious and fearful. However, this is partially due to medical education prioritising measuring negative emotions. Further theoretical integration when examining emotions and stress may help broaden the scope towards other kinds of emotions and better conceptualisations of their impact. We call for simulation education researchers to reflect on how they understand emotions, and whether their understanding may neglect any specific aspect of affective experiences their simulation participants may have.

"Asking for help is a strength"-how to promote undergraduate medical students' teamwork through simulation training and interprofessional faculty

The ability to team up and safely work in any kind of healthcare team is a critical asset and should be taught early on in medical education. Medical students should be given the chance to "walk the talk" of teamwork by training and reflecting in teams. Our goal was to design, implement and evaluate the feasibility of a simulation-based teamwork training (TeamSIM) for undergraduate medical students that puts generic teamwork skills centerstage. We designed TeamSIM to include 12 learning objectives. For this pre-post, mixed-methods feasibility study, third-year medical students, organized in teams of 11-12 students, participated and observed each other in eight simulations of different clinical situation with varying degrees of complexity (e.g., deteriorating patient in ward; trauma; resuscitation). Guided by an interprofessional clinical faculty with simulation-based instructor training, student teams reflected on their shared experience in structured team debriefings. Using published instruments, we measured (a) students' reactions to TeamSIM and their perceptions of psychological safety via self-report, (b) their ongoing reflections via experience sampling, and (c) their teamwork skills via behavior observation. Ninety four students participated. They reported positive reactions to TeamSIM (M = 5.23, SD = 0.5). Their mean initial reported level of psychological safety was M = 3.8 (SD = 0.4) which rose to M = 4.3 (SD = 0.5) toward the end of the course [T(21) = -2.8, 95% CI -0.78 to-0.12, p = 0.011 (two-tailed)]. We obtained n = 314 headline reflections from the students and n = 95 from the faculty. For the students, the most frequent theme assigned to their headlines involved the concepts taught in the course such as "10 s for 10 min." For the faculty, the most frequent theme assigned to their headlines were reflections on how their simulation session worked for the students. The faculty rated students' teamwork skills higher after the last compared to the first debriefing. Undergraduate medical students can learn crucial teamwork skills in simulations supported by an experienced faculty and with a high degree of psychological safety. Both students and faculty appreciate the learning possibilities of simulation. At the same time, this learning can be challenging, intense and overwhelming. It takes a team to teach teamwork.

Do Moral Judgments in Moral Dilemmas Make One More Inclined to Choose a Medical Degree?

The role of moral intuitions and moral judgments has become increasingly prominent in educational and academic choices. The present research aims to examine if the moral judgments elicited in sacrificial trolley dilemmas have a distinct pattern for the decisions made by junior medical students, in comparison to those of senior high school students. We work with this sample because it represents the population out of which medical students are recruited in the case of Bucharest, Romania. Our findings show that moral judgments are indeed a significant predictor for a respondent's status as medical students. This result, albeit with limitations, bears multiple practical implications, from developing empirically informed medical ethics courses in medical schools to evidence-based policy designs which consider factors such as morality alongside financial outcomes and incentives.

The nationwide impact of COVID-19 on life support courses. A retrospective evaluation by Resuscitation Council UK

Aim

To determine the impact of the COVID-19 pandemic on Resuscitation Council UK Advanced Life Support (ALS) and Immediate Life Support (ILS) course numbers and outcomes.

Methods

We conducted a before-after study using course data from the Resuscitation Council UK Learning Management System between January 2018 and December 2021, using 23 March 2020 as the cut-off between pre- and post-pandemic periods. Demographics and outcomes were analysed using chi-squared tests and regression models.

Results

There were 90,265 ALS participants (51,464 pre-; 38,801 post-) and 368,140 ILS participants (225,628 pre-; 142,512 post-). There was a sharp decline in participants on ALS/ILS courses due to COVID-19. ALS participant numbers rebounded to exceed pre-pandemic levels, whereas ILS numbers recovered to a lesser degree with increased uptake of e-learning versions. Mean ALS course participants reduced from 20.0 to 14.8 post-pandemic (P < 0.001).Post-pandemic there were small but statistically significant decreases in ALS Cardiac Arrest Simulation Test pass rates (from 82.1 % to 80.1 % (OR = 0.90, 95 % CI = 0.86-0.94, P < 0.001)), ALS MCQ score (from 86.6 % to 86.0 % (mean difference = -0.35, 95 % CI -0.44 to -0.26, P < 0.001)), and overall ALS course results (from 95.2 %to 94.7 %, OR = 0.92, CI = 0.85-0.99, P = 0.023). ILS course outcomes were similar post-pandemic (from 99.4 % to 99.4 %, P = 0.037).

Conclusion

COVID-19 caused a sharp decline in the number of participants on ALS/ILS courses and an accelerated uptake of e-learning versions, with the average ALS course size reducing significantly. The small reduction in performance on ALS courses requires further research to clarify the contributing factors.

Online non-technical skills faculty training

BACKGROUND

The Medical Students Non-Technical Skills (Medi-StuNTS) system is a behavioural marker system (BMS) designed to identify and debrief non-technical skills (NTS) for medical students during immersive simulation. Educators must be adequately trained in using the BMS. This study aimed to design and implement an online platform to deliver a faculty development course on using Medi-StuNTS and evaluate the feasibility of this platform in training faculty to identify and debrief NTS.

APPROACH

The online platform was developed by faculty with expertise in NTS, based on guidance for faculty training programme requirements and the multimodal model for online education. Content was arranged in modules, using presentations, videos of simulation scenarios and interactive discussion boards.

EVALUATION

Fifteen participants completed the course and feedback over a two-month period. A feedback form was completed to assess feasibility, based on a feasibility framework. The areas of focus were acceptability, demand, implementation, practicality, adaptation, integration, expansion and limited efficacy. Feedback indicated that the course shows promise in improving the ability of faculty to identify and debrief NTS.

IMPLICATIONS

The platform was successfully developed and implemented and was able to reach a national audience due to its online nature. Specific strengths include increased flexibility and accessibility compared to in-person training. Feasibility assessment suggests that this newly developed online platform can work as an effective method for faculty development in order to increase skills in identifying and debriefing NTS using Medi-StuNTS. Future work will focus on expansion of the online platform and dissemination to an international audience.

Professional Skills Teaching within Veterinary Education and Possible Future Directions

Once ignored for their potential to take up precious time within the veterinary curriculum at the expense of hard science and technical competencies, professional skills such as ethical conduct, professional conduct, and communication skills are now considered essential in the creation of successful and employable graduates. Despite the requirement to include professional skills in veterinary curricula, limited communication among colleges and inconsistent documentation of curricular developments have led to a wide range of teaching and assessment methods with no consistent standards existing among colleges. Integration of professional competency teaching into the general curriculum is lauded widely, but barriers such as faculty buy-in have kept many colleges from moving toward a standard in which professional competencies are integrated into the general curriculum. The aim of this article is to provide veterinary educators and curriculum designers with an understanding of the rationale for including professional skills teaching within the curriculum while also presenting currently used, as well as recommended, strategies for effective instruction of professional skills.

Emotion as reflexive practice: A new discourse for feedback practice and research

INTRODUCTION

Like medicine and health care, feedback is a practice imbued with emotions: saturated with feelings relevant to one's identity and status within a given context. Often this emotional dimension of feedback is cast as an impediment to be ignored or managed. Such a perspective can be detrimental to feedback practices as emotions are fundamentally entwined with learning. In this critical review, we ask: What are the discourses of emotion in the feedback literature and what 'work' do they do?

METHODS

We conducted a critical literature review of emotion and feedback in the three top journals of the field: Academic Medicine, Medical Education and Advances in Health Sciences Education. Analysis was informed by a Foucauldian critical discourse approach and involved identifying discourses of emotion and interpreting how they shape feedback practices.

FINDINGS

Of 32 papers, four overlapping discourses of emotion were identified. Emotion as physiological casts emotion as internal, biological, ever-present, immutable and often problematic. Emotion as skill positions emotion as internal, mainly cognitive and amenable to regulation. A discourse of emotion as reflexive practice infers a social and interpersonal understanding of emotions, whereas emotion as socio-cultural discourse extends the reflexive practice discourse seeing emotion as circulating within learning environments as a political force.

DISCUSSION

Drawing on scholarship within the sociology of emotions, we suggest the merits of studying emotion as inevitable (not pathological), as potentially paralysing and motivating and as situated within (and often reinforcing) a hierarchical social health care landscape. For future feedback research, we suggest shifting towards recognising the discourse-theory-practice connection with emotion in health professional education drawing from reflexive and socio-cultural discourses of emotion.

How can we better prepare new doctors for the tasks and challenges of ward rounds?: An observational study of junior doctors' experiences

PURPOSE

Ward rounds play a crucial role in the delivery of patient care in inpatient settings, but involve a complex mix of tasks, skills and challenges for junior doctors to negotiate. This study informs the development of high-quality training by identifying the activities that junior doctors perform, and those associated with stress during real-life ward rounds.

MATERIALS AND METHODS

All activities performed by FY1 doctors (n = 60) over 2 ward rounds were coded in real-time by a trained observer using the work observation method by activity timing (WOMBAT). Doctors' heart rate was continuously recorded and non-metabolic peaks in heart rate used as a physiological indicator of stress.

RESULTS

During ward rounds, FY1 doctors commonly engaged in indirect patient care, professional communication, documentation and observation. Very little time was spent on direct patient care (6%) or explicit supervision/education (0.01%). Heart rate data indicated that stress was highest during administrative tasks while interacting directly with patients while stepping out of rounds to complete personal tasks, when answering bleeps and while multi-tasking.

CONCLUSIONS

Training that specifically covers the activities involved, skills required, and challenges inherent in real-life ward rounds may better prepare FY doctors for this complex area of practice.

Exploring transformative learning for trainee pharmacists through interprofessional simulation: a constructivist interview study

BACKGROUND

The expanding roles of UK pharmacists have prompted substantial changes to the initial pharmacy education and training, including increasing recognition of the value of learning alongside other professional groups in acute settings. Interprofessional immersive simulation training appears to represent a useful educational tool to meet the evolving needs of the profession, but the impact of such training on workplace behaviour and relationships has not been explored. This study aimed to explore how interprofessional simulation training facilitates transformative learning in pre-registration pharmacists.

METHODS

Across three different locations in Scotland, pre-registration pharmacists were paired with medical students to participate in immersive simulation scenarios with post-scenario debriefs. Pre-registration pharmacists were individually interviewed shortly after their simulation session, using a semi-structured interview schedule based on the transformative learning framework. Transcripts were analysed using template analysis, with Mezirow's phases of perspective transformation forming the initial coding template.

RESULTS

Fifteen interviews following five simulation sessions at three different sites were undertaken. Phases 1-6 of the transformative learning framework all resonated with the pre-registration pharmacists to varying degrees. Two prominent threads became evident in the data: a change in participants' perceptions of risk, and deepened understanding of their role within an acute context. These themes were woven throughout phases 2-6 of the transformative learning framework.

CONCLUSIONS

Interprofessional immersive simulation training involving acute clinical scenarios has been found to be helpful for pre-registration pharmacists and can foster transformative learning. Through this powerful process, they developed new ways to see the world, themselves and their professional relationships. Positive future actions and roles were planned. As the patient-facing roles of pharmacists expand, educational practices that translate into meaningful change to workplace behaviour and relationships become increasingly important. Carefully constructed interprofessional immersive simulation training should be utilised within pharmacy education more widely.

Reliability of assessment of medical students' non-technical skills using a behavioural marker system: does clinical experience matter?

Introduction

Non-technical skills are recognised to play an integral part in safe and effective patient care. Medi-StuNTS (Medical Students' Non-Technical Skills) is a behavioural marker system developed to enable assessment of medical students' non-technical skills. This study aimed to assess whether newly trained raters with high levels of clinical experience could achieve reliability coefficients of >0.7 and to compare differences in inter-rater reliability of raters with varying clinical experience.

Methods

Forty-four raters attended a workshop on Medi-StuNTS before independently rating three videos of medical students participating in immersive simulation scenarios. Data were grouped by raters' levels of clinical experience. Inter-rater reliability was assessed by calculating intraclass correlation coefficients (ICC).

Results

Eleven raters with more than 10 years of clinical experience achieved single-measure ICC of 0.37 and average-measures ICC of 0.87. Fourteen raters with more than or equal to 5 years and less than 10 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC of 0.59. Nineteen raters with less than 5 years of clinical experience achieved single-measure ICC of 0.09 and average-measures ICC 0.65.

Conclusions

Using 11 newly trained raters with high levels of clinical experience produced highly reliable ratings that surpassed the prespecified inter-rater reliability standard; however, a single rater from this group would not achieve sufficiently reliable ratings. This is consistent with previous studies using other medical behavioural marker systems. This study demonstrated a decrease in inter-rater reliability of raters with lower levels of clinical experience, suggesting caution when using this population as raters for assessment of non-technical skills.

Validity of the Medi-StuNTS behavioural marker system: assessing the non-technical skills of medical students during immersive simulation

Background

The Medical Students' Non-Technical Skills (Medi-StuNTS) behavioural marker system (BMS) is the first BMS to be developed specifically for medical students to facilitate training in non-technical skills (NTS) within immersive simulated acute care scenarios. In order to begin implementing the tool in practice, validity evidence must be sought. We aimed to assess the validity of the Medi-StuNTS system with reference to Messick's contemporary validity framework.

Methods

Two raters marked video-recorded performances of acute care simulation scenarios using the Medi-StuNTS system. Three groups were marked: third-year and fourth-year medical students (novices), final-year medical students (intermediates) and core medical trainees (experts). The scores were used to make assessments of relationships to the variable of clinical experience through expert-novice comparisons, inter-rater reliability, observability, exploratory factor analysis, inter-rater disagreements and differential item functioning.

Results

A significant difference was found between the three groups (p<0.005), with experts scoring significantly better than intermediates (p<0.005) and intermediates scoring significantly better than novices (p=0.001). There was a strong positive correlation between the two raters' scores (r=0.79), and an inter-rater disagreement of more than one point in less than one-fifth of cases. Across all scenarios, 99.7% of skill categories and 84% of skill elements were observable. Factor analysis demonstrated appropriate grouping of skill elements. Inconsistencies in test performance across learner groups were shown specifically in the skill categories of situation awareness and decision making and prioritisation.

Conclusion

We have demonstrated evidence for several aspects of validity of the Medi-StuNTS system when assessing medical students' NTS during immersive simulation. We can now begin to introduce this system into simulation-based education to maximise NTS training in this group.