The gap in transfer research

Development and validation of a simulation-based assessment tool in colonoscopy

BACKGROUND

Colonoscopy is difficult to learn. Virtual reality simulation training is helpful, but how and when novices should progress to patient-based training has yet to be established. To date, there is no assessment tool for credentialing novice endoscopists prior to clinical practice. The aim of this study was to develop such an assessment tool based on metrics provided by the simulator. The metrics used for the assessment tool should be able to discriminate between novices, intermediates, and experts and include essential checklist items for patient safety.

METHODS

The validation process was conducted based on the Standards for Educational and Psychological Testing. An expert panel decided upon three essential checklist items for patient safety based on Lawshe's method: perforation, hazardous tension to the bowel wall, and cecal intubation. A power calculation was performed. In this study, the Simbionix GI Mentor II simulator was used. Metrics with discriminatory ability were identified with variance analysis and combined to form an aggregate score. Based on this score and the essential items, pass/fail standards were set and reliability was tested.

RESULTS

Twenty-four participants (eight novices, eight intermediates, and eight expert endoscopists) performed two simulated colonoscopies. Four metrics with discriminatory ability were identified. The aggregate score ranged from 4.2 to 51.2 points. Novices had a mean score of 10.00 (SD 5.13), intermediates 24.63 (SD 7.91), and experts 30.72 (SD 11.98). The difference in score between novices and the other two groups was statistically significant (p<0.01). Although expert endoscopists had a higher score, the difference was not statistically significant (p=0.40). Reliability was good (Cronbach's alpha=0.86). A pass/fail score was defined at 17.1 points with correct completion of three essential checklist items, resulting in three experts and three intermediates failing and one novice passing the assessment.

CONCLUSION

We established a valid and reliable assessment tool with a pass/fail standard on the simulator. We suggest using the assessment after simulation-based training before commencing work-based learning.

Connecting Biochemistry Knowledge to Patient Care in the Clinical Workplace: Senior Medical Students' Perceptions about Facilitators and Barriers

Phenomenon: Medical students have difficulties applying knowledge about biomedical mechanisms learned before clerkships to patient care activities. Many studies frame this challenge as a problem of basic science knowledge transfer predominantly influenced by students' individual cognitive processes. Social cognitive theory would support extending this framing to the interplay between the individual's cognition, the environment, and their behaviors. This study investigates senior medical students' experiences of biochemistry knowledge use during workplace learning and examines how their experiences were influenced by interactions with people and other elements of the clinical learning environment. Approach: The authors used a qualitative approach with a constructivist orientation. From September to November 2020 they conducted semi-structured interviews with 11 fourth-year medical students at one institution who had completed the pre-clerkship curriculum, core clinical clerkships, and the United States Medical Licensing Exam Step 1. The authors identified themes using thematic analysis. Findings: Participants reported that they infrequently used or connected to biochemistry knowledge in workplace patient care activities, yet all had examples of such connections that they found valuable to learning. Most participants felt the responsibility for making connections between biochemistry knowledge and activities in the clinical workplace should be shared between themselves and supervisors, but connections were often recognized and acted on only by the student. Connections that participants described prompted their effort to retrieve knowledge or fill a perceived learning gap. Participants identified multiple barriers and facilitators to connecting, including supervisors' behaviors and perceived knowledge, and "patients seen" in clerkships. Participants also reported learning biochemistry during USMLE Step 1 study that did not connect to patient care activities, underscoring a perception of disconnect. Insights: This study identifies specific personal, social, and physical environmental elements that influence students' perceived use of biochemistry during patient care activities. Though these findings may be most significant for biochemistry, they likely extend to other basic science disciplines. Students' self-directed efforts to connect to their biochemistry knowledge could be augmented by increased social support from clinical supervisors, which in turn likely requires faculty development. Opportunities for connection could be enhanced by embedding into the environment instructional strategies or technologies that build on known authentic connections between biochemistry and "patients seen" in clerkships. These efforts could strengthen student learning, improve clinical supervisors' self-efficacy, and better inform curriculum design.

Using Immersive Virtual Reality Simulation to Ensure Competence in Contrast-Enhanced Ultrasound

Contrast-enhanced ultrasound (CEUS) is used in various medical specialties as a diagnostic imaging tool and for procedural guidance. Experience in the procedure is currently attained via supervised clinical practice that is challenged by patient availability and risks. Prior simulation-based training and subsequent assessment could improve and ensure competence before performance on patients, but no simulator currently exists. Immersive virtual reality (IVR) is a new promising simulation tool that can replicate complex interactions and environments that are unfeasible to achieve by traditional simulators. This study was aimed at developing an IVR simulation-based test for core CEUS competencies and gathering validity evidence for the test in accordance with Messick's framework. The test was developed by IVR software specialists and clinical experts in CEUS and medical education and imitated a CEUS examination of a patient with a focal liver lesion with emphasis on the pre-contrast preparations. Twenty-five medical doctors with varying CEUS experience were recruited as test participants, and their results were used to analyze test quality and to establish a pass/fail standard. The final test of 23 test items had good internal reliability (Cronbach's α = 0.85) and discriminatory abilities. The risks of false positives and negatives (9.1% and 23.6%, respectively) were acceptable for the test to be used as a certification tool prior to supervised clinical training in CEUS.

Contextual Competence: How residents develop competent performance in new settings

INTRODUCTION

Medical education continues to diversify its settings. For postgraduate trainees, moving across diverse settings, especially community-based rotations, can be challenging personally and professionally. Competent performance is embedded in context; as a result, trainees who move to new contexts are challenged to use their knowledge, skills and experience to adjust. What trainees need to adapt to and what that requires of them are poorly understood. This research takes a capability approach to understand how trainees entering a new setting develop awareness of specific contextual changes that they need to navigate and learn from.

METHODS

We used constructivist grounded theory with in-depth interviews. A total of 29 trainees and recent graduates from three internal medicine training programmes in Canada participated. All participants had completed at least one community-based rotation geographically far from their home training site. Interviews were recorded, transcribed and anonymised. The interview framework was adjusted several times following initial data analysis.

RESULTS

Contextual competence results from trainees' ability to attend to five key stages. Participants had first to meet their physiological and practical needs, followed by developing a sense of belonging and legitimacy, which paved the way for a re-constitution of competence and appropriate autonomy. Trainee's attention to these stages of adaptation was facilitated by a process of continuously moving between using their knowledge and skill foundation and recognising where and when contextual differences required new learning and adaptations.

DISCUSSION

An ability to recognise contextual change and adapt accordingly is part of Nussbaum and Sen's concept of capability development. We argue this key skill has not received the attention it deserves in current training models and in the support postgraduate trainees receive in practice. Recommendations include supporting residents in their capability development by debriefing their experiences of moving between settings and supporting clinical teachers as they actively coach residents through this process.

Transfer of learning from simulated setting to the clinical setting: identifying instructional design features

Background: Transfer of learning (ToL) is the endpoint of simulation-based training (SBT). It is affected by numerous factors, which can be classified into 3 categories: learner characteristics, work environment, and training design. The first 2 have been identified to some extent in previous research. In this study, the aim was to identify the instructional design (ID) features affecting the ToL in SBT. Methods: This qualitative study was conducted in 2 phases. Phase 1 covers thematic analysis of comparative studies in the field of SBT. A systematic search was performed on 6 databases of Ovid MEDLINE, EMBASE, PsycINFO, CENTRAL, Scopus, and Web of Science, and the references of related systematic reviews were also checked. In phase 2, semi-structured interviews were conducted with key informants (instructors and learners) and analyzed using directed content analysis. The results of the 2 phases were combined, and finally ID features of SBT were identified and categorized. Results: In the first phase, 121 comparative studies were reviewed and in the second phase, 17 key informants were interviewed. After combining the results of the phases, the ID features affecting the ToL in SBT were classified into 3 broad categories and 15 subcategories as follows: (1) presimulation: preparation, briefing, and teaching cognitive base; (2) underlying theories: deliberate practice, mastery learning, and proficiency-based training; (3) and methods and techniques: distributed practice, variability, increasing complexity, opportunity for practice, repetitive practice, active learning, feedback/debriefing, simulator type, and simulator fidelity. Conclusion: Although learning is transferred from the simulated setting to the clinical setting, this process is not automatic and straightforward. Numerous factors affect this transfer. The results of this research can be used in designing and evaluating the SBT programs.

The myth of ivory tower versus practice-oriented research: A systematic review of randomised studies in medical education

CONTEXT

A long-standing myth in medical education research is a divide between two different poles: research aiming to advance theory with little focus on practical applications ('ivory tower' research) and practically oriented research aiming to serve educators and decision-makers with little focus on advancing theory ('in-the-trenches' practice). We explored this myth in a sample of randomised medical education studies using Stokes' four-quadrant framework for the classification of research perspective.

METHODS

We searched MEDLINE, Embase, CINAHL, PsychINFO, ERIC, Web of Science and Scopus for studies in medical education using a randomised design that were published between 1 January 2018 and 31 December 2018. We used Stokes' four-quadrant framework to categorise the studies according to their use of theory, concepts and their justification for practical use. We compared medical education research published in medical education journals and clinical journals.

RESULTS

A total of 150 randomised studies were included in the analysis. The largest segment of studies (46.7%) was categorised as use-inspired basic research (Pasteur's Quadrant), closely followed by pure applied research (40.7%, Edison's Quadrant). Only a few studies were categorised as aiming to advance knowledge with no thought for practical educational application (2.0%, Bohr's Quadrant). The proportion of studies that included educational concepts and theory differed according to publication in clinical journals or medical education journals: 40.5% vs 71.8%, respectively, P < .001. There were no differences between journals with regard to the proportion of studies that included a practical educational or clinical rationale (P = .99).

CONCLUSION

In a large sample of studies using randomised designs, we found no evidence to support the myth that medical education research divides between two singular poles represented by 'ivory tower research' and 'in-the-trenches practice'. We did confirm prevailing assumptions regarding an emphasis on non-theoretical medical education research in clinical journals.

Interdisciplinary expertise in medical practice: Challenges of using and producing knowledge in complex problem-solving

Purpose: Clarification of interdisciplinary expertise as the ability to deal with the cognitive and epistemological challenges of multi- and interdisciplinary problem-solving-such as in developing and implementing medical technology for diagnoses and treatment of patients in collaborations between clinicians, technicians, and engineers-and of the higher-order cognitive skills needed as part of this expertise. Method: Clarify the epistemological difficulties of combining scientific knowledge, methodologies and technologies from different disciplines in problem-solving, by drawing on recent developments in the philosophy of science. Conclusion: We argue that interdisciplinary expertise involves the cognitive ability to connect, translate and establish links between disciplinary knowledge, as well as the metacognitive ability to understand and explain the role of the disciplinary perspective-consisting of, e.g. basic concepts, theories, models, methodologies, technologies, and specific ways of measuring, reasoning and modeling in a discipline-in how knowledge is used and produced.

Advice for authors from the editors of Perspectives on Medical Education : Getting your research published

None