Knowing How and Knowing Why: testing the effect of instruction designed for cognitive integration on procedural skills transfer

Going beyond the comparison: toward experimental instructional design research with impact

To design effective instruction, educators need to know what design strategies are generally effective and why these strategies work, based on the mechanisms through which they operate. Experimental comparison studies, which compare one instructional design against another, can generate much needed evidence in support of effective design strategies. However, experimental comparison studies are often not equipped to generate evidence regarding the mechanisms through which strategies operate. Therefore, simply conducting experimental comparison studies may not provide educators with all the information they need to design more effective instruction. To generate evidence for the what and the why of design strategies, we advocate for researchers to conduct experimental comparison studies that include mediation or moderation analyses, which can illuminate the mechanisms through which design strategies operate. The purpose of this article is to provide a conceptual overview of mediation and moderation analyses for researchers who conduct experimental comparison studies in instructional design. While these statistical techniques add complexity to study design and analysis, they hold great promise for providing educators with more powerful information upon which to base their instructional design decisions. Using two real-world examples from our own work, we describe the structure of mediation and moderation analyses, emphasizing the need to control for confounding even in the context of experimental studies. We also discuss the importance of using learning theories to help identify mediating or moderating variables to test.

Using a boundary crossing lens to understand basic science educator and clinical educator collaboration in instructional design

PURPOSE

Collaborations between basic science educators (BE) and clinical educators (CE) in medical education are common and necessary to create integrated learning materials. However, few studies describe experiences of or processes used by educators engaged in interdisciplinary teamwork. We use the lens of boundary crossing to explore processes described by BE and CE that support the co-creation of integrated learning materials, and the impact that this work has on them.

MATERIALS AND METHODS

We conducted qualitative content analysis on program evaluation data from 27 BE and CE who worked on 12 teams as part of a multi-institutional instructional design project.

RESULTS

BE and CE productively engaged in collaboration using boundary crossing mechanisms. These included respecting diverse perspectives and expertise and finding efficient processes for completing shared work that allow BE and CE to build on each other's contributions. BE and CE developed confidence in connecting clinical concepts with causal explanations, and willingness to engage in and support such collaborations at their own institutions.

CONCLUSIONS

BE and CE report the use of boundary crossing mechanisms that support collaboration in instructional design. Such practices could be harnessed in future collaborations between BE and CE.

Integrating basic sciences into clerkship rotation utilizing Kern's six-step model of instructional design: lessons learned

BACKGROUND

It is generally agreed that basic and clinical sciences should be integrated throughout the undergraduate medical education, however, there is still need for continued formal integration of basic sciences into clinical clerkship in many medical schools across the globe.

METHODS

Utilizing Kern's Six-Step Model of Instructional Design, we aimed to develop an intervention that would facilitate cognitive integration of basic and clinical sciences. After problem identification and targeted needs assessment through focused group discussion with the students and faculty, objectives were devised with an implementation plan of using flipped class approach to develop a content-focused and learner-centered teaching strategy. This intervention was piloted in the 2-week cardiology clerkship in Year 5. Evaluation of the content, integration, student and faculty experiences were recorded through in-depth interviews, FGDs and a formative MCQ test.

RESULTS

Flipped classroom based integrated sessions were successfully developed. The implementation phase was met with challenges that primarily stemmed from the diverse teaching styles among faculty members, hesitance to deviate from conventional practices, variations in clinic timings, and demanding schedules. Noteworthy observations were in terms of ownership of the project, the need for faculty development in modern student-centered teaching pedagogies, opportunities for content improvement, scheduling of sessions, and suggestion of revisiting fundamental concepts in basic sciences through a brief boot camp-style session at the onset of the clerkship. The role of flipped case model and clinical cases in integrating basic sciences into clinical sciences were appreciated by the students. Standardization in teaching practices was identified as the major challenge by the faculty.

CONCLUSIONS

A functional, learner-centered framework of cognitive integration of basic sciences in clinical sciences curriculum of cardiology rotation was developed with a potential to be implemented in other clerkship rotations.

Use of a Novel Three-Dimensional Model to Teach Ultrasound-guided Subclavian Vein Cannulation

Background

Central venous cannulation is an essential skill in perioperative and critical care medicine. Ultrasound guidance is the standard of care for femoral and internal jugular vein access, with the subclavian vein being perceived to be less amenable to ultrasound-guided (UG) insertion, resulting in a lack of procedural competency and low cannulation rate. There is a paucity of resources and a lack of experience among staff physicians to effectively instruct trainees. Simulation-based medical education has the potential to help maintain high-stakes, infrequently performed skills and counteract possible unrecognized skill decline. We aimed to create a novel, low-cost, high-fidelity three-dimensional (3D) model for UG subclavian vein (UG-SCV) access with an accompanying curriculum to improve this important skill.

Methods

A curriculum was created consisting of preparatory material reviewing UG-SCV access, followed by an in-person didactic lecture focusing on ultrasound use and management of complications and a deliberate practice session scanning volunteers and practicing UG vascular puncture on a 3D model. A qualitative usability test design was used to assess the validity of the curriculum in trainees with advanced vascular access skills (anesthesiologists). Participants were second-year anesthesia residents, anesthesia fellows, and staff physicians. Focus groups conducted after each session explored the face validity of the model and curriculum. By applying a usability design, the curriculum was optimized and finalized.

Results

Between September 2020 and February 2021, 28 participants tested the curriculum. The focus groups ensured that the curriculum achieved its objective, with iterative changes made after each session in a quality improvement framework Plan-Do-Study-Act approach. After the third cycle, minimal changes were suggested, and the curriculum and 3D model were finalized. An additional group of participants was used to ensure that no new input would help improve the curriculum further.

Conclusions

A focused curriculum for enhancing skills in UG-SCV cannulation using a novel 3D model was successfully implemented and validated through a usability test design. This curriculum is better targeted for practitioners experienced in central venous access to master a subclavian approach and maintain their skill level.

Adaptive Expertise in Undergraduate Pharmacy Education

Pharmacy educators are grappling with concerns around curriculum overload and core pharmacist competencies in a rapidly changing and increasingly complex healthcare landscape. Adaptive expertise provides a conceptual framework to guide educators as they design instructional activities that can support students on their journey towards becoming pharmacists who can perform procedural tasks efficiently, as well as creatively handle new and difficult-to-anticipate problems that arise regularly in pharmacy practice. This article explores undergraduate pharmacy education through a cognitive psychology lens and foregrounds three instructional design strategies which support the development of adaptive expertise: (1) cognitive integration, (2) productive failure, and (3) inventing with contrasting cases. These three evidence-based strategies cultivate long-term learning and provide a practical mechanism to combat curriculum overload and backwards-facing assessments. Pharmacy education can encourage the development of procedural and conceptual knowledge and position pharmacy students to excel as they move into more complicated and ambiguous roles in our healthcare system.

An Assessment of Upper-Extremity Surgical Technique Videos and Their Relationship With Procedural Learning Theory

PURPOSE

Our purpose was to ascertain how well award-winning and highly viewed upper-extremity surgical videos meet the needs of users and adhere to procedural learning theory. We hypothesized that upper-extremity videos hosted on academic society websites meet user needs better than upper-extremity videos hosted on a commercial website.

METHODS

Twenty-five upper-extremity videos were evaluated by 3 reviewers. A standardized scoring sheet was used to assess each video's content, production quality, and adequacy. Video lengths were compared. The inclusion frequencies of specific content categories, the adequacy of content, and meeting certain production standards, all of which assess consistency with procedural learning theory, were reported, stratified by video host. Associations between the video host and video content, production quality, and adequacy were assessed.

RESULTS

The median lengths of academically hosted and commercially hosted videos were similar. Regardless of the video host, no video contained information in all content categories. Sixty percent of the scored categories were present in less than 75% of evaluated videos. Academically hosted videos contained scored content more frequently than commercially hosted videos in 68.4% of categories. There were significant associations between academic hosts and inclusion of a case presentation, surgical indications, outcomes literature, a preoperative examination, follow-up visit intervals, and alternative surgical techniques. Overall, academically hosted videos had a higher percentage of adequate content categories compared with commercially hosted videos.

CONCLUSIONS

Videos on academic websites more consistently meet users' content needs and production expectations, as informed by procedural learning theory, while having higher rates of adequate content compared with videos on commercial websites.

CLINICAL RELEVANCE

While academically hosted videos appear to more consistently adhere to the tenets of procedural learning theory, opportunity exists for video creators to more consistently apply procedural learning theory, allowing for the creation of even more educationally beneficial online surgical videos.

Use of Videos by Health Care Professionals for Procedure Support in Acute Cardiac Care: A Scoping Review

BACKGROUND

Cardiology procedures are often life-saving and time-critical, but some are so infrequent that health care staff may have rarely encountered them in practice or need to refresh their skills rapidly. Videos demonstrating procedures have the potential to assist health care professionals and support safe patient care. This scoping review explores the research literature involving the use of video by health care professionals in hospitals.

AIM

To identify what is known from research regarding the use of video to support clinical procedures in hospitals or health care facilities.

METHOD

The Joanna Briggs Institute Scoping review methodology guided our systematic search of peer-reviewed evidence related to video use to support procedures in a hospital or health care facility. Data sources included the electronic databases: ProQuest, Cumulative Index of Nursing and Allied Health Literature (CINAHL), Medline, Excerpta Medica dataBASE (EMBASE), Scopus, and PubMed.

FINDINGS

Seventeen (17) studies that met the inclusion criteria were included in the review, all published between 2012 and 2022. Since 2005, YouTube has become the dominant platform for publishing or sourcing videos related to clinical procedures. Studies to date can be summarised under five themes: 1) video content and purpose, 2) target audience, 3) video hosting site-internal websites versus YouTube, 4) curated versus original 'homegrown' video content, and 5) video development process.

DISCUSSION/CONCLUSION

Research on the development and utility of videos to support clinical procedures is emerging, with the ability to host videos on platforms such as YouTube becoming more accessible in recent years. All videos were designed to enhance health care professionals' existing knowledge and skills within their scope of practice. The available literature suggests that video can be a valuable clinical resource for both simple and skilled procedures. Video resources can help clinicians perform or assist with rare procedures, providing 'just in time' patient safety prompts and information.

The Sequence of Steps: A Key Concept Missing in Surgical Training-A Systematic Review and Recommendations to Include It

BACKGROUND

Surgical procedures have an inherent feature, which is the sequence of steps. Moreover, studies have shown variability in surgeons' performances, which is valuable to expose residents to different ways to perform a procedure. However, it is unclear how to include the sequence of steps in training programs.

METHODS

We conducted a systematic review, including studies reporting explicit teaching of a standard sequence of steps, where assessment considered adherence to a standard sequence, and where faculty or students at any level participated. We searched for articles on PubMed, EMBASE, CINAHL, Web of Science, and Google Scholar databases.

RESULTS

We selected nine articles that met the inclusion criteria. The main strategy to teach the sequence was to use videos to demonstrate the procedure. The simulation was the main strategy to assess the learning of the sequence of steps. Non-standardized scoring protocols and written tests with variable validity evidence were the instruments used to assess the learning, and were focused on adherence to a standard sequence and the omission of steps.

CONCLUSIONS

Teaching and learning assessment of a standard sequence of steps is scarcely reported in procedural skills training literature. More research is needed to evaluate whether the new strategies to teach and assess the order of steps work. We recommend the use of Surgical Process Models and Surgical Data Science to incorporate the sequence of steps when teaching and assessing procedural skills.

Beyond the tensions within transfer theories: implications for adaptive expertise in the health professions

Ensuring trainees develop the flexibility with their knowledge to address novel problems, and to efficiently build upon prior knowledge to learn new knowledge is a common goal in health profession education. How trainees come to develop this capacity to transfer and transform knowledge across contexts can be described by adaptive expertise, which focuses on the ability of some experts to innovate upon their existing knowledge to develop novel solutions to novel problems. While adaptive expertise is often presented as an alternative framework to more traditional cognitivist and constructivist expertise models, it is unclear whether the non-routine and routine forms of transfer it describes are distinct from those described by other accounts of transfer. Furthermore, whether what (e.g., knowledge) is transferred and how (e.g., cognitive processes) differs between these views is still debated. In this review, we describe various theories of transfer and present a synthesis clarifying the relationship between transfer and adaptive expertise. Informed by our analysis, we argue that the mechanisms of transfer in adaptive expertise share important commonalities with traditional accounts of transfer, which when understood, can complement efforts by educators and researchers to foster and study adaptive expertise. We present three instructional principles that may better support transfer and adaptive expertise in trainees: i) identifying and incorporating meaningful variability in practice, ii) integrating conceptual knowledge during practice iii) using assessments of trainees' transfer. Taken together, we offer an integrative perspective to how educational systems and experiences can be designed to develop and encourage adaptive expertise and transfer.

Re-examining the integration of routine and adaptive expertise: there is no such thing as routine from a motor control perspective

The study of adaptive expertise in health professions education has focused almost exclusively on cognitive skills, largely ignoring the processes of adaptation in the performance of precision technical skills. We present a focused review of literature to argue that repetitive practice is much less repetitive than often perceived. Our main thesis is that all skilled movement reflects components of adaptive expertise. Through an overview of perspectives from the field of motor control and learning, we emphasize the interplay between the inherent noisiness of the human motor architecture and the stability of motor skill performances. Ultimately, we challenge the very idea of routine. Our goal is threefold: to reconcile common misconceptions about the rote nature of routine precision skill performance, to offer educators principles to enhance adaptive expertise as an outcome of precision skill training, and to expand the conversation between 'routine' and 'adaptive' forms of expertise in health professions education.

Does group size matter during collaborative skills learning? A randomised study

BACKGROUND

Collaborative skills learning in the form of dyad learning compared with individual learning has been shown to lead to non-inferior skills retention and transfer. However, we have limited knowledge on which learning activities improve collaborative skills training and how the number of collaborators may impact skills transfer. We explored the effects of skills training individually, in dyads, triads or tetrads on learning activities during training and on subsequent skills transfer.

METHODS

In a randomised, controlled study, participants completed a pre-post-transfer-test set-up in groups of one to four. Participants completed 2 hours of obstetric ultrasound training. In the dyad, triad and tetrad group participants took turns actively handling the ultrasound probe. All performances were rated by two blinded experts using the Objective Structured Assessment of Ultrasound Skills (OSAUS) scale and a Global Rating Scale (GRS). All training was video recorded, and learning activities were analysed using the Interactive-Constructive-Active-Passive (ICAP) framework.

RESULTS

One hundred one participants completed the simulation-based training, and ninety-seven completed the transfer test. Performance scores improved significantly from pre- to post-test for all groups (p < 0.001, ηp²  = 0.55). However, group size did not affect transfer test performance on OSAUS scores (p = 0.13, ηp²  = 0.06) or GRS scores (p = 0.23, ηp²  = 0.05). ICAP analyses of training activities showed that time spent on non-learning and passive learning activities increased with group size (p < 0.001, ηp²  = 0.31), whereas time spent on constructive and interactive learning activities was constant between groups compared with singles (p < 0.001, ηp²  = 0.72).

CONCLUSION

Collaborative skills learning in groups of up to four did not impair skills transfer despite less hands-on time. This may be explained by a compensatory shift towards constructive and interactive learning activities that outweigh the effect of shorter hands-on time.

Mastery versus invention learning: impacts on future learning of simulated procedural skills

BACKGROUND

Invention and mastery learning approaches differ in their foundational educational paradigms, proposed mechanisms of learning, and potential impacts on learning outcomes. They also differ in their resource requirements. We explored the relative effects of 'invent and problem-solve, followed by instruction' (PS-I) learning compared to mastery learning (i.e., standards-based training) on immediate post-test and Preparation for Future Learning (PFL) assessments. PFL assessments measure learners' capacity to use their existing knowledge and strategies to learn about and solve novel problems.

METHODS

In this non-inferiority trial, pre-clerkship medical students were randomized to either PS-I, Mastery Learning (ML), or instruction then practice (CON) during simulation-based training of infant lumbar puncture (LP). After a 2-week delay, participants returned to learn and complete a PFL assessment of simulated Knee Arthrocentesis. Two independent raters assessed performances with a 5-point global rating scale.

RESULTS

Based on our non-inferiority margin, analyses showed that for both the immediate post-test and the PFL assessment, the PS-I condition resulted in non-inferior outcomes relative to the ML condition. Results for the CON condition were mixed with respect to non-inferiority compared to either PS-I or ML.

CONCLUSIONS

We suggest cautiously that the PS-I approach was not inferior to the ML approach, based on skill acquisition and PFL assessment outcomes. With ML anecdotally and empirically requiring more time, greater faculty involvement, and higher costs, our findings question the preference ML has received relative to other instructional designs, especially in the healthcare simulation community. We encourage researchers to study the educational and resource impacts of instructional designs using non-inferiority designs.

Postgraduate medical procedural skills: attainment of curricular competencies using enhanced simulation-based mastery learning at a novel national boot camp

INTRODUCTION

A new UK medical postgraduate curriculum prompted the creation of a novel national medical postgraduate 'boot camp'. An enhanced simulation-based mastery learning (SBML) methodology was created to deliver procedural skills teaching within this national boot camp. This study aimed to explore the impact of SBML in a UK medical boot camp.

METHODS

One-hundred and two Scottish medical trainees attended a 3-day boot camp starting in August 2019. The novel enhanced SBML pathway entailed online pre-learning resources, deliberate practice, and simulation assessment and feedback. Data were gathered via pre- and post-boot camp questionnaires and assessment checklists.

RESULTS

The vast majority of learners achieved the required standard of performance. Learners reported increased skill confidence levels, including skills not performed at the boot camp.

CONCLUSION

An enhanced SBML methodology in a boot camp model enabled streamlined, standardised procedural skill teaching to a national cohort of junior doctors. Training curricular competencies were achieved alongside increased skill confidence.

Making Concepts Material: A Randomized Trial Exploring Simulation as a Medium to Enhance Cognitive Integration and Transfer of Learning

BACKGROUND

Simulation affords opportunities to represent functional relationships between conceptual (eg, anatomy) and procedural knowledge (eg, needle insertion technique) in ways that make them accessible to our many senses. Despite deprioritizing realism, such simulations may encourage trainees to create cognitive connections between these knowledge (ie, cognitive integration), which may improve transfer of learning. However, the impact of such "integrated instruction" has not been examined in simulation-based training. We developed integrated video- and simulator-based instructional modules for lumbar puncture training and compared their impacts on participants' retention, transfer, and conceptual knowledge.

METHODS

During 1 hour of simulation-based training, we randomized 66 medical students to receive either (a) video-based procedural-only instruction, (b) integrated video-based instruction, or (c) integrated simulator-based instruction. One week later, we tested participants' retention and transfer performances and their conceptual knowledge on a written test.

RESULTS

Simple mediation analyses revealed that compared with participants receiving procedural-only instruction, participants receiving integrated instruction had superior retention and transfer outcomes, mediated by gains in conceptual knowledge (all P < 0.01). We found no significant differences between the integrated groups for retention, transfer, or conceptual knowledge (all P > 0.01).

CONCLUSIONS

We extended previous findings, showing integrated instruction (video- or simulator-based) improved trainees' conceptual knowledge, which mediated their improved retention and transfer. As an innovation, we demonstrated how simulators can facilitate cognitive integration by making abstract conceptual-procedural relationships material. In suggesting how researchers might capitalize further on simulator-based integration, we offer an alternative framework for designing simulations that emphasizes cognitive processes rather than simulator fidelity.

Experience Level Influences Users' Interactions With and Expectations For Online Surgical Videos: A Mixed-Methods Study

PURPOSE

Upper-extremity surgeons and trainees widely use online surgical videos, and the use of these videos can assist with procedural learning. The purpose of this study was to characterize online video use and understand the role videos play in the learning process of orthopedic residents and practicing surgeons. We hypothesized that the use of surgical videos and video content desired among orthopedic learners differs based on their experience level.

METHODS

Four focus groups were conducted to discuss online surgical videos and their role in the learning process of orthopedic learners. Participants were separated based on their experience level. Three reviewers qualitatively analyzed the transcripts of the focus groups using constant comparative methods to identify overarching themes and categories. Findings regarding the participants' desires for video content and production quality were translated into a survey. The survey results were analyzed to assess their associations with experience level.

RESULTS

The focus group analysis helped identify 4 overarching themes that reflected users' interactions with videos: prewatching experience, choosing a video, video use, and video design, with the users' comments differing based on their experience level. The survey results showed that the median ideal length for a video was 10 minutes and that all users, regardless of their experience level, ranked showing the surgical procedure as the most important part of a video. Junior residents more frequently desired background information and a written outline of surgical steps, whereas more senior learners placed greater emphasis on advanced surgical decision-making and the use of particular implants/devices.

CONCLUSIONS

Experience level influences users' interactions with videos, including how they are chosen and used, and their expectations in terms of content and production.

CLINICAL RELEVANCE

Video creators should specify their targeted audience's experience level and adjust content to meet users' needs. Our results can provide video creators and hosts a checklist for appropriate content and production standards.

A Descriptive Analysis of the Cumulative Experiences of Emergency Medicine Residents in the Pediatric Emergency Department

OBJECTIVES

Most children seeking emergency care are evaluated in general emergency departments (EDs). The cumulative pediatric clinical experiences of emergency medicine (EM) residents are largely unknown. This study examined EM resident pediatric clinical experience through the lens of the Accreditation Council for Graduate Medical Education requirements and the Model of the Clinical Practice of Emergency Medicine.

METHODS

Retrospective, observational study of the cumulative clinical experience of two classes of EM residents from a 4-year training program at two pediatric EDs of a quaternary care pediatric center. A database of resident patient encounters was generated from the electronic medical record. Experiences classified included: diagnosis categories per the Model of the Clinical Practice of Emergency Medicine, procedures, and resuscitations. Results were stratified by age, acuity, and disposition.

RESULTS

Twenty-five EM residents evaluated 17,642 patients (median = 723). Most patients (73.5%) were emergent acuity (Emergency Severity Index triage level 2 or 3 or non-intensive care admission); 2% were critical. Residents participated in 598 (median = 22) medical resuscitations and 483 (median = 19) trauma resuscitations. Minor procedures (e.g., laceration repair) were commonly performed; critical procedures (e.g., intubation) were rare. Exposure to neonates was infrequent and pediatric deaths were rare. Abdominal pain (5.7%), asthma exacerbation (4.6%), and fever (3.8%) were the most common diagnoses.

CONCLUSIONS

Emergency medicine residents encountered a wide array of pediatric diagnoses throughout training and performed a substantial number of common pediatric procedures. Exposure to critical acuity and procedures, neonatal pathology, and certain pediatric-specific diagnoses, such as congenital heart disease, was limited despite training in a large, quaternary care children's hospital. Curriculum development and collaboration should focus on these areas.

Integrating trans health knowledge through instructional design: preparing learners for a continent - not an island - of primary care with trans people

In recent years the need to teach primary care providers to better care for transgender and non-binary (trans) patients has garnered significant scholarly and public attention. The alarming why motivating this surge in trans health primary care education has already been firmly established and needs no further comment. Instead, we offer new perspectives on how to do trans health primary care education. From treasured 'trans 101' educational interventions to trans health 'clinical pearls', the prevailing model used to teach primary care learners represents time-limited cultural competency-based education, which we argue creates an isolated education 'island'. In rethinking this approach, we present an introduction to the concepts of knowledge integration and the transfer of learning and apply them to show how trans health knowledge and skills should be structured within existing curricula to support effective learning and application. These instructional design considerations have yet to be extensively explored when teaching primary care learners trans health content and may be critical to building pedagogy that ultimately improves healthcare delivery. We conclude that trans health - and trans patients themselves - must not be treated as an isolated education island of knowledge and practice. Rather, it is the responsibility of educators to design instruction that encourages learners to integrate this knowledge with foundational principles of primary care; building bridges across a continent of primary care practice landscapes in turn.

How cognitive psychology changed the face of medical education research

In this article, the contributions of cognitive psychology to research and development of medical education are assessed. The cognitive psychology of learning consists of activation of prior knowledge while processing new information and elaboration on the resulting new knowledge to facilitate storing in long-term memory. This process is limited by the size of working memory. Six interventions based on cognitive theory that facilitate learning and expertise development are discussed: (1) Fostering self-explanation, (2) elaborative discussion, and (3) distributed practice; (4) help with decreasing cognitive load, (5) promoting retrieval practice, and (6) supporting interleaving practice. These interventions contribute in different measure to various instructional methods in use in medical education: problem-based learning, team-based learning, worked examples, mixed practice, serial-cue presentation, and deliberate reflection. The article concludes that systematic research into the applicability of these ideas to the practice of medical education presently is limited and should be intensified.

Simulation 2.0: Integrating Basic Scientists and Clinicians in a Simulation Environment

An increasing number of medical schools are implementing curricular changes that better integrate clinical and basic sciences throughout all four years of medical school. One of the most frequently cited reasons is to improve medical student clinical reasoning skills while simultaneously aiming to decrease the attrition of basic science knowledge. Multiple pedagogical strategies have been explored to achieve this goal. We have found that simulation is a viable medium to integrate basic science within standardized patient encounters for early medical students.

Using Learning Curves to Identify and Explain Growth Patterns of Learners in Bronchoscopy Simulation: A Mixed-Methods Study

PURPOSE

Learning curves can illustrate how trainees acquire skills and the path to competence. This study examined the growth trajectories of novice trainees while practicing on a bronchoscopy virtual reality (VR) simulator compared with those of experts.

METHOD

This was a sequential explanatory mixed-methods design. Twenty pediatric subspecialty trainees and 7 faculty practiced with the VR simulator (October 2017 to March 2018) at the Hospital for Sick Children, Toronto, Canada. The authors examined the relationship between number of repetitions and VR outcomes and patterns of growth using a growth mixture modeling. Using an instrumental case study design, field notes and semistructured interviews with trainees and simulation instructor were examined to explain the patterns of growth. The authors used a constant comparative approach to identify themes iteratively. Team analysis continued until a stable thematic structure was developed and applied to the entire data.

RESULTS

The growth mixture model identified 2 patterns of growth. A slower growth included learners that had inherent difficulty with the skill, did not integrate the knowledge of anatomy in simulation practice, and used the simulator for simple repetitive practice with no strategy for improvement in between trials. The faster growth included learners who used an adaptive expertise approach: integrating knowledge of anatomy, finding flexible solutions, and creating a deeper conceptual understanding.

CONCLUSIONS

The authors provide validity evidence for use of growth models in education and explain patterns of growth such as a "slow growth" with a mechanistic repetitive practice and a "fast growth" with adaptive expertise.

The Benefits of Tying Yourself in Knots: Unraveling the Learning Mechanisms of Guided Discovery Learning in an Open Surgical Skills Course

PURPOSE

Teaching technical skills through the use of guided discovery learning (GDL) is an ongoing topic of research. In this approach, learners practice and struggle before receiving formal instruction. This has shown promise in other domains of learning, yet in the realm of procedural skills, clarity is still needed. This study seeks to address these gaps by investigating efficacy and mechanisms relating to application for a GDL approach in teaching basic surgical skills.

METHOD

In 2018, young surgical trainees (N = 16) undertook a 6-week open surgical course applying the principles of GDL, each lesson beginning with a discovery phase before subsequent instruction and practice. A concurrent triangulation mixed-methods approach was used with direct observation and collection of semistructured interviews using a framework designed from productive failure literature. At the end of the course, all participants took a conceptual knowledge test and a performance-based skills test. Performance on the skills test was rated using global ratings and checklists.

RESULTS

The GDL cohort outperformed the historical cohort on the written exam (F [1,65] = 4.96, P = .029, d = .62), as well as on the summative suturing test (F [1,65] = 6.23, P = .015, d = .68). Furthermore, 3 main themes that highlight the mechanisms and mediators of efficient GDL were: (1) building conceptual knowledge, (2) motivating self-regulated learning, and (3) the type of skill and psychological safety.

CONCLUSIONS

GDL can be an efficient approach to teaching procedural skills. Implications for future research and curricular design are discussed.

The Effect of Simulator-Integrated Tutoring for Guidance in Virtual Reality Simulation Training

INTRODUCTION

Simulation-integrated tutoring in virtual reality (VR) simulation training by green lighting is a common learning support in simulation-based temporal bone surgical training. However, tutoring overreliance can negatively affect learning. We therefore wanted to investigate the effects of simulator-integrated tutoring on performance and learning.

METHODS

A prospective, educational cohort study of a learning intervention (simulator-integrated tutoring) during repeated and distributed VR simulation training for directed, self-regulated learning of the mastoidectomy procedure. Two cohorts of novices (medical students) were recruited: 16 participants were trained using the intervention program (intermittent simulator-integrated tutoring) and 14 participants constituted a nontutored reference cohort. Outcomes were final-product performance assessed by 2 blinded raters and simulator-recorded metrics.

RESULTS

Simulator-integrated tutoring had a large and positive effect on the final-product performance while turned on (mean difference = 3.8 points, P < 0.0001). However, this did not translate to a better final-product performance in subsequent nontutored procedures. The tutored cohort had a better metrics-based score, reflecting higher efficiency of drilling (mean difference = 3.6%, P = 0.001). For the individual metrics, simulator-integrated tutoring had mixed effects both during procedures and on the tutored cohort in general (learning effect).

CONCLUSIONS

Simulator-integrated tutoring by green lighting did not induce a better final-product performance but increased efficiency. The mixed effects on learning could be caused by tutoring overreliance, resulting from a lack of cognitive engagement when the tutor function is on. Further learning strategies such as feedback should be explored to support novice learning and cognitive engagement.

Assigning Medical Students Learning Goals: Do They Do It, and What Happens When They Don't?

Theory: Medical curricula now include more time for trainees to manage their studying independently, yet evidence suggests that time is not well spent without guidance. Social-cognitivist models of self-regulated learning suggest value when guiding learners to set goals related to their performance processes (actions producing outcomes) versus their performance outcomes (products of performance). Hypotheses: We expected participants oriented to set process goals would demonstrate better suturing skill retention compared with participants oriented to set outcome goals. Method: We randomly assigned 41 medical students to two groups: outcome oriented or process oriented. They self-scored their performance using a visual analog scale on every third trial during 25 training trials, and during 10 retention trials 2 weeks later. Two raters assessed participants' suturing performances (process) and final products (outcome). After finding weak support for our hypothesis, we calculated a "self-monitoring calibration coefficient" as the Pearson's correlation between the raters' average score and each participant's self-scores. We used a mixed-effects analysis of variance to compare participants' performance scores as well as t tests and an analysis of variance to compare their self-monitoring calibration coefficients. Results: Analysis of skill retention data revealed a significant Group × Trial interaction, suggesting a benefit for the process group only for the 10th retention trial (p = .03). During training, the process group had significantly better (p = .02) self-monitoring calibration (r = .71 ± .29) than the outcome group (r = .38 ± .55). In retention, participants in both groups were significantly better calibrated (p = .04) with rater's scores of performance processes (r = .39 ± .60) versus performance outcomes (r = .11 ± .63). Conclusions: Our findings provide limited evidence for our original hypothesis. Perhaps more important, however, our self-monitoring calibration data highlighted inconsistencies between our interventions and our participants' apparent preferences. Not all participants adopted their assigned goal setting orientation, showing that researchers and educators must consider the extent to which trainees adopt imposed instructions in any educational intervention.

Interdisciplinary expertise in medical practice: Challenges of using and producing knowledge in complex problem-solving

Purpose: Clarification of interdisciplinary expertise as the ability to deal with the cognitive and epistemological challenges of multi- and interdisciplinary problem-solving-such as in developing and implementing medical technology for diagnoses and treatment of patients in collaborations between clinicians, technicians, and engineers-and of the higher-order cognitive skills needed as part of this expertise. Method: Clarify the epistemological difficulties of combining scientific knowledge, methodologies and technologies from different disciplines in problem-solving, by drawing on recent developments in the philosophy of science. Conclusion: We argue that interdisciplinary expertise involves the cognitive ability to connect, translate and establish links between disciplinary knowledge, as well as the metacognitive ability to understand and explain the role of the disciplinary perspective-consisting of, e.g. basic concepts, theories, models, methodologies, technologies, and specific ways of measuring, reasoning and modeling in a discipline-in how knowledge is used and produced.

Why Content and Cognition Matter: Integrating Conceptual Knowledge to Support Simulation-Based Procedural Skills Transfer

BACKGROUND

Curricular constraints require being selective about the type of content trainees practice in their formal training. Teaching trainees procedural knowledge about "how" to perform steps of a skill along with conceptual knowledge about "why" each step is performed can support skill retention and transfer (i.e., the ability to adapt knowledge to novel problems). However, how best to organize how and why content for procedural skills training is unknown.

OBJECTIVES

We examined the impact of different approaches to integrating why and how content on trainees' skill retention and transfer of simulation-based lumbar puncture (LP).

DESIGN AND PARTICIPANTS

We randomized medical students (N = 66) to practice LP for 1 h using one of three videos. One video presented only the how content for LP (Procedural Only). Two other videos presented how and why content (e.g., anatomy) in two ways: Integrated in Sequence, with why content followed by how content, or Integrated for Causation, with how and why content integrated throughout.

MAIN MEASURES

Pairs of blinded raters scored participants' retention and transfer LP performances on a global rating scale (GRS), and written tests assessed participants' procedural and conceptual knowledge.

KEY RESULTS

Simple mediation regression analyses showed that participants receiving an integrated instructional video performed significantly better on transfer through their intervention's positive impact on conceptual knowledge (all p < 0.01). Further, the Integrated for Causation group performed significantly better on transfer than the Integrated in Sequence group (p < 0.01), again mediated by improved conceptual knowledge. We observed no mediation of participants' skill retention (all p > 0.01).

CONCLUSIONS

When teaching supports cognitive integration of how and why content, trainees are able to transfer learning to new problems because of their improved conceptual understanding. Instructional designs for procedural skills that integrate how and why content can help educators optimize what trainees learn from each repetition of practice.

Up or down? A randomized trial comparing image orientations during transvaginal ultrasound training

INTRODUCTION

There is no international consensus on the orientation of transvaginal ultrasound images and no evidence exists to support the superiority of one image orientation over the other. The aim of this study was to compare learning curves and skills transfer in a group of novices randomized to top-down or bottom-up image orientation, and to determine whether individual preferences for image orientation affect learning and skills transfer.

MATERIAL AND METHODS

60 senior medical students, with no prior ultrasound experience, were randomized to orient the image top-down or bottom-up during training on an ultrasound simulator until attaining expert levels of performance. Participants then completed a transfer test involving a systematic ultrasound examination on a physical mannequin using real ultrasound equipment. Performance was assessed during the transfer test by two independent raters using the objective structured assessment of ultrasound skills (OSAUS) score and a global rating score.

RESULTS

The bottom-up group reached the expert level with significantly fewer attempts than did the top-down group [median ± interquartile range: 4 ± 2 vs 5 ± 3] (U = 285.5, P = 0.014). The bottom-up group used less time to achieve the expert level (median ± interquartile range: 3 h 2 m ± 1 h 14 m vs 3 h 28 m ± 2 h 21 m) (U = 301.5, P = 0.029). The two groups performed similarly during the transfer test with respect to their OSAUS scores (top-down 56.7% vs bottom-up 53.2%, P = 0.13). The global rating scores were higher in the top-down group (top-down 57.1% vs bottom-up 50.0%, P = 0.02).

CONCLUSIONS

Orientation of the images bottom-up rather than top-down, led to a steeper learning curve, but had little or no impact on the subsequent transfer of skills.

Beyond "formative": assessments to enrich student learning

Formative assessments can enhance and enrich student learning. Typically, these have been used to provide feedback against end-of-course standards and prepare students for summative assessments of performance or measurement of competence. Here, we present the case for using assessments for learning to encompass a wider range of important outcomes. We discuss 1) the rationale for using assessment for learning; 2) guiding theories of expertise that inform assessment for learning; 3) theoretical and empirical evidence; 4) approaches to rigor and validation; and 5) approaches to implementation at multiple levels of the curriculum. The literature strongly supports the use of assessments as an opportunity to reinforce and enhance learning. Physiology teachers have a wide range of theories, models, and interventions from which to prepare students for retention, application, transfer, and future learning by using assessments.