Optimizing resource utilization during proficiency-based training of suturing skills in medical students: a randomized controlled trial of faculty-led, peer tutor-led, and holography-augmented methods of teaching

Physical and biophysical markers of assessment in medical training: A scoping review of the literature

PURPOSE

Assessment in medical education has changed over time to measure the evolving skills required of current medical practice. Physical and biophysical markers of assessment attempt to use technology to gain insight into medical trainees' knowledge, skills, and attitudes. The authors conducted a scoping review to map the literature on the use of physical and biophysical markers of assessment in medical training.

MATERIALS AND METHODS

The authors searched seven databases on 1 August 2022, for publications that utilized physical or biophysical markers in the assessment of medical trainees (medical students, residents, fellows, and synonymous terms used in other countries). Physical or biophysical markers included: heart rate and heart rate variability, visual tracking and attention, pupillometry, hand motion analysis, skin conductivity, salivary cortisol, functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The authors mapped the relevant literature using Bloom's taxonomy of knowledge, skills, and attitudes and extracted additional data including study design, study environment, and novice vs. expert differentiation from February to June 2023.

RESULTS

Of 6,069 unique articles, 443 met inclusion criteria. The majority of studies assessed trainees using heart rate variability (n = 160, 36%) followed by visual attention (n = 143, 32%), hand motion analysis (n = 67, 15%), salivary cortisol (n = 67, 15%), fMRI (n = 29, 7%), skin conductivity (n = 26, 6%), fNIRs (n = 19, 4%), and pupillometry (n = 16, 4%). The majority of studies (n = 167, 38%) analyzed non-technical skills, followed by studies that analyzed technical skills (n = 155, 35%), knowledge (n = 114, 26%), and attitudinal skills (n = 61, 14%). 169 studies (38%) attempted to use physical or biophysical markers to differentiate between novice and expert.

CONCLUSION

This review provides a comprehensive description of the current use of physical and biophysical markers in medical education training, including the current technology and skills assessed. Additionally, while physical and biophysical markers have the potential to augment current assessment in medical education, there remains significant gaps in research surrounding reliability, validity, cost, practicality, and educational impact of implementing these markers of assessment.

The effects of mentorship and educational videos on nursing students' physical examination skills: a clinical audit

BACKGROUND

Poor competence in clinical examination skills among nurses has been reported in practice, and there is a strong consensus that physical examination (PE) education must be improved. However, deficiencies cannot be resolved by theoretical training alone, and new training approaches are required to enable nursing students to perform PE accurately. This study aimed to determine and compare the effect of two new educational methods (mentorship and educational video methods) on the physical examination skills of Iranian nursing students.

METHODS

This study was a clinical audit with three group pretest posttest design. Eligible nursing students were recruited through the census method and assigned to three groups (mentorship, educational videos, and control group) using permuted block randomization. Students were taught PE skills in three vital systems through three approaches (mentors, educational videos, and routine educational methods). Data were collected using a two-part instrument consisting of a demographic information questionnaire and a 32-item checklist for assessing the students' skills in examining the respiratory system (10 items), cardiovascular system (13 items), and 12 cranial nerves (9 items). Data were analyzed using SPSS software version 16 and descriptive and analytical statistical tests.

RESULTS

At baseline, students in all groups scored less than half of the possible scores in all three systems, and the mean scores of the three groups were not statistically different (P > 0.05). After the intervention, the mean scores of students in the mentorship group increased significantly in all three systems (P < 0.001), whereas the mean scores of students in the educational video group and the control group did not change significantly (P > 0.05). Furthermore, after the intervention, the mean scores of the educational video group and the control group did not significantly differ in any of the three systems (P > 0.05). The ANCOVA showed that with posttest score as the covariate, PE skills in all three systems improved one week after the intervention in the mentor group compared to the control group and the educational video group. However, PE skills in all three systems did not improve one week after the intervention in the educational video group compared to the control group.

CONCLUSIONS

The mentorship method is more effective than the educational video and routine methods for teaching PE skills to nursing students. Nursing schools can use the mentor method in training nursing students, and it is recommended to revise the PE lesson unit in the nursing curriculum and exchange it from a purely theoretical-based unit to a mixture of theoretical and clinical-based training. Educational videos alone cannot improve nursing students' PE skills; thus, if educational videos are used to teach clinical skills, it is necessary to hold hands-on sessions to provide feedback to students and correct their mistakes.

Improved Confidence and Clinical Application: The Effects of a Longitudinal Suture Curriculum for Medical Students

OBJECTIVE

The ability to competently suture is an expected skill for graduating medical students, but many graduates report feeling unprepared to perform this skill. This study aimed to improve student confidence and clinical readiness for third-year clerkships by implementing a novel, mandatory 7.5-hour longitudinal suturing skills curriculum across the first 3 years of medical school.

METHODS

The required suturing skills curriculum was implemented for all medical students throughout the first 3 years of medical school at a large academic health center in the mid-South United States. Precurriculum (n = 167) and postcourse (n = 148) surveys were administered to first-year students in the first year of the curriculum (2017-2018), and a parallel follow-up survey was administered to this cohort in 2020 after students completed their clinical clerkship year (n = 82). Aggregate changes in students' survey responses were analyzed for proper instrument position, simple interrupted sutures, and instrument ties using independent groups Mann-Whitney U tests and Rosenthal correlation coefficients for effect sizes.

RESULTS

Statistically significant improvement from pre to post was observed in student comfort in performing three basic skills: proper instrument position (P < 0.001), simple interrupted suture (P < 0.001), and instrument ties (P < 0.001). These pre-post gains were sustained at 2-year follow-up (P < 0.001). Also, the majority of students (66%) reported they were very or completely prepared to suture wounds during their clerkships. Most (83%) also reported they had successfully sutured patient wounds during third-year clerkships without needing significant direction or guidance.

CONCLUSIONS

We found that a longitudinal suture curriculum with dedicated faculty involvement can improve student confidence in suturing and overall preparedness for third-year clerkships. Although the study is limited to ratings of student comfort and self-reported performance as well as some attrition of responses at postcourse survey and postclerkship survey, the findings highlight the importance of a focused curriculum dedicated to teaching basic suturing skills. Our findings also contribute to the limited body of work examining longitudinal surgical skills development for medical students.

The HoloLens in medicine: A systematic review and taxonomy

The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.

Effectiveness of peer teaching in health professions education: A systematic review and meta-analysis

BACKGROUND

The reform in health professions education requires the focus to shift from fact memorization to exploring, analyzing, assimilating, and synthesizing information to promote active and collaborative learning. Peer teaching is one of the educational strategies.

AIMS

This review aimed to explore and synthesize quantitative evidence to determine the overall effect of peer teaching in enhancing students' theoretical knowledge and practical skills (e.g., procedural skills and resuscitation) in health professions education.

METHODS

PubMed, ScienceDirect, CINAHL, ERIC, ProQuest, reference lists of relevant studies, and reviews were searched till November 2021. Results were pooled using random-effects meta-analysis or narrative synthesis.

RESULTS

A total of 44 RCTs were included. This review showed a significant effect of peer teaching on procedural skills improvement and a comparable effect on theoretical knowledge and resuscitation skills acquisition compared to the conventional teaching method. Near-peer teaching seemed to be the most effective method for skill improvement. Subgroup analysis showed no significant differences between peer teaching and conventional teaching groups (e.g., expert/faculty teaching, self-study or lectures).

CONCLUSIONS

Peer teaching seems to be a promising teaching and learning strategy in health professions education, positively affecting theoretical knowledge and procedural skills. Future research should explore the effect of peer teaching in developing countries to provide a comprehensive picture of peer teaching.

Near-Peer Teaching in Conjunction with Flipped Classroom to Teach First-Year Medical Students Basic Surgical Skills

BACKGROUND

There is increasing evidence that students are completing medical school with insufficient surgical education. Near-peer tutoring and flipped classroom formatting may be used to enhance learning while simultaneously relieving faculty burden of teaching. Here, we qualitatively evaluate a 3-month course that integrates the use of near-peer teaching and flipped classroom formatting, with the goal of increasing first-year medical students' self-perceived confidence in performing basic sutures and knot-ties as well as interest in surgery.

METHODS

Twenty-one first-year medical students participated in a suturing and knot-tying course led by senior medical students. The course consisted of 2-h sessions held every 2 weeks for a total of five sessions. Students were sent publicly available videos prior to each session by which to learn the upcoming techniques and received live feedback from instructors during sessions. Questionnaires were completed pre-course and post-course.

RESULTS

Compared to pre-course ratings, post-course ratings of self-perceived confidence to perform various knot-ties and sutures all increased significantly (p < 0.05). All students stated that the course strengthened their desire to pursue a career in surgery. Student feedback of the course was overall positive.

CONCLUSIONS

Near-peer teaching can be used in conjunction with flipped classroom to increase first-year medical students' self-perceived confidence in surgical suturing and knot-tying as well as interest in surgery. This curriculum may serve as an outline for student-led courses at other institutions.

Outcomes, Measurement Instruments, and Their Validity Evidence in Randomized Controlled Trials on Virtual, Augmented, and Mixed Reality in Undergraduate Medical Education: Systematic Mapping Review

BACKGROUND

Extended reality, which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), is increasingly used in medical education. Studies assessing the effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence.

OBJECTIVE

Our aim is to determine the choice of outcomes, measurement instruments, and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of VR, AR, and MR in medical student education.

METHODS

We conducted a systematic mapping review. We searched 7 major bibliographic databases from January 1990 to April 2020, and 2 reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines.

RESULTS

Of the 126 retrieved RCTs, 115 (91.3%) were on VR and 11 (8.7%) were on AR. No RCT on MR in medical student education was found. Of the 115 studies on VR, 64 (55.6%) were on VR simulators, 30 (26.1%) on screen-based VR, 9 (7.8%) on VR patient simulations, and 12 (10.4%) on VR serious games. Most studies reported only a single outcome and immediate postintervention assessment data. Skills outcome was the most common outcome reported in studies on VR simulators (97%), VR patient simulations (100%), and AR (73%). Knowledge was the most common outcome reported in studies on screen-based VR (80%) and VR serious games (58%). Less common outcomes included participants' attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement or self-efficacy beliefs, emotional state, competency developed, and patient outcomes. At least one form of validity evidence was found in approximately half of the studies on VR simulators (55%), VR patient simulations (56%), VR serious games (58%), and AR (55%) and in a quarter of the studies on screen-based VR (27%). Most studies used assessment methods that were implemented in a nondigital format, such as paper-based written exercises or in-person assessments where examiners observed performance (72%).

CONCLUSIONS

RCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. The studies largely report immediate postintervention outcome data and use assessment methods that are in a nondigital format. Future RCTs should include a broader set of outcomes, report on the validity evidence of the measurement instruments used, and explore the use of assessments that are implemented digitally.

Peer-assisted learning in medical education: A systematic review and meta-analysis

OBJECTIVES

The prevalence of peer-assisted learning (PAL) featuring alongside the core medical curriculum is increasing; however, the evidence base for PAL's efficacy on academic performance is limited. This systematic review of randomised studies of PAL in medical school sets out to assess the impact of PAL on academic outcomes in medical school and evaluate whether PAL confers a benefit in specific educational contexts.

METHODS

A literature search was conducted across MEDLINE, Ovid Embase, Web of Science and Education Research Complete. Titles and abstracts were screened, and records were selected following strict eligibility criteria. Following full-text assessment for eligibility, two reviewers independently extracted data from the final selection of records and a meta-analysis was performed. Studies were classified using a modified version of Kirkpatrick's levels of learning. Student test scores were standardised by calculating the standardised mean difference (SMD).

RESULTS

Twenty-seven randomised controlled trials were eligible for inclusion, and twenty-one provided sufficient and complete data to enable meta-analysis. There was a significant improvement in the academic performance of medical students who received PAL compared with those in the control group (SMD = 0.52 [95% confidence interval 0.18-0.85]; p = .003). The impact of PAL was greater amongst clinical medical students (SMD = 0.63; p = .02) than preclinical medical students (SMD = 0.39; p = .08) and when used for teaching practical skills (SMD = 0.69; p = .001) compared with theory (SMD = -0.11; p = .21). Students taught by PAL also achieved better results in assessments conducted more than four weeks after course completion (SMD = 1.20; p = .04). [Correction added on 8 November 2020 after first publication. The data values in the Results section have been corrected in this version.] CONCLUSIONS: Medical students experiencing PAL benefit in terms of academic performance, relative to those not receiving PAL. PAL is of greatest value in the clinical stages of training and for practical skills. The long-term outcomes of PAL remain a priority for future research.

Standard Setting in Simulation-Based Training of Surgical Procedures: A Systematic Review

OBJECTIVE

This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures.

SUMMARY OF BACKGROUND

Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods.

METHODS

Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE.

RESULTS

Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence.

CONCLUSION

Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.

A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes

OBJECTIVE

To analyze all published prospective, randomized, and blinded clinical studies on the proficiency-based progression (PBP) training using objective performance metrics.

BACKGROUND

The benefit of PBP methodology to learning clinical skills in comparison to conventional training is not settled.

METHODS

Search of PubMed, Cochrane library's Central, EMBASE, MEDLINE, and Scopus databases, from inception to 1st March 2020. Two independent reviewers extracted the data. The Medical Education Research Study Quality Instrument (MERSQI) was used to assess the methodological quality of included studies. Results were pooled using biased corrected standardized mean difference and ratio-of-means. Summary effects were evaluated using a series of fixed and random effects models. The primary outcome was the number of procedural errors performed comparing PBP and non-PBP-based training pathways. Secondary outcomes were the number of procedural steps completed and the time to complete the task/procedure.

RESULTS

From the initial pool of 468 studies, 12 randomized clinical studies with a total of 239 participants were included in the analysis. In comparison to the non-PBP training, ratio-of-means results showed that PBP training reduced the number of performance errors by 60% (P < 0.001) and procedural time by 15% (P = 0.003) and increased the number of steps performed by 47% (P < 0.001).

CONCLUSIONS AND RELEVANCE

Our systematic review and meta-analysis confirms that PBP training in comparison to conventional or quality assured training improved trainees' performances, by decreasing procedural errors and procedural time, while increasing the number of correct steps taken when compared to standard simulation-based training.

Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters

BACKGROUND

A major challenge with Competency Based Medical Education (CBME) is that of increased assessment burden on faculty. To reduce this burden, the accuracy and reliability of peer-assessment for surgical skills requires further exploration.

METHODS

Forty-two second year medical students were video recorded while performing a simple interrupted suture and an instrument tie. Four novice raters underwent a short training session on the use of the Objective Structured Assessment of Technical Skills (OSATS) checklists. Videos of the suturing task were then independently assessed by the four novice raters and two expert raters on two occasions. Agreement between novice and expert rater scores was calculated using the intraclass correlation coefficient (ICC).

RESULTS

For both simple interrupted suturing (ICC = 0.78, CI = 0.66-0.86, p < 0.001) and instrument ties (ICC = 0.87, CI = 0.80-0.92, p < 0.001), there was good agreement between novice and expert raters.

CONCLUSIONS

Novice raters can be taught to use the OSATS checklists to assess peers on simple suturing and instrument tying tasks.

Video-Based Guided Simulation without Peer or Expert Feedback is Not Enough: A Randomized Controlled Trial of Simulation-Based Training for Medical Students

BACKGROUND

Feedback is a pivotal cornerstone and a challenge in psychomotor training. There are different teaching methodologies; however, some may be less effective.

METHODS

A prospective randomized controlled trial was conducted in 130 medical students to compare the effectiveness of the video-guided learning (VLG), peer-feedback (PFG) and the expert feedback (EFG) for teaching suturing skills. The program lasted 4 weeks. Students were recorded making 3-simple stitches (pre-assessment and post-assessment). The primary outcome was a global scale (OSATS). The secondary outcomes were performance time, specific rating scale (SRS) and the impact of the intervention (IOI), defined as the variation between the final and initial OSATS and SRS scores.

RESULTS

No significant differences were found between PFG and EFG in post-assessment results of OSATS, SRS scores or in the IOI for OSATS and SRS scores. Post-assessment results of PFG and EFG were significantly superior to VLG in OSATS and SRS scores [(19.8 (18.5-21); 16.6 (15.5-17.5)) and (20.3 (19.88-21); 16.8 (16-17.5)) vs (15.7 (15-16); 13.3 (12.5-14)) (p < 0.05)], respectively. The results of PFG and EFG were significantly superior to VLG in the IOI for OSATS [7 (4.5-9) and 7.4 (4.88-10) vs 3.5 (1.5-6) (p < 0.05)] and SRS scores [5.4 (3.5-7) and 6.3 (4-8.5) vs 3.1 (1.13-4.88) (p < 0.05)], respectively.

CONCLUSION

The video-guided learning methodology without any kind of feedback is not enough for teaching suturing skills compared to expert or peer feedback. The peer feedback methodology appears to be a viable alternative to handling the emerging demands in medical education.