1
|
Miller DT, Michael S, Bell C, Brevik CH, Kaplan B, Svoboda E, Kendall J. Physical and biophysical markers of assessment in medical training: A scoping review of the literature. MEDICAL TEACHER 2024:1-9. [PMID: 38688520 DOI: 10.1080/0142159x.2024.2345269] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/16/2024] [Accepted: 04/16/2024] [Indexed: 05/02/2024]
Abstract
PURPOSE Assessment in medical education has changed over time to measure the evolving skills required of current medical practice. Physical and biophysical markers of assessment attempt to use technology to gain insight into medical trainees' knowledge, skills, and attitudes. The authors conducted a scoping review to map the literature on the use of physical and biophysical markers of assessment in medical training. MATERIALS AND METHODS The authors searched seven databases on 1 August 2022, for publications that utilized physical or biophysical markers in the assessment of medical trainees (medical students, residents, fellows, and synonymous terms used in other countries). Physical or biophysical markers included: heart rate and heart rate variability, visual tracking and attention, pupillometry, hand motion analysis, skin conductivity, salivary cortisol, functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The authors mapped the relevant literature using Bloom's taxonomy of knowledge, skills, and attitudes and extracted additional data including study design, study environment, and novice vs. expert differentiation from February to June 2023. RESULTS Of 6,069 unique articles, 443 met inclusion criteria. The majority of studies assessed trainees using heart rate variability (n = 160, 36%) followed by visual attention (n = 143, 32%), hand motion analysis (n = 67, 15%), salivary cortisol (n = 67, 15%), fMRI (n = 29, 7%), skin conductivity (n = 26, 6%), fNIRs (n = 19, 4%), and pupillometry (n = 16, 4%). The majority of studies (n = 167, 38%) analyzed non-technical skills, followed by studies that analyzed technical skills (n = 155, 35%), knowledge (n = 114, 26%), and attitudinal skills (n = 61, 14%). 169 studies (38%) attempted to use physical or biophysical markers to differentiate between novice and expert. CONCLUSION This review provides a comprehensive description of the current use of physical and biophysical markers in medical education training, including the current technology and skills assessed. Additionally, while physical and biophysical markers have the potential to augment current assessment in medical education, there remains significant gaps in research surrounding reliability, validity, cost, practicality, and educational impact of implementing these markers of assessment.
Collapse
Affiliation(s)
- Danielle T Miller
- Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA
| | - Sarah Michael
- Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA
| | - Colin Bell
- Department of Emergency Medicine, University of Calgary, Calgary, Canada
| | - Cody H Brevik
- Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA
| | - Bonnie Kaplan
- Department of Emergency Medicine, University of Colorado School of Medicine, Aurora, CO, USA
| | - Ellie Svoboda
- Education Informationist, Strauss Health Sciences Library, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - John Kendall
- Department of Emergency Medicine, Stanford School of Medicine, Palo Alto, CA, USA
| |
Collapse
|
2
|
Nasiri MA, Sabery M, Rezaei M, Gilasi H. The effects of mentorship and educational videos on nursing students' physical examination skills: a clinical audit. BMC Nurs 2023; 22:463. [PMID: 38057848 DOI: 10.1186/s12912-023-01626-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2023] [Accepted: 11/26/2023] [Indexed: 12/08/2023] Open
Abstract
BACKGROUND Poor competence in clinical examination skills among nurses has been reported in practice, and there is a strong consensus that physical examination (PE) education must be improved. However, deficiencies cannot be resolved by theoretical training alone, and new training approaches are required to enable nursing students to perform PE accurately. This study aimed to determine and compare the effect of two new educational methods (mentorship and educational video methods) on the physical examination skills of Iranian nursing students. METHODS This study was a clinical audit with three group pretest posttest design. Eligible nursing students were recruited through the census method and assigned to three groups (mentorship, educational videos, and control group) using permuted block randomization. Students were taught PE skills in three vital systems through three approaches (mentors, educational videos, and routine educational methods). Data were collected using a two-part instrument consisting of a demographic information questionnaire and a 32-item checklist for assessing the students' skills in examining the respiratory system (10 items), cardiovascular system (13 items), and 12 cranial nerves (9 items). Data were analyzed using SPSS software version 16 and descriptive and analytical statistical tests. RESULTS At baseline, students in all groups scored less than half of the possible scores in all three systems, and the mean scores of the three groups were not statistically different (P > 0.05). After the intervention, the mean scores of students in the mentorship group increased significantly in all three systems (P < 0.001), whereas the mean scores of students in the educational video group and the control group did not change significantly (P > 0.05). Furthermore, after the intervention, the mean scores of the educational video group and the control group did not significantly differ in any of the three systems (P > 0.05). The ANCOVA showed that with posttest score as the covariate, PE skills in all three systems improved one week after the intervention in the mentor group compared to the control group and the educational video group. However, PE skills in all three systems did not improve one week after the intervention in the educational video group compared to the control group. CONCLUSIONS The mentorship method is more effective than the educational video and routine methods for teaching PE skills to nursing students. Nursing schools can use the mentor method in training nursing students, and it is recommended to revise the PE lesson unit in the nursing curriculum and exchange it from a purely theoretical-based unit to a mixture of theoretical and clinical-based training. Educational videos alone cannot improve nursing students' PE skills; thus, if educational videos are used to teach clinical skills, it is necessary to hold hands-on sessions to provide feedback to students and correct their mistakes.
Collapse
Affiliation(s)
- Mohammad-Amin Nasiri
- Medical Surgical Nursing Department, School of Nursing and Midwifery, Kashan University of Medical Sciences, Kashan, Iran
| | - Mahdieh Sabery
- Trauma Nursing Research Center, Kashan University of Medical Sciences, Kashan, Iran.
| | - Mahboubeh Rezaei
- Trauma Nursing Research Center, Kashan University of Medical Sciences, Kashan, Iran
| | - Hamidreza Gilasi
- Department of Epidemiology & Biostatistics, Kashan University of Medical Sciences, Kashan, Iran
| |
Collapse
|
3
|
Shewmake C, Speed O, Gammill S, Hartzell LD, Thrush C, Phelan KD, Golinko M, Bhavaraju A. Improved Confidence and Clinical Application: The Effects of a Longitudinal Suture Curriculum for Medical Students. South Med J 2023; 116:806-811. [PMID: 37788814 DOI: 10.14423/smj.0000000000001612] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/05/2023]
Abstract
OBJECTIVE The ability to competently suture is an expected skill for graduating medical students, but many graduates report feeling unprepared to perform this skill. This study aimed to improve student confidence and clinical readiness for third-year clerkships by implementing a novel, mandatory 7.5-hour longitudinal suturing skills curriculum across the first 3 years of medical school. METHODS The required suturing skills curriculum was implemented for all medical students throughout the first 3 years of medical school at a large academic health center in the mid-South United States. Precurriculum (n = 167) and postcourse (n = 148) surveys were administered to first-year students in the first year of the curriculum (2017-2018), and a parallel follow-up survey was administered to this cohort in 2020 after students completed their clinical clerkship year (n = 82). Aggregate changes in students' survey responses were analyzed for proper instrument position, simple interrupted sutures, and instrument ties using independent groups Mann-Whitney U tests and Rosenthal correlation coefficients for effect sizes. RESULTS Statistically significant improvement from pre to post was observed in student comfort in performing three basic skills: proper instrument position (P < 0.001), simple interrupted suture (P < 0.001), and instrument ties (P < 0.001). These pre-post gains were sustained at 2-year follow-up (P < 0.001). Also, the majority of students (66%) reported they were very or completely prepared to suture wounds during their clerkships. Most (83%) also reported they had successfully sutured patient wounds during third-year clerkships without needing significant direction or guidance. CONCLUSIONS We found that a longitudinal suture curriculum with dedicated faculty involvement can improve student confidence in suturing and overall preparedness for third-year clerkships. Although the study is limited to ratings of student comfort and self-reported performance as well as some attrition of responses at postcourse survey and postclerkship survey, the findings highlight the importance of a focused curriculum dedicated to teaching basic suturing skills. Our findings also contribute to the limited body of work examining longitudinal surgical skills development for medical students.
Collapse
Affiliation(s)
| | - Olivia Speed
- the Department of Otolaryngology, University of Arkansas for Medical Sciences, Little Rock
| | - Sarah Gammill
- the Department of Surgery, Louisiana State University, New Orleans
| | - Larry D Hartzell
- the Department of Otolaryngology, University of Arkansas for Medical Sciences, Little Rock
| | | | - Kevin D Phelan
- the Department of Neurobiology & Developmental Sciences, University of Arkansas for Medical Sciences, Little Rock
| | - Michael Golinko
- the Department of Plastic Surgery, Vanderbilt University Medical Center, Nashville, Tennessee
| | | |
Collapse
|
4
|
Gsaxner C, Li J, Pepe A, Jin Y, Kleesiek J, Schmalstieg D, Egger J. The HoloLens in medicine: A systematic review and taxonomy. Med Image Anal 2023; 85:102757. [PMID: 36706637 DOI: 10.1016/j.media.2023.102757] [Citation(s) in RCA: 20] [Impact Index Per Article: 20.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 01/05/2023] [Accepted: 01/18/2023] [Indexed: 01/22/2023]
Abstract
The HoloLens (Microsoft Corp., Redmond, WA), a head-worn, optically see-through augmented reality (AR) display, is the main player in the recent boost in medical AR research. In this systematic review, we provide a comprehensive overview of the usage of the first-generation HoloLens within the medical domain, from its release in March 2016, until the year of 2021. We identified 217 relevant publications through a systematic search of the PubMed, Scopus, IEEE Xplore and SpringerLink databases. We propose a new taxonomy including use case, technical methodology for registration and tracking, data sources, visualization as well as validation and evaluation, and analyze the retrieved publications accordingly. We find that the bulk of research focuses on supporting physicians during interventions, where the HoloLens is promising for procedures usually performed without image guidance. However, the consensus is that accuracy and reliability are still too low to replace conventional guidance systems. Medical students are the second most common target group, where AR-enhanced medical simulators emerge as a promising technology. While concerns about human-computer interactions, usability and perception are frequently mentioned, hardly any concepts to overcome these issues have been proposed. Instead, registration and tracking lie at the core of most reviewed publications, nevertheless only few of them propose innovative concepts in this direction. Finally, we find that the validation of HoloLens applications suffers from a lack of standardized and rigorous evaluation protocols. We hope that this review can advance medical AR research by identifying gaps in the current literature, to pave the way for novel, innovative directions and translation into the medical routine.
Collapse
Affiliation(s)
- Christina Gsaxner
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria.
| | - Jianning Li
- Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| | - Antonio Pepe
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria
| | - Yuan Jin
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; Research Center for Connected Healthcare Big Data, Zhejiang Lab, Hangzhou, 311121 Zhejiang, China
| | - Jens Kleesiek
- Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| | - Dieter Schmalstieg
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; BioTechMed, 8010 Graz, Austria
| | - Jan Egger
- Institute of Computer Graphics and Vision, Graz University of Technology, 8010 Graz, Austria; Institute of AI in Medicine, University Medicine Essen, 45131 Essen, Germany; BioTechMed, 8010 Graz, Austria; Cancer Research Center Cologne Essen, University Medicine Essen, 45147 Essen, Germany
| |
Collapse
|
5
|
Zhang H, Liao AWX, Goh SH, Wu XV, Yoong SQ. Effectiveness of peer teaching in health professions education: A systematic review and meta-analysis. NURSE EDUCATION TODAY 2022; 118:105499. [PMID: 35961134 DOI: 10.1016/j.nedt.2022.105499] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/17/2022] [Revised: 06/26/2022] [Accepted: 08/02/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND The reform in health professions education requires the focus to shift from fact memorization to exploring, analyzing, assimilating, and synthesizing information to promote active and collaborative learning. Peer teaching is one of the educational strategies. AIMS This review aimed to explore and synthesize quantitative evidence to determine the overall effect of peer teaching in enhancing students' theoretical knowledge and practical skills (e.g., procedural skills and resuscitation) in health professions education. METHODS PubMed, ScienceDirect, CINAHL, ERIC, ProQuest, reference lists of relevant studies, and reviews were searched till November 2021. Results were pooled using random-effects meta-analysis or narrative synthesis. RESULTS A total of 44 RCTs were included. This review showed a significant effect of peer teaching on procedural skills improvement and a comparable effect on theoretical knowledge and resuscitation skills acquisition compared to the conventional teaching method. Near-peer teaching seemed to be the most effective method for skill improvement. Subgroup analysis showed no significant differences between peer teaching and conventional teaching groups (e.g., expert/faculty teaching, self-study or lectures). CONCLUSIONS Peer teaching seems to be a promising teaching and learning strategy in health professions education, positively affecting theoretical knowledge and procedural skills. Future research should explore the effect of peer teaching in developing countries to provide a comprehensive picture of peer teaching.
Collapse
Affiliation(s)
- Hui Zhang
- Alice Lee Centre of Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
| | | | - Sam Hongli Goh
- Alice Lee Centre of Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
| | - Xi Vivien Wu
- Alice Lee Centre of Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
| | - Si Qi Yoong
- Alice Lee Centre of Nursing Studies, Yong Loo Lin School of Medicine, National University of Singapore, Singapore.
| |
Collapse
|
6
|
Lu EL, Harris MK, Gao TZ, Good L, Harris DP, Renton D. Near-Peer Teaching in Conjunction with Flipped Classroom to Teach First-Year Medical Students Basic Surgical Skills. MEDICAL SCIENCE EDUCATOR 2022; 32:1015-1022. [PMID: 35936650 PMCID: PMC9340706 DOI: 10.1007/s40670-022-01602-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Accepted: 07/15/2022] [Indexed: 06/15/2023]
Abstract
BACKGROUND There is increasing evidence that students are completing medical school with insufficient surgical education. Near-peer tutoring and flipped classroom formatting may be used to enhance learning while simultaneously relieving faculty burden of teaching. Here, we qualitatively evaluate a 3-month course that integrates the use of near-peer teaching and flipped classroom formatting, with the goal of increasing first-year medical students' self-perceived confidence in performing basic sutures and knot-ties as well as interest in surgery. METHODS Twenty-one first-year medical students participated in a suturing and knot-tying course led by senior medical students. The course consisted of 2-h sessions held every 2 weeks for a total of five sessions. Students were sent publicly available videos prior to each session by which to learn the upcoming techniques and received live feedback from instructors during sessions. Questionnaires were completed pre-course and post-course. RESULTS Compared to pre-course ratings, post-course ratings of self-perceived confidence to perform various knot-ties and sutures all increased significantly (p < 0.05). All students stated that the course strengthened their desire to pursue a career in surgery. Student feedback of the course was overall positive. CONCLUSIONS Near-peer teaching can be used in conjunction with flipped classroom to increase first-year medical students' self-perceived confidence in surgical suturing and knot-tying as well as interest in surgery. This curriculum may serve as an outline for student-led courses at other institutions.
Collapse
Affiliation(s)
- Emily L. Lu
- The Ohio State University College of Medicine, Columbus, OH USA
| | - Micah K. Harris
- The Ohio State University College of Medicine, Columbus, OH USA
| | - Thomas Z. Gao
- The Ohio State University College of Medicine, Columbus, OH USA
| | - Logan Good
- The Ohio State University College of Medicine, Columbus, OH USA
| | - Daniel P. Harris
- Case Western Reserve University School of Medicine, Cleveland, OH USA
| | - David Renton
- Department of General Surgery, The Ohio State University Wexner Medical Center, Columbus, OH USA
| |
Collapse
|
7
|
Tudor Car L, Kyaw BM, Teo A, Fox TE, Vimalesvaran S, Apfelbacher C, Kemp S, Chavannes N. Outcomes, Measurement Instruments, and Their Validity Evidence in Randomized Controlled Trials on Virtual, Augmented, and Mixed Reality in Undergraduate Medical Education: Systematic Mapping Review. JMIR Serious Games 2022; 10:e29594. [PMID: 35416789 PMCID: PMC9047880 DOI: 10.2196/29594] [Citation(s) in RCA: 7] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2021] [Revised: 09/20/2021] [Accepted: 12/15/2021] [Indexed: 12/18/2022] Open
Abstract
BACKGROUND Extended reality, which encompasses virtual reality (VR), augmented reality (AR), and mixed reality (MR), is increasingly used in medical education. Studies assessing the effectiveness of these new educational modalities should measure relevant outcomes using outcome measurement tools with validity evidence. OBJECTIVE Our aim is to determine the choice of outcomes, measurement instruments, and the use of measurement instruments with validity evidence in randomized controlled trials (RCTs) on the effectiveness of VR, AR, and MR in medical student education. METHODS We conducted a systematic mapping review. We searched 7 major bibliographic databases from January 1990 to April 2020, and 2 reviewers screened the citations and extracted data independently from the included studies. We report our findings in line with the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. RESULTS Of the 126 retrieved RCTs, 115 (91.3%) were on VR and 11 (8.7%) were on AR. No RCT on MR in medical student education was found. Of the 115 studies on VR, 64 (55.6%) were on VR simulators, 30 (26.1%) on screen-based VR, 9 (7.8%) on VR patient simulations, and 12 (10.4%) on VR serious games. Most studies reported only a single outcome and immediate postintervention assessment data. Skills outcome was the most common outcome reported in studies on VR simulators (97%), VR patient simulations (100%), and AR (73%). Knowledge was the most common outcome reported in studies on screen-based VR (80%) and VR serious games (58%). Less common outcomes included participants' attitudes, satisfaction, cognitive or mental load, learning efficacy, engagement or self-efficacy beliefs, emotional state, competency developed, and patient outcomes. At least one form of validity evidence was found in approximately half of the studies on VR simulators (55%), VR patient simulations (56%), VR serious games (58%), and AR (55%) and in a quarter of the studies on screen-based VR (27%). Most studies used assessment methods that were implemented in a nondigital format, such as paper-based written exercises or in-person assessments where examiners observed performance (72%). CONCLUSIONS RCTs on VR and AR in medical education report a restricted range of outcomes, mostly skills and knowledge. The studies largely report immediate postintervention outcome data and use assessment methods that are in a nondigital format. Future RCTs should include a broader set of outcomes, report on the validity evidence of the measurement instruments used, and explore the use of assessments that are implemented digitally.
Collapse
Affiliation(s)
- Lorainne Tudor Car
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.,Department of Primary Care and Public Health, School of Public Health, Imperial College London, London, United Kingdom
| | - Bhone Myint Kyaw
- Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Andrew Teo
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Tatiana Erlikh Fox
- Centre for Population Health Sciences, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore.,Department of Internal Medicine, Onze Lieve Vrouwen Gasthuis, Amsterdam, Netherlands
| | - Sunitha Vimalesvaran
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Christian Apfelbacher
- Institute of Social Medicine and Health Systems Research, Otto von Guericke University Magdeburg, Magdegurg, Germany.,Family Medicine and Primary Care, Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore, Singapore
| | - Sandra Kemp
- Faculty of Health Sciences, Curtin Medical School, Curtin University, Bentley, Australia
| | - Niels Chavannes
- Department of Public Health and Primary Care, Leiden University Medical Centre, Leiden, Netherlands
| |
Collapse
|
8
|
Brierley C, Ellis L, Reid ER. Peer-assisted learning in medical education: A systematic review and meta-analysis. MEDICAL EDUCATION 2022; 56:365-373. [PMID: 34595769 DOI: 10.1111/medu.14672] [Citation(s) in RCA: 34] [Impact Index Per Article: 17.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/25/2021] [Revised: 08/24/2021] [Accepted: 09/22/2021] [Indexed: 06/13/2023]
Abstract
OBJECTIVES The prevalence of peer-assisted learning (PAL) featuring alongside the core medical curriculum is increasing; however, the evidence base for PAL's efficacy on academic performance is limited. This systematic review of randomised studies of PAL in medical school sets out to assess the impact of PAL on academic outcomes in medical school and evaluate whether PAL confers a benefit in specific educational contexts. METHODS A literature search was conducted across MEDLINE, Ovid Embase, Web of Science and Education Research Complete. Titles and abstracts were screened, and records were selected following strict eligibility criteria. Following full-text assessment for eligibility, two reviewers independently extracted data from the final selection of records and a meta-analysis was performed. Studies were classified using a modified version of Kirkpatrick's levels of learning. Student test scores were standardised by calculating the standardised mean difference (SMD). RESULTS Twenty-seven randomised controlled trials were eligible for inclusion, and twenty-one provided sufficient and complete data to enable meta-analysis. There was a significant improvement in the academic performance of medical students who received PAL compared with those in the control group (SMD = 0.52 [95% confidence interval 0.18-0.85]; p = .003). The impact of PAL was greater amongst clinical medical students (SMD = 0.63; p = .02) than preclinical medical students (SMD = 0.39; p = .08) and when used for teaching practical skills (SMD = 0.69; p = .001) compared with theory (SMD = -0.11; p = .21). Students taught by PAL also achieved better results in assessments conducted more than four weeks after course completion (SMD = 1.20; p = .04). [Correction added on 8 November 2020 after first publication. The data values in the Results section have been corrected in this version.] CONCLUSIONS: Medical students experiencing PAL benefit in terms of academic performance, relative to those not receiving PAL. PAL is of greatest value in the clinical stages of training and for practical skills. The long-term outcomes of PAL remain a priority for future research.
Collapse
Affiliation(s)
| | - Leila Ellis
- Torbay and South Devon NHS Foundation Trust, Torquay, UK
| | | |
Collapse
|
9
|
Abstract
OBJECTIVE This systematic review aims to examine the use of standard-setting methods in the context of simulation-based training of surgical procedures. SUMMARY OF BACKGROUND Simulation-based training is increasingly used in surgical education. However, it is important to determine which level of competency trainees must reach during simulation-based training before operating on patients. Therefore, pass/fail standards must be established using systematic, transparent, and valid methods. METHODS Systematic literature search was done in four databases (Ovid MEDLINE, Embase, Web of Science, and Cochrane Library). Original studies investigating simulation-based assessment of surgical procedures with application of a standard setting were included. Quality of evidence was appraised using GRADE. RESULTS Of 24,299 studies identified by searches, 232 studies met the inclusion criteria. Publications using already established standard settings were excluded (N = 70), resulting in 162 original studies included in the final analyses. Most studies described how the standard setting was determined (N = 147, 91%) and most used the mean or median performance score of experienced surgeons (n = 65, 40%) for standard setting. We found considerable differences across most of the studies regarding study design, set-up, and expert level classification. The studies were appraised as having low and moderate evidence. CONCLUSION Surgical education is shifting towards competency-based education, and simulation-based training is increasingly used for acquiring skills and assessment. Most studies consider and describe how standard settings are established using more or less structured methods but for current and future educational programs, a critical approach is needed so that the learners receive a fair, valid and reliable assessment.
Collapse
|
10
|
Mazzone E, Puliatti S, Amato M, Bunting B, Rocco B, Montorsi F, Mottrie A, Gallagher AG. A Systematic Review and Meta-analysis on the Impact of Proficiency-based Progression Simulation Training on Performance Outcomes. Ann Surg 2021; 274:281-289. [PMID: 33630473 DOI: 10.1097/sla.0000000000004650] [Citation(s) in RCA: 52] [Impact Index Per Article: 17.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
Abstract
OBJECTIVE To analyze all published prospective, randomized, and blinded clinical studies on the proficiency-based progression (PBP) training using objective performance metrics. BACKGROUND The benefit of PBP methodology to learning clinical skills in comparison to conventional training is not settled. METHODS Search of PubMed, Cochrane library's Central, EMBASE, MEDLINE, and Scopus databases, from inception to 1st March 2020. Two independent reviewers extracted the data. The Medical Education Research Study Quality Instrument (MERSQI) was used to assess the methodological quality of included studies. Results were pooled using biased corrected standardized mean difference and ratio-of-means. Summary effects were evaluated using a series of fixed and random effects models. The primary outcome was the number of procedural errors performed comparing PBP and non-PBP-based training pathways. Secondary outcomes were the number of procedural steps completed and the time to complete the task/procedure. RESULTS From the initial pool of 468 studies, 12 randomized clinical studies with a total of 239 participants were included in the analysis. In comparison to the non-PBP training, ratio-of-means results showed that PBP training reduced the number of performance errors by 60% (P < 0.001) and procedural time by 15% (P = 0.003) and increased the number of steps performed by 47% (P < 0.001). CONCLUSIONS AND RELEVANCE Our systematic review and meta-analysis confirms that PBP training in comparison to conventional or quality assured training improved trainees' performances, by decreasing procedural errors and procedural time, while increasing the number of correct steps taken when compared to standard simulation-based training.
Collapse
Affiliation(s)
- Elio Mazzone
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy
- ORSI Academy, Melle, Belgium
| | - Stefano Puliatti
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Marco Amato
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Brendan Bunting
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Northern Ireland, UK
| | - Bernardo Rocco
- Department of Urology, University of Modena and Reggio Emilia, Modena, Italy
| | - Francesco Montorsi
- Division of Oncology/Unit of Urology, URI, IRCCS Ospedale San Raffaele, Milan, Italy; Vita-Salute San Raffaele University, Milan, Italy
| | - Alexandre Mottrie
- ORSI Academy, Melle, Belgium
- Department of Urology, OLV, Aalst, Belgium
| | - Anthony G Gallagher
- ORSI Academy, Melle, Belgium
- School of Medicine, Faculty of Life and Health Sciences, Ulster University, Northern Ireland, UK
- Faculty of Medicine, KU Leuven, Leuven, Belgium
| |
Collapse
|
11
|
Objective Structured Assessment of technical skill (OSATS) in the Surgical Skills and Technology Elective Program (SSTEP): Comparison of peer and expert raters. Am J Surg 2021; 223:276-279. [PMID: 33838866 DOI: 10.1016/j.amjsurg.2021.03.064] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/28/2020] [Revised: 02/26/2021] [Accepted: 03/20/2021] [Indexed: 11/21/2022]
Abstract
BACKGROUND A major challenge with Competency Based Medical Education (CBME) is that of increased assessment burden on faculty. To reduce this burden, the accuracy and reliability of peer-assessment for surgical skills requires further exploration. METHODS Forty-two second year medical students were video recorded while performing a simple interrupted suture and an instrument tie. Four novice raters underwent a short training session on the use of the Objective Structured Assessment of Technical Skills (OSATS) checklists. Videos of the suturing task were then independently assessed by the four novice raters and two expert raters on two occasions. Agreement between novice and expert rater scores was calculated using the intraclass correlation coefficient (ICC). RESULTS For both simple interrupted suturing (ICC = 0.78, CI = 0.66-0.86, p < 0.001) and instrument ties (ICC = 0.87, CI = 0.80-0.92, p < 0.001), there was good agreement between novice and expert raters. CONCLUSIONS Novice raters can be taught to use the OSATS checklists to assess peers on simple suturing and instrument tying tasks.
Collapse
|
12
|
Tejos R, Crovari F, Achurra P, Avila R, Inzunza M, Jarry C, Martinez J, Riquelme A, Alseidi A, Varas J. Video-Based Guided Simulation without Peer or Expert Feedback is Not Enough: A Randomized Controlled Trial of Simulation-Based Training for Medical Students. World J Surg 2020; 45:57-65. [PMID: 32892271 DOI: 10.1007/s00268-020-05766-x] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 08/07/2020] [Indexed: 11/30/2022]
Abstract
BACKGROUND Feedback is a pivotal cornerstone and a challenge in psychomotor training. There are different teaching methodologies; however, some may be less effective. METHODS A prospective randomized controlled trial was conducted in 130 medical students to compare the effectiveness of the video-guided learning (VLG), peer-feedback (PFG) and the expert feedback (EFG) for teaching suturing skills. The program lasted 4 weeks. Students were recorded making 3-simple stitches (pre-assessment and post-assessment). The primary outcome was a global scale (OSATS). The secondary outcomes were performance time, specific rating scale (SRS) and the impact of the intervention (IOI), defined as the variation between the final and initial OSATS and SRS scores. RESULTS No significant differences were found between PFG and EFG in post-assessment results of OSATS, SRS scores or in the IOI for OSATS and SRS scores. Post-assessment results of PFG and EFG were significantly superior to VLG in OSATS and SRS scores [(19.8 (18.5-21); 16.6 (15.5-17.5)) and (20.3 (19.88-21); 16.8 (16-17.5)) vs (15.7 (15-16); 13.3 (12.5-14)) (p < 0.05)], respectively. The results of PFG and EFG were significantly superior to VLG in the IOI for OSATS [7 (4.5-9) and 7.4 (4.88-10) vs 3.5 (1.5-6) (p < 0.05)] and SRS scores [5.4 (3.5-7) and 6.3 (4-8.5) vs 3.1 (1.13-4.88) (p < 0.05)], respectively. CONCLUSION The video-guided learning methodology without any kind of feedback is not enough for teaching suturing skills compared to expert or peer feedback. The peer feedback methodology appears to be a viable alternative to handling the emerging demands in medical education.
Collapse
Affiliation(s)
- Rodrigo Tejos
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile
| | - Fernando Crovari
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile
| | - Pablo Achurra
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile
| | - Ruben Avila
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile
| | - Martín Inzunza
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile
| | - Cristian Jarry
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile
| | - Jorge Martinez
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile
| | - Arnoldo Riquelme
- Department of Gastroenterology and Centre for Medical Education, Faculty of Medicine, Pontificia Universidad Católica de Chile, Diagonal Paraguay 362, 8330077, Santiago, Chile
| | - Adnan Alseidi
- Department of Surgery, University of California San Francisco, 400 Parnassus Ave, San Francisco, CA, 94143, USA
| | - Julian Varas
- Experimental Surgery and Simulation Center, Department of Digestive Surgery, Faculty of Medicine, Pontificia Universidad Católica de Chile, Marcoleta 377, 8330024, Santiago, Chile.
| |
Collapse
|