1
|
White A, Turner SR, Moon MC, Zheng B. Assessment of a Novel, Adjustable Task Trainer for Cardiac Surgical Skills. Simul Healthc 2024; 19:333-339. [PMID: 37851383 DOI: 10.1097/sih.0000000000000753] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/19/2023]
Abstract
INTRODUCTION A recent needs assessment in Canadian cardiac surgery programs identified the desire for a coronary artery bypass (CABG) and aortic valve replacement (AVR) simulation model for home practice. We aimed to develop and assess a portable, adjustable task trainer for cardiac surgical skills with high functional task alignment. METHODS Intraoperative measurements were taken from patients undergoing elective CABG and AVR (N = 30). Measurements were taken in 3 axes and used to create a chest cavity that resembles the mediastinal constraints of a patient undergoing CABG and AVR. The task trainer is adjustable on the following 3 levels: (1) size of the incision, (2) depth of the chest, and (3) relative position of coronary artery or aortic valve model within the chest. Three groups (novices, intermediates, and experts) of cardiac surgery members evaluated the task trainer for functional task alignment and construct validity. RESULTS The CABG and AVR model had high functional task alignment. There was a high satisfaction for both models and all participants would recommend the AVR and CABG model as an educational tool. Performance time significantly differed between the groups for both models (CABG: P = 0.032 and AVR: P = 0.001), as well as number of errors (CABG: P = 0.04 and AVR: P = 0.043). CONCLUSIONS Using real patient data, we were able to develop an adjustable task trainer for training principles of CABG and AVR. Our pilot study provides preliminary sources of evidence for validity and future study will look to assess transferability of skill to the operating room.
Collapse
Affiliation(s)
- Abigail White
- From the Department of Surgery, University of Alberta, Edmonton, Canada
| | | | | | | |
Collapse
|
2
|
O'Keeffe DA, Traynor O, Tekian A, Park YS. Evaluating the Validity of National Multiassessment System in Postgraduate Surgical Training: A Retrospective Cohort Study. JOURNAL OF SURGICAL EDUCATION 2024; 81:1709-1719. [PMID: 39299054 DOI: 10.1016/j.jsurg.2024.08.008] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/23/2024] [Revised: 08/11/2024] [Accepted: 08/20/2024] [Indexed: 09/22/2024]
Abstract
OBJECTIVE The objective of this study was to evaluate the validity evidence supporting the use and interpretation of a multifaceted assessment system in the early years of surgical training. DESIGN This was a national retrospective cohort study analyzing the validity and reliability of an assessment process for surgical residents over a 2-year period. Data from all elements of the assessment process was evaluated using Messick's unified validity framework. Assessments were categorized as Workplace-based, Structured assessment performed in the academic center and Multiple Mini Interview. SETTING Our Institution is a health sciences university and the body responsible for the training and certification of all surgeons in our national program. Residents on the Core Surgical Training program undergo multiple assessments over the first 2 years of postgraduate training, both in the workplace and the academic training center, which inform their progression into higher surgical training in their chosen specialty. PARTICIPANTS Data was collected from 2 cohorts of the entire population of postgraduate trainees nationally (N = 114). RESULTS Best practice standards for educational testing aligned with the results supporting the use of this assessment process. Findings indicate a robust assessment system, demonstrating validity evidence in content, response process, internal structure, relations to other variables, and consequences. Composite score reliability of the assessment was 0.89 which demonstrates a highly reliable process. Correlation between workplace-based assessments and standardized tests performed in the simulation setting was also very high (0.93). CONCLUSIONS The Core Training Assessment System (CTAS) provides a psychometrically rigorous system to measure trainee competence during the initial years of training. Residency programs of all sizes can replicate the methods described here to demonstrate the validity of their assessment processes, thereby being able to stand over decisions on surgical competency.
Collapse
Affiliation(s)
- Dara A O'Keeffe
- National Surgical and Clinical Skills Centre, Royal College of Surgeons in Ireland University of Medicine and Health Sciences, Dublin, Ireland.
| | - Oscar Traynor
- National Surgical and Clinical Skills Centre, Royal College of Surgeons in Ireland University of Medicine and Health Sciences, Dublin, Ireland
| | - Ara Tekian
- Department of Medical Education, College of Medicine, University of Illinois, Chicago, Illinois
| | - Yoon Soo Park
- Department of Medical Education, College of Medicine, University of Illinois, Chicago, Illinois
| |
Collapse
|
3
|
Muir TM, Pruszynski J, Kho KA, Ramirez CI, Donnellan NM, Chao L. Video-Based Coaching for Laparoscopic Salpingectomy: A Randomized Controlled Trial. Obstet Gynecol 2024; 144:358-365. [PMID: 39053009 DOI: 10.1097/aog.0000000000005677] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2024] [Accepted: 05/23/2024] [Indexed: 07/27/2024]
Abstract
OBJECTIVE To evaluate the effect of video-based coaching on the gynecology resident performance of laparoscopic salpingectomy. METHODS PGY-1 and PGY-2 residents were randomized before their gynecology rotations to standard gynecology curriculum (control group) or standard curriculum plus two video-coaching sessions by a fellowship-trained minimally invasive gynecologic surgeon (VBC group). Residents were video recorded intraoperatively performing three unilateral laparoscopic salpingectomies. Participants in the VBC group were coached between the procedures. The primary outcome was the improvement in modified GOALS (Global Operative Assessment of Laparoscopic Skills) and OSA-LS (Objective Structured Assessment of Laparoscopic Salpingectomy) scores, compared with baseline, in the VBC and control groups, with videos independently graded by three blinded minimally invasive gynecologic surgeons. A minimum sample size of 18 participants (nine per group) was needed to achieve 90% power to detect a difference of 5.0 points. RESULTS From October 2021 to December 2022, 28 PGY-1 and PGY-2 residents completed the study with 14 participants per group. Baseline characteristics were similar between groups. In the VBC group, modified GOALS scores significantly improved by 3.0 points from video 1 to video 2 ( P =.04) and by 3.2 points from video 1 to video 3 ( P =.02). Modified OSA-LS scores also increased significantly in the VBC group, by 6.1 points from video 1 to video 3 ( P =.02). In the control group, modified GOALS and OSA-LS scores improved from baseline but were not significant ( P =.2, P =.5). Video-based coaching also enhanced resident comfort and confidence in performing laparoscopic surgery. CONCLUSION Video-based coaching improves resident performance of laparoscopic salpingectomy and can be used as an adjunct to resident surgical training. CLINICAL TRIAL REGISTRATION ClinicalTrials.gov , NCT05103449.
Collapse
Affiliation(s)
- Teale M Muir
- Department of Obstetrics & Gynecology, University of Texas Southwestern Medical Center, Dallas, and the Department of Gynecologic Surgery and Obstetrics, Brooke Army Medical Center, Fort Sam Houston, Texas; the Department of Obstetrics, Gynecology and Reproductive Sciences, Magee-Womens Hospital, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania; and the Department of Obstetrics and Gynecology, Stanford University School of Medicine, Stanford, California
| | | | | | | | | | | |
Collapse
|
4
|
Soenens G, Gorden L, Doyen B, Wheatcroft M, de Mestral C, Palter V, Van Herzeele I. Editor's Choice - Development and Testing of Step, Error, and Event Frameworks to Evaluate Technical Performance in Peripheral Endovascular Interventions. Eur J Vasc Endovasc Surg 2024; 68:227-235. [PMID: 38492630 DOI: 10.1016/j.ejvs.2024.03.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2023] [Revised: 12/22/2023] [Accepted: 03/11/2024] [Indexed: 03/18/2024]
Abstract
OBJECTIVE Tools for endovascular performance assessment are necessary in competency based education. This study aimed to develop and test a detailed analysis tool to assess steps, errors, and events in peripheral endovascular interventions (PVI). METHODS A modified Delphi consensus was used to identify steps, errors, and events in iliac-femoral-popliteal endovascular interventions. International experts in vascular surgery, interventional radiology, cardiology, and angiology were identified, based on their scientific track record. In an initial open ended survey round, experts volunteered a comprehensive list of steps, errors, and events. The items were then rated on a five point Likert scale until consensus was reached with a pre-defined threshold (Cronbach's alpha > 0.7) and > 70% expert agreement. An experienced endovascular surgeon applied the finalised frameworks on 10 previously videorecorded elective PVI cases. RESULTS The expert consensus panel was formed by 28 of 98 invited proceduralists, consisting of three angiologists, seven interventional radiologists, five cardiologists, and 13 vascular surgeons, with 29% from North America and 71% from Europe. The Delphi process was completed after three rounds (Cronbach's alpha; αsteps = 0.79; αerrors = 0.90; αevents = 0.90), with 15, 26, and 18 items included in the final step (73 - 100% agreement), error (73 - 100% agreement), and event (73 - 100% agreement) frameworks, respectively. The median rating time per case was 4.3 hours (interquartile range [IQR] 3.2, 5 hours). A median of 55 steps (IQR 40, 67), 27 errors (IQR 21, 49), and two events (IQR 1, 6) were identified per case. CONCLUSION An evaluation tool for the procedural steps, errors, and events in iliac-femoral-popliteal endovascular procedures was developed through a modified Delphi consensus and applied to recorded intra-operative data to identify hazardous steps, common errors, and events. Procedural mastery may be promoted by using the frameworks to provide endovascular proceduralists with detailed technical performance feedback.
Collapse
Affiliation(s)
- Gilles Soenens
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium. https://www.twitter.com/GillesSoenens
| | - Lauren Gorden
- Division of Vascular Surgery, University of Toronto, Canada; International Centre for Surgical Safety, Li Ka Shing Knowledge Institute of St. Michael's Hospital, Toronto, Canada; Biomedical Engineering (BME), University of Toronto, Canada
| | - Bart Doyen
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium
| | - Mark Wheatcroft
- Division of Vascular Surgery, University of Toronto, Canada; Department of Surgery, St. Michael's Hospital, Toronto, Canada
| | - Charles de Mestral
- Division of Vascular Surgery, University of Toronto, Canada; Department of Surgery, St. Michael's Hospital, Toronto, Canada
| | - Vanessa Palter
- International Centre for Surgical Safety, Li Ka Shing Knowledge Institute of St. Michael's Hospital, Toronto, Canada
| | - Isabelle Van Herzeele
- Department of Thoracic and Vascular Surgery, Ghent University Hospital, Ghent, Belgium.
| |
Collapse
|
5
|
Tesfai FM, Nagi J, Morrison I, Boal M, Olaitan A, Chandrasekaran D, Stoyanov D, Lanceley A, Francis N. Objective assessment tools in laparoscopic or robotic-assisted gynecological surgery: A systematic review. Acta Obstet Gynecol Scand 2024; 103:1480-1497. [PMID: 38610108 PMCID: PMC11266631 DOI: 10.1111/aogs.14840] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 02/28/2024] [Accepted: 03/19/2024] [Indexed: 04/14/2024]
Abstract
INTRODUCTION There is a growing emphasis on proficiency-based progression within surgical training. To enable this, clearly defined metrics for those newly acquired surgical skills are needed. These can be formulated in objective assessment tools. The aim of the present study was to systematically review the literature reporting on available tools for objective assessment of minimally invasive gynecological surgery (simulated) performance and evaluate their reliability and validity. MATERIAL AND METHODS A systematic search (1989-2022) was conducted in MEDLINE, Embase, PubMed, Web of Science in accordance with PRISMA. The trial was registered with the Prospective Register of Systematic Reviews (PROSPERO) ID: CRD42022376552. Randomized controlled trials, prospective comparative studies, prospective single-group (with pre- and post-training assessment) or consensus studies that reported on the development, validation or usage of assessment tools of surgical performance in minimally invasive gynecological surgery, were included. Three independent assessors assessed study setting and validity evidence according to a contemporary framework of validity, which was adapted from Messick's validity framework. Methodological quality of included studies was assessed using the modified medical education research study quality instrument (MERSQI) checklist. Heterogeneity in data reporting on types of tools, data collection, study design, definition of expertise (novice vs. experts) and statistical values prevented a meaningful meta-analysis. RESULTS A total of 19 746 titles and abstracts were screened of which 72 articles met the inclusion criteria. A total of 37 different assessment tools were identified of which 13 represented manual global assessment tools, 13 manual procedure-specific assessment tools and 11 automated performance metrices. Only two tools showed substantive evidence of validity. Reliability and validity per tool were provided. No assessment tools showed direct correlation between tool scores and patient related outcomes. CONCLUSIONS Existing objective assessment tools lack evidence on predicting patient outcomes and suffer from limitations in transferability outside of the research environment, particularly for automated performance metrics. Future research should prioritize filling these gaps while integrating advanced technologies like kinematic data and AI for robust, objective surgical skill assessment within gynecological advanced surgical training programs.
Collapse
Affiliation(s)
- Freweini Martha Tesfai
- The Griffin InstituteNorthwick Park & St Marks' HospitalLondonUK
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS)University College LondonLondonUK
| | | | - Iona Morrison
- Yeovil District HospitalSomerset Foundation NHS TrustYeovilUK
| | - Matt Boal
- The Griffin InstituteNorthwick Park & St Marks' HospitalLondonUK
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS)University College LondonLondonUK
| | | | - Dhivya Chandrasekaran
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Department of Gynecological OncologyUniversity College of London HospitalsLondonUK
| | - Danail Stoyanov
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Wellcome/EPSRC Center for Interventional and Surgical Sciences (WEISS)University College LondonLondonUK
| | - Anne Lanceley
- EGA Institute for Women's HealthUniversity College LondonLondonUK
| | - Nader Francis
- The Griffin InstituteNorthwick Park & St Marks' HospitalLondonUK
- EGA Institute for Women's HealthUniversity College LondonLondonUK
- Yeovil District HospitalSomerset Foundation NHS TrustYeovilUK
| |
Collapse
|
6
|
Podolsky DJ, Yasabala B, Wong Riff KW, Fisher DM. Evaluation of a high-fidelity cleft alveolar bone graft simulator. J Plast Reconstr Aesthet Surg 2024; 93:269-278. [PMID: 38728899 DOI: 10.1016/j.bjps.2024.04.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2024] [Revised: 03/03/2024] [Accepted: 04/05/2024] [Indexed: 05/12/2024]
Abstract
INTRODUCTION Cleft alveolar bone graft surgery is technically challenging to perform as well as difficult to learn and teach. A high-fidelity cleft alveolar bone graft simulator was previously developed. However, further evaluation of the simulator is necessary to assess its efficacy. METHODS Two cleft simulation workshops were conducted in which participants were led through a simulated cleft alveolar bone graft. The first simulation workshop involved six plastic surgery trainees. The second workshop involved 43 practicing cleft surgeons. The participants were provided with a Likert-type questionnaire assessing the simulators' features, realism, and value as a training tool. The change in self-reported confidence was assessed by providing each participant with a pre- and post-simulation confidence questionnaire. RESULTS There was overall agreement in the realism of the simulators' features (average score of 4.67 and 3.80 out of 5 for the trainees and surgeons, respectively). There was overall agreement to strong agreement in the simulators value as a training tool (average score of 5 and 4.43 out of 5 for the trainees and surgeons, respectively). The self-reported confidence increased for all questionnaire items for both the trainees and surgeons. This was significant (p < 0.05) for five out of eight and all questionnaire items for the trainees and surgeons, respectively. The magnitude of the confidence increase was generally greater for less experienced participants. CONCLUSION The cleft alveolar bone graft simulator was found to be realistic and valuable as a training platform. Use of the simulator improved self-reported confidence in cleft alveolar bone graft surgery.
Collapse
Affiliation(s)
- Dale J Podolsky
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada; Posluns Center for Image Guided Innovation and Therapeutic Intervention, Toronto, Ontario, Canada; Simulare Medical, Division of Smile Train, Toronto, Ontario, Canada.
| | | | - Karen W Wong Riff
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - David M Fisher
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| |
Collapse
|
7
|
Hertz P, Rattenborg S, Haug TR, Houlind K, Konge L, Bjerrum F. Training and assessment for colorectal surgery and appendicectomy- a systematic review. Colorectal Dis 2024; 26:597-608. [PMID: 38396135 DOI: 10.1111/codi.16905] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/21/2023] [Revised: 11/07/2023] [Accepted: 12/28/2023] [Indexed: 02/25/2024]
Abstract
AIM There is currently an increased focus on competency-based training, in which training and assessment play a crucial role. The aim of this systematic review is to create an overview of hands-on training methods and assessment tools for appendicectomy and colon and rectal surgery procedures using either an open, laparoscopic or robot-assisted approach. METHOD A systematic review of Medline, Embase, Cochrane and Scopus databases was conducted following the PRISMA guidelines. We conducted the last search on 9 March 2023. All published papers describing hands-on training, evaluation of performance data and development of assessment tools were eligible. The quality of studies and the validity evidence of assessment tools are reported. RESULTS Fifty-one studies were identified. Laparoscopic assessment tools are abundant, but the literature still lacks good-quality assessment tools for open appendicectomy, robotic colectomy and open rectal surgery. Overall, there is a lack of discussion regarding the establishment of pass/fail standards and the consequences of assessment. Virtual reality simulation is used more for appendicectomy than colorectal procedures. Only a few of the studies investigating training were of acceptable quality. There is a need for high-quality studies in open and robotic-assisted colon surgery and all approaches to rectal surgery. CONCLUSION This review provides an overview of current training methods and assessment tools and identifies where more research is needed based on the quality of the studies and the current validity evidence.
Collapse
Affiliation(s)
- Peter Hertz
- Department of Surgery, Hospital Lillebaelt, University of Southern Denmark, Kolding, Denmark
- Department of Regional Health Research, University of Southern Denmark, Kolding, Denmark
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, Copenhagen, Denmark
| | - Søren Rattenborg
- Department of Regional Health Research, University of Southern Denmark, Kolding, Denmark
- Department of Surgery, Hospital Lillebaelt Vejle, Colorectal Cancer Center South, University of Southern Denmark DK, Kolding, Denmark
| | - Tora R Haug
- Department of Surgery, Gødstrup Hospital, Herning, Denmark
- Aarhus University, Aarhus, Denmark
| | - Kim Houlind
- Department of Regional Health Research, University of Southern Denmark, Kolding, Denmark
- Department of Vascular Surgery, Hospital Lillebaelt, University of Southern Denmark, Kolding, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Flemming Bjerrum
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR and Education, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
- Gastrounit, Surgical Section, Copenhagen University Hospital - Amager and Hvidovre, Hvidovre, Denmark
| |
Collapse
|
8
|
Sachdeva AK, Tekian A, Park YS, Cheung JJH. Surgical skills training for practicing surgeons founded on established educational theories and frameworks. MEDICAL TEACHER 2024; 46:556-563. [PMID: 37813106 DOI: 10.1080/0142159x.2023.2262101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/11/2023]
Abstract
Following completion of structured training in residency or fellowship programs, surgeons need to acquire new skills throughout their careers to address changes in their surgical practices, and to perform new procedures and use new technologies as these are developed. The opportunities available to surgeons to acquire the new skills, safely introduce these skills into surgical practice, and then maintain the skills, vary greatly and may not fully support a surgeon's needs. The article shares background information relating to the vexing problems, and describes established educational theories and frameworks as well as specific frameworks relating to surgical skills training and verification that may be used in designing and implementing comprehensive and impactful surgical skills training programs aimed at practicing surgeons. A number of practical strategies based on these theories and frameworks are highlighted. National professional organizations and academic medical centers need to play a critical role in pursuing such efforts, which will remain pivotal in supporting the professional work of surgeons and in providing optimal surgical care well into the future.
Collapse
Affiliation(s)
- Ajit K Sachdeva
- Division of Education, American College of Surgeons, Chicago, IL, USA
| | - Ara Tekian
- Department of Medical Education, University of Illinois College of Medicine, Chicago, IL, USA
| | - Yoon Soo Park
- Department of Medical Education, University of Illinois College of Medicine, Chicago, IL, USA
| | - Jeffrey J H Cheung
- Department of Medical Education, University of Illinois College of Medicine, Chicago, IL, USA
| |
Collapse
|
9
|
Liu ZY, Zhong Q, Wang ZB, Shang-Guan ZX, Lu J, Li YF, Huang Q, Wu J, Li P, Xie JW, Chen QY, Huang CM, Zheng CH. Appraisal of surgical outcomes and oncological efficiency of intraoperative adverse events in robotic radical gastrectomy for gastric cancer. Surg Endosc 2024; 38:2027-2040. [PMID: 38424283 DOI: 10.1007/s00464-024-10736-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/07/2023] [Accepted: 01/28/2024] [Indexed: 03/02/2024]
Abstract
BACKGROUND Surgical quality control is a crucial determinant of evaluating the tumor efficacy. OBJECTIVE To assess the ClassIntra grade for quality control and oncological outcomes of robotic radical surgery for gastric cancer (GC). METHODS Data of patients undergoing robotic radical surgery for GC at a high-volume center were retrospectively analyzed. Patients were categorized into two groups, the intraoperative adverse event (iAE) group and the non-iAE group, based on the occurrence of intraoperative adverse events. The iAEs were further classified into five sublevels (ranging from I to V according to severity) based on the ClassIntra grade. Surgical performance was assessed using the Objective Structured Assessment of Technical Skill (OSATS) and the General Error Reporting Tool. RESULTS This study included 366 patients (iAE group: n = 72 [19.7%] and non-iAE group: n = 294 [80.3%]). The proportion of ClassIntra grade II patients was the highest in the iAE group (54.2%). In total and distal gastrectomies, iAEs occurred most frequently in the suprapancreatic area (50.0% and 54.8%, respectively). In total gastrectomy, grade IV iAEs were most common during lymph node dissection in the splenic hilum area (once for bleeding [grade IV] and once for injury [grade IV]). The overall survival (OS) and disease-free survival of the non-iAE group were significantly better than those of the iAE group (Log rank P < 0.001). Uni- and multi-variate analyses showed that iAEs were key prognostic indicators, independent of tumor stage and adjuvant chemotherapy (P < 0.001). CONCLUSION iAEs in patients who underwent robotic radical gastrectomy significantly correlated with the occurrence of postoperative complications and a poor long-term prognosis. Therefore, utilization and inclusion of ClassIntra grading as a crucial surgical quality control and prognostic indicator in the routine surgical quality evaluation system are recommended.
Collapse
Affiliation(s)
- Zhi-Yu Liu
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
| | - Qing Zhong
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
| | - Zeng-Bin Wang
- Department of Immunology, School of Basic Medical Sciences, Fujian Medical University, Fuzhou, China
| | - Zhi-Xin Shang-Guan
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
| | - Jun Lu
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer, Fujian Medical University, Fuzhou, China
| | - Yi-Fan Li
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer, Fujian Medical University, Fuzhou, China
| | - Qiang Huang
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer, Fujian Medical University, Fuzhou, China
| | - Ju Wu
- Department of General Surgery, Affiliated Zhongshan Hospital of Dalian University, Dalian, China
| | - Ping Li
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer, Fujian Medical University, Fuzhou, China
| | - Jian-Wei Xie
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China
| | - Qi-Yue Chen
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China.
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China.
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer, Fujian Medical University, Fuzhou, China.
| | - Chang-Ming Huang
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China.
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China.
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer, Fujian Medical University, Fuzhou, China.
| | - Chao-Hui Zheng
- Department of Gastric Surgery, Fujian Medical University Union Hospital, No. 29 Xinquan Road, Fuzhou, 350001, Fujian Province, China.
- Department of General Surgery, Fujian Medical University Union Hospital, Fuzhou, China.
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer, Fujian Medical University, Fuzhou, China.
| |
Collapse
|
10
|
White A, Zarzycki A, Bisleri G. Simulating mitral repair: lessons learned. Curr Opin Cardiol 2024; 39:73-78. [PMID: 38305721 DOI: 10.1097/hco.0000000000001106] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/03/2024]
Abstract
PURPOSE OF REVIEW With the growing complexity of cardiac surgical cases, increased focus on patient safety, and minimally invasive techniques, simulation-based training has experienced a renaissance. This review highlights important elements of simulation-based training, focusing specifically on available simulators for mitral valve repair and the uses for simulation. RECENT FINDINGS Referring to simulators as being high or low fidelity is oversimplified. Fidelity is a multifactorial concept, and for surgical task trainers, structural and functional fidelity should be discussed. For mitral valve repair, there are a spectrum of simulators, including tissue-based models, bench-top models, and hybrid models. All these simulator modalities serve a role in training if they align with predetermined objectives. There have been advancements in mitral valve repair simulation, notably patient-specific 3D printed silicone replicas of disease. SUMMARY There is evidence to support that simulation improves performance in the simulated environment, but future investigation should look to determine whether simulation improves performance in the clinical setting and ultimately patient outcomes.
Collapse
|
11
|
Nikolian VC, Camacho D, Earle D, Lehmann R, Nau P, Ramshaw B, Stulberg J. Development and preliminary validation of a new task-based objective procedure-specific assessment of inguinal hernia repair procedural safety. Surg Endosc 2024; 38:1583-1591. [PMID: 38332173 DOI: 10.1007/s00464-024-10677-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Accepted: 12/30/2023] [Indexed: 02/10/2024]
Abstract
BACKGROUND Surgical videos coupled with structured assessments enable surgical training programs to provide independent competency evaluations and align with the American Board of Surgery's entrustable professional activities initiative. Existing assessment instruments for minimally invasive inguinal hernia repair (IHR) have limitations with regards to reliability, validity, and usability. A cross-sectional study of six surgeons using a novel objective, procedure-specific, 8-item competency assessment for minimally invasive inguinal hernia repair (IHR-OPSA) was performed to assess inter-rater reliability using a "safe" vs. "unsafe" scoring rubric. METHODS The IHR-OPSA was developed by three expert IHR surgeons, field tested with five IHR surgeons, and revised based upon feedback. The final instrument included: (1) incision/port placement; (2) dissection of peritoneal flap (TAPP) or dissection of peritoneal flap (TEP); (3) exposure; (4) reducing the sac; (5) full dissection of the myopectineal orifice; (6) mesh insertion; (7) mesh fixation; and (8) operation flow. The IHR-OPSA was applied by six expert IHR surgeons to 20 IHR surgical videos selected to include a spectrum of hernia procedures (15 laparoscopic, 5 robotic), anatomy (14 indirect, 5 direct, 1 femoral), and Global Case Difficulty (easy, average, hard). Inter-rater reliability was assessed against Gwet's AC2. RESULTS The IHR-OPSA inter-rater reliability was good to excellent, ranging from 0.65 to 0.97 across the eight items. Assessments of robotic procedures had higher reliability with near perfect agreement for 7 of 8 items. In general, assessments of easier cases had higher levels of agreement than harder cases. CONCLUSIONS A novel 8-item minimally invasive IHR assessment tool was developed and tested for inter-rater reliability using a "safe" vs. "unsafe" rating system with promising results. To promote instrument validity the IHR-OPSA was designed and evaluated within the context of intended use with iterative engagement with experts and testing of constructs against real-world operative videos.
Collapse
Affiliation(s)
- Vahagn C Nikolian
- Department of Surgery, Oregon Health & Science University, 3181 S.W. Sam Jackson Park Rd., Portland, OR, 97239, USA.
| | - Diego Camacho
- Minimally Invasive and Endoscopic Surgery at Montefiore Medical Center, New York, NY, USA
| | - David Earle
- New England Hernia Center, Lowell, MA, USA
- Tufts University School of Medicine, Boston, MA, USA
| | - Ryan Lehmann
- Department of Surgery, Section of Bariatric Surgery, University of Iowa Hospitals & Clinics, Iowa City, IA, USA
| | - Peter Nau
- Department of Surgery, Section of Bariatric Surgery, University of Iowa Hospitals & Clinics, Iowa City, IA, USA
| | - Bruce Ramshaw
- CQInsights PBC, Knoxville, TN, USA
- Caresyntax Corporation, Boston, MA, USA
| | - Jonah Stulberg
- Department of Surgery, McGovern Medical School University of Texas Health Science Center at Houston, Houston, TX, USA
| |
Collapse
|
12
|
Podolsky DJ, Murphy BD, Ahmad J, Fisher DM, Wong Riff KWY, Drake JM, Forrest CR. Development and Evaluation of a High-Fidelity Rhinoplasty Simulator. Plast Reconstr Surg 2024; 153:619-622. [PMID: 37159902 DOI: 10.1097/prs.0000000000010636] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/11/2023]
Abstract
SUMMARY Rhinoplasty is a challenging procedure with a steep learning curve. Surgical simulators provide a safe platform to gain hands-on experience without compromising patient outcomes. Therefore, rhinoplasty is an ideal procedure to benefit from an effective surgical simulator. A high-fidelity rhinoplasty simulator was developed using three-dimensional computer modeling, three-dimensional printing, and polymer techniques. The simulator was tested by six surgeons with experience in rhinoplasty to assess realism, anatomic accuracy, and value as a training tool. The surgeons performed common rhinoplasty techniques and were provided a Likert-type questionnaire assessing the anatomic features of the simulator. A variety of surgical techniques were performed successfully using the simulator, including open and closed approaches. Bony techniques performed included endonasal osteotomies and rasping. Submucous resection with harvest of septal cartilage, cephalic trim, and tip suturing, as well as grafting techniques including alar rim, columellar strut, spreader, and shield grafts, were performed successfully. Overall, there was agreement on the simulator's anatomic accuracy of bony and soft-tissue features. There was strong agreement on the simulator's overall realism and value as a training tool. The simulator provides a high-fidelity, comprehensive training platform to learn rhinoplasty techniques to augment real operating experience without compromising patient outcomes.
Collapse
Affiliation(s)
- Dale J Podolsky
- From the Divisions of Plastic and Reconstructive Surgery
- Posluns Center for Image Guided Innovation and Therapeutic Intervention
| | - Blake D Murphy
- Department of Surgery, University of Toronto, St. Michael's Hospital
| | - Jamil Ahmad
- Department of Surgery, University of Toronto, Plastic Surgery Clinic
| | - David M Fisher
- From the Divisions of Plastic and Reconstructive Surgery
| | | | - James M Drake
- Neurosurgery, The Hospital for Sick Children
- Posluns Center for Image Guided Innovation and Therapeutic Intervention
| | | |
Collapse
|
13
|
Zaga-Galante J, Tse R, Hopper RA, Arnold A, Fisher DM, Wong-Riff KW, Podolsky DJ. Bilateral Cleft lip Simulation. Cleft Palate Craniofac J 2024:10556656241230882. [PMID: 38354301 DOI: 10.1177/10556656241230882] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/16/2024] Open
Abstract
OBJECTIVE To evaluate the features, anatomic accuracy, and educational value of a high-fidelity bilateral cleft lip simulator. DESIGN Evaluation of the simulator by expert cleft surgeons after performing a simulated bilateral cleft lip repair. SETTING The simulator was evaluated by the surgeons during the Latin American Craniofacial Association meeting. PARTICIPANTS Eleven experienced cleft surgeons evaluated the simulator. The cleft surgeons were selected based on their availability during the meeting. INTERVENTIONS The participants performed a simulated bilateral cleft lip repair. They were each provided with a questionnaire assessing the simulator's features, realism and value as a training tool. MAIN OUTCOME MEASURE (S) The main outcome measure are the scores obtained from a Likert-type questionnaire assessing the simulators features, realism and value. RESULTS Overall, the surgeons agreed with the simulator's realism and anatomic accuracy (average score of 3.7 out of 5). Overall, the surgeons strongly agreed with the value of the simulator as a training tool (average score of 4.6 out of 5). CONCLUSIONS A high-fidelity bilateral cleft lip simulator was developed that is realistic and valuable as a training tool. The simulator provides a comprehensive training platform to gain hands-on experience in bilateral cleft lip repair before operating on real patients.
Collapse
Affiliation(s)
- Jonathan Zaga-Galante
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Raymond Tse
- Craniofacial Center, Seattle Children's Hospital, Seattle, Washington, USA
| | - Richard A Hopper
- Craniofacial Center, Seattle Children's Hospital, Seattle, Washington, USA
| | | | - David M Fisher
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Karen W Wong-Riff
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Dale J Podolsky
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
- Posluns Center for Image Guided Innovation and Therapeutic Intervention (PCIGITI), Toronto, Ontario, Canada
| |
Collapse
|
14
|
Li Y, Chan R, Menon MR, Ryan JF, Mador B, Campbell SM, Turner SR. Validity Evidence for Procedure-specific Competence Assessment Tools in Orthopaedic Surgery: A Scoping Review. J Am Acad Orthop Surg Glob Res Rev 2024; 8:01979360-202401000-00005. [PMID: 38237074 PMCID: PMC10796133 DOI: 10.5435/jaaosglobal-d-23-00065] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 11/05/2023] [Accepted: 11/07/2023] [Indexed: 01/22/2024]
Abstract
INTRODUCTION Competency-based training requires frequent assessment of residents' skills to determine clinical competence. This study reviews existing literature on procedure-specific competence assessment tools in orthopaedic surgery. METHODS A systematic search of eight databases up to May 2023 was conducted. Two reviewers independently assessed validity evidence and educational utility of each assessment tool and evaluated studies' methodological quality. RESULTS Database searching identified 2,556 unique studies for title and abstract screening. Full texts of 290 studies were reviewed; 17 studies met the inclusion criteria. Bibliography review identified another five studies, totaling 22 studies examining 24 assessment tools included in the analysis. These tools assessed various orthopaedic surgery procedures within trauma, sports medicine, spine, and upper extremity. Overall validity evidence was low across all studies, and was lowest for consequences and highest for content. Methodological quality of studies was moderate. Educational utility assessment was not explicitly done for most tools. DISCUSSION The paucity of current procedure-specific assessment tools in orthopaedic surgery lacks the validity evidence required to be used reliably in high-stake summative assessments. Study strengths include robust methodology and use of an evidence-based validity evidence framework. Poor-quality existing evidence is a limitation and highlights the need for evidence-based tools across more subspecialties.
Collapse
Affiliation(s)
- Yibo Li
- From the Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada (Dr. Li, Dr. Chan, Dr. Menon, Dr. Ryan, Dr. Mador, and Dr. Turner); the Western Upper Limb Facility, Sturgeon Community Hospital, St. Albert, Alberta, Canada (Dr. Chan); and the John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada (Ms. Campbell)
| | - Robert Chan
- From the Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada (Dr. Li, Dr. Chan, Dr. Menon, Dr. Ryan, Dr. Mador, and Dr. Turner); the Western Upper Limb Facility, Sturgeon Community Hospital, St. Albert, Alberta, Canada (Dr. Chan); and the John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada (Ms. Campbell)
| | - Matthew R.G. Menon
- From the Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada (Dr. Li, Dr. Chan, Dr. Menon, Dr. Ryan, Dr. Mador, and Dr. Turner); the Western Upper Limb Facility, Sturgeon Community Hospital, St. Albert, Alberta, Canada (Dr. Chan); and the John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada (Ms. Campbell)
| | - Joanna F. Ryan
- From the Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada (Dr. Li, Dr. Chan, Dr. Menon, Dr. Ryan, Dr. Mador, and Dr. Turner); the Western Upper Limb Facility, Sturgeon Community Hospital, St. Albert, Alberta, Canada (Dr. Chan); and the John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada (Ms. Campbell)
| | - Brett Mador
- From the Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada (Dr. Li, Dr. Chan, Dr. Menon, Dr. Ryan, Dr. Mador, and Dr. Turner); the Western Upper Limb Facility, Sturgeon Community Hospital, St. Albert, Alberta, Canada (Dr. Chan); and the John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada (Ms. Campbell)
| | - Sandra M. Campbell
- From the Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada (Dr. Li, Dr. Chan, Dr. Menon, Dr. Ryan, Dr. Mador, and Dr. Turner); the Western Upper Limb Facility, Sturgeon Community Hospital, St. Albert, Alberta, Canada (Dr. Chan); and the John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada (Ms. Campbell)
| | - Simon R. Turner
- From the Department of Surgery, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada (Dr. Li, Dr. Chan, Dr. Menon, Dr. Ryan, Dr. Mador, and Dr. Turner); the Western Upper Limb Facility, Sturgeon Community Hospital, St. Albert, Alberta, Canada (Dr. Chan); and the John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada (Ms. Campbell)
| |
Collapse
|
15
|
Pearce J, Chiavaroli N, Tavares W. On the use and abuse of metaphors in assessment. ADVANCES IN HEALTH SCIENCES EDUCATION : THEORY AND PRACTICE 2023; 28:1333-1345. [PMID: 36729196 DOI: 10.1007/s10459-022-10203-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/16/2022] [Accepted: 12/29/2022] [Indexed: 06/18/2023]
Abstract
This paper is motivated by a desire to advance assessment in the health professions through encouraging the judicious and productive use of metaphors. Through five specific examples (pixels, driving lesson/test, jury deliberations, signal processing, and assessment as a toolbox), we interrogate how metaphors are being used in assessment to consider what value they add to understanding and implementation of assessment practices. By unpacking these metaphors in action, we probe each metaphor's rationale and function, the gains each metaphor makes, and explore the unintended meanings they may carry. In summarizing common uses of metaphors, we elucidate how there may be both advantages and/or disadvantages. Metaphors can play important roles in simplifying, complexifying, communicating, translating, encouraging reflection, and convincing. They may be powerfully rhetorical, leading to intended consequences, actions, and other pragmatic outcomes. Although metaphors can be extremely helpful, they do not constitute thorough critique, justified evidence or argumentation. We argue that although metaphors have utility, they must be carefully considered if they are to serve assessment needs in intended ways. We should pay attention to how metaphors may be misinterpreted, what they ignore or unintentionally signal, and perhaps mitigate against this with anticipated corrections or nuanced qualifications. Failure to do so may lead to implementing practices that miss underlying and relevant complexities for assessment science and practice. Using metaphors requires careful attention with respect to their role, contributions, benefits and limitations. We highlight the value that comes from critiquing metaphors, and demonstrate the care required to ensure their continued utility.
Collapse
Affiliation(s)
- Jacob Pearce
- Tertiary Education, Australian Council for Educational Research, Camberwell, Australia.
| | - Neville Chiavaroli
- Tertiary Education, Australian Council for Educational Research, Camberwell, Australia
| | - Walter Tavares
- Department of Health and Society and Wilson Centre, Temerty Faculty of Medicine, University of Toronto, Toronto, ON., Canada
| |
Collapse
|
16
|
Khan R, Homsi H, Gimpaya N, Lisondra J, Sabrie N, Gholami R, Bansal R, Scaffidi MA, Lightfoot D, James PD, Siau K, Forbes N, Wani S, Keswani RN, Walsh CM, Grover SC. Validity evidence for observational ERCP competency assessment tools: a systematic review. Endoscopy 2023; 55:847-856. [PMID: 36822219 DOI: 10.1055/a-2041-7546] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 02/25/2023]
Abstract
BACKGROUND : Assessment of competence in endoscopic retrograde cholangiopancreatography (ERCP) is critical for supporting learning and documenting attainment of skill. Validity evidence supporting ERCP observational assessment tools has not been systematically evaluated. METHODS : We conducted a systematic search using electronic databases and hand-searching from inception until August 2021 for studies evaluating observational assessment tools of ERCP performance. We used a unified validity framework to characterize validity evidence from five sources: content, response process, internal structure, relations to other variables, and consequences. Each domain was assigned a score of 0-3 (maximum score 15). We assessed educational utility and methodological quality using the Accreditation Council for Graduate Medical Education framework and the Medical Education Research Quality Instrument, respectively. RESULTS : From 2769 records, we included 17 studies evaluating 7 assessment tools. Five tools were studied for clinical ERCP, one for simulated ERCP, and one for simulated and clinical ERCP. Validity evidence scores ranged from 2 to 12. The Bethesda ERCP Skills Assessment Tool (BESAT), ERCP Direct Observation of Procedural Skills Tool (ERCP DOPS), and The Endoscopic Ultrasound (EUS) and ERCP Skills Assessment Tool (TEESAT) had the strongest validity evidence, with scores of 10, 12, and 11, respectively. Regarding educational utility, most tools were easy to use and interpret, and required minimal additional resources. Overall methodological quality (maximum score 13.5) was strong, with scores ranging from 10 to 12.5. CONCLUSIONS : The BESAT, ERCP DOPS, and TEESAT had strong validity evidence compared with other assessments. Integrating tools into training may help drive learners' development and support competency decision making.
Collapse
Affiliation(s)
- Rishad Khan
- Division of Gastroenterology and Hepatology, Department of Medicine, University of Toronto, Toronto, Canada
| | - Hoomam Homsi
- Schulich School of Medicine and Dentistry, Western University, London, Canada
| | - Nikko Gimpaya
- Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada
| | - James Lisondra
- Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada
| | | | - Reza Gholami
- Division of Gastroenterology and Hepatology, Department of Medicine, University of Toronto, Toronto, Canada
- Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada
| | - Rishi Bansal
- Michael G. DeGroote School of Medicine, McMaster University, Hamilton, Canada
| | | | - David Lightfoot
- Health Science Library, Unity Health Toronto, St. Michael's Hospital, Toronto, Canada
| | - Paul D James
- Division of Gastroenterology, University Health Network, Toronto, Canada
| | - Keith Siau
- Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, United Kingdom
- Immunology and Immunotherapy, University of Birmingham College of Medical and Dental Sciences, Birmingham, United Kingdom
| | - Nauzer Forbes
- Department of Medicine, Division of Gastroenterology and Hepatology, University of Calgary, Calgary, Alberta, Canada
- Department of Community Health Sciences, University of Calgary, Calgary, Alberta, Canada
| | - Sachin Wani
- Division of Gastroenterology and Hepatology, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
| | - Rajesh N Keswani
- Division of Gastroenterology, Department of Medicine, Northwestern University, Chicago, Illinois, United States
| | - Catharine M Walsh
- The Wilson Centre, University of Toronto, Toronto, Canada
- SickKids Research and Learning Institute, The Hospital for Sick Children, Toronto, Canada
- Division of Gastroenterology, Hepatology, and Nutrition, The Hospital for Sick Children, Toronto, Canada
- Department of Paediatrics, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Samir C Grover
- Division of Gastroenterology and Hepatology, Department of Medicine, University of Toronto, Toronto, Canada
- Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada
- Department of Medicine, University of Toronto, Toronto, Canada
- Li Ka Shing Knowledge Institute, Toronto, Canada
| |
Collapse
|
17
|
Grüter AAJ, Van Lieshout AS, van Oostendorp SE, Henckens SPG, Ket JCF, Gisbertz SS, Toorenvliet BR, Tanis PJ, Bonjer HJ, Tuynman JB. Video-based tools for surgical quality assessment of technical skills in laparoscopic procedures: a systematic review. Surg Endosc 2023; 37:4279-4297. [PMID: 37099157 PMCID: PMC10234871 DOI: 10.1007/s00464-023-10076-z] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2023] [Accepted: 04/08/2023] [Indexed: 04/27/2023]
Abstract
BACKGROUND Quality of surgery has substantial impact on both short- and long-term clinical outcomes. This stresses the need for objective surgical quality assessment (SQA) for education, clinical practice and research purposes. The aim of this systematic review was to provide a comprehensive overview of all video-based objective SQA tools in laparoscopic procedures and their validity to objectively assess surgical performance. METHODS PubMed, Embase.com and Web of Science were systematically searched by two reviewers to identify all studies focusing on video-based SQA tools of technical skills in laparoscopic surgery performed in a clinical setting. Evidence on validity was evaluated using a modified validation scoring system. RESULTS Fifty-five studies with a total of 41 video-based SQA tools were identified. These tools were used in 9 different fields of laparoscopic surgery and were divided into 4 categories: the global assessment scale (GAS), the error-based assessment scale (EBAS), the procedure-specific assessment tool (PSAT) and artificial intelligence (AI). The number of studies focusing on these four categories were 21, 6, 31 and 3, respectively. Twelve studies validated the SQA tool with clinical outcomes. In 11 of those studies, a positive association between surgical quality and clinical outcomes was found. CONCLUSION This systematic review included a total of 41 unique video-based SQA tools to assess surgical technical skills in various domains of laparoscopic surgery. This study suggests that validated SQA tools enable objective assessment of surgical performance with relevance for clinical outcomes, which can be used for training, research and quality improvement programs.
Collapse
Affiliation(s)
- Alexander A J Grüter
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands.
- Cancer Center Amsterdam, Treatment and Quality of Life, Amsterdam, The Netherlands.
| | - Annabel S Van Lieshout
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
- Cancer Center Amsterdam, Treatment and Quality of Life, Amsterdam, The Netherlands
| | - Stefan E van Oostendorp
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
- Department of Surgery, Rode Kruis Ziekenhuis, Vondellaan 13, Beverwijk, The Netherlands
| | - Sofie P G Henckens
- Cancer Center Amsterdam, Treatment and Quality of Life, Amsterdam, The Netherlands
- Department of Surgery, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands
| | - Johannes C F Ket
- Medical Library, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands
| | - Suzanne S Gisbertz
- Department of Surgery, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands
| | | | - Pieter J Tanis
- Department of Surgery, Amsterdam UMC Location University of Amsterdam, Meibergdreef 9, Amsterdam, The Netherlands
- Department of Surgical Oncology and Gastrointestinal Surgery, Erasmus MC, Doctor Molewaterplein 40, Rotterdam, The Netherlands
| | - Hendrik J Bonjer
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
| | - Jurriaan B Tuynman
- Department of Surgery, Amsterdam UMC Location Vrije Universiteit Amsterdam, De Boelelaan 1117, Amsterdam, The Netherlands
| |
Collapse
|
18
|
Lu J, Huang JB, Wu D, Xie JW, Wang JB, Lin JX, Zheng CH, Huang CM, Li P. Factors affecting the quality of laparoscopic D2 lymph node dissection for gastric cancer: a cohort study from two randomized controlled trials. Int J Surg 2023; 109:1249-1256. [PMID: 37026819 PMCID: PMC10389415 DOI: 10.1097/js9.0000000000000290] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Accepted: 02/08/2023] [Indexed: 04/08/2023]
Abstract
BACKGROUND Laparoscopic D2 lymph node dissection (LND) for gastric cancer has a wide range and high difficulty. In the past, the quality of surgery was often judged by the time of the operation or the amount of blood loss, but the analysis based on surgical video was rarely reported. The purpose of this study was to analyze the relationship between the quality of laparoscopic D2 LND for gastric cancer and postoperative complications. METHODS The surgical video and clinicopathological data of 610 patients in two randomized controlled trials in our center from 2013 to 2016 were retrospectively analyzed. Klass-02-QC LND scale and general error score tool were used to quantitatively evaluate the intraoperative performance of D2 LND. Logistic regression was used to analyze the influencing factors of postoperative complications. RESULTS The overall incidence of complications (CD classification≥2) was 20.6%; the incidence of surgical complications was 6.9%. According to whether the LND score reached 44, patients were divided into a qualified group (73%) and a not-qualified group (27%). Event score (ES) by quartile was divided into grade 1 (21.7%), grade 2 (26%), grade 3 (28%), and grade 4 (24.3%) from low to high. Univariate logistic regression analysis showed that ES greater than or equal to 3, tumor size greater than or equal to 35 mm, and cTNM >II were independent risk factors for not-qualified LND. Male,tumor size greater than or equal to 35 mm and cTNM >II were independent risk factors for grade 4 ES. Not-qualified LND (OR=1.62, 95% CI: 1.16-3.89, P =0.021), grade 4 ES (OR=3.21, 95% CI: 1.52-3.90, P =0.035), and cTNM >II (OR=1.74, 95% CI: 1.39-7.33, P =0.041) were independent risk factors for postoperative surgical complications. CONCLUSIONS The qualification of LND and intraoperative events based on surgical video are the independent influencing factors of postoperative complications of laparoscopic gastric cancer surgery. Specialist training and teaching based on surgical video may help to improve the surgical skills of specialists and improve the postoperative outcome of patients.
Collapse
Affiliation(s)
- Jun Lu
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Jiao-Bao Huang
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Dong Wu
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Jian-Wei Xie
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Jia-Bin Wang
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Jian-Xian Lin
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Chao-Hui Zheng
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Chang-Ming Huang
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| | - Ping Li
- Department of Gastric Surgery, Fujian Medical University Union Hospital
- Key Laboratory of Ministry of Education of Gastrointestinal Cancer
- Fujian Key Laboratory of Tumor Microbiology, Fujian Medical University, Fuzhou, China
| |
Collapse
|
19
|
Park YS, Sachdeva AK, Liscum K, Alseidi A, Gesbeck M, Blair PG, Salcedo E, Sullivan M, Bordage G. The American College of Surgeons Entering Resident Readiness Assessment (ACS ERRA): Validity Evidence From a Three-Year National Study. Ann Surg 2023; 277:704-711. [PMID: 34954752 DOI: 10.1097/sla.0000000000005358] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE To gather validity evidence supporting the use and interpretation of scores from the American College of Surgeons Entering Resident Readiness Assessment (ACS ERRA) Program. SUMMARY AND BACKGROUND DATA ACS ERRA is an online formative assessment program developed to assess entering surgery residents' ability to make critical clinical decisions, and includes 12 clinical areas and 20 topics identified by a national panel of surgeon educators and residency program directors. METHODS Data from 3 national testing administrations of ACS ERRA (2018-2020) were used to gather validity evidence regarding content, response process, internal structure (reliability), relations to other variables, and consequences. RESULTS Over the 3 administrations, 1975 surgery residents participated from 125 distinct residency programs. Overall scores [Mean = 64% (SD = 7%)] remained consistent across the 3 years ( P = 0.670). There were no significant differences among resident characteristics (gender, age, international medical graduate status). The mean case discrimination index was 0.54 [SD = 0.15]. Kappa inter-rater reliability for scoring was 0.87; the overall test score reliability (G-coefficient) was 0.86 (Ф-coefficient = 0.83). Residents who completed residency readiness programs had higher ACS ERRA scores (66% versus 63%, Cohen's d = 0.23, P < 0.001). On average, 15% of decisions made (21/140 per test) involved potentially harmful actions. Variability in scores from graduating medical schools (7%) carried over twice as much weight than from matched residency programs (3%). CONCLUSIONS ACS ERRA scores provide valuable information to entering surgery residents and surgery program directors to aid in development of individual and group learning plans.
Collapse
Affiliation(s)
- Yoon Soo Park
- Massachusetts General Hospital, Harvard Medical School, Boston, MA
- Department of Medical Education, University of Illinois at Chicago, Chicago, IL
| | - Ajit K Sachdeva
- Division of Education, American College of Surgeons, Chicago, IL
| | - Kathy Liscum
- Division of Education, American College of Surgeons, Chicago, IL
| | - Adnan Alseidi
- Department of Surgery, University of California San Francisco, San Francisco, CA
| | - Melissa Gesbeck
- Division of Education, American College of Surgeons, Chicago, IL
| | | | - Edgardo Salcedo
- Department of Surgery, University of California Davis, Sacramento, CA
| | - Maura Sullivan
- Department of Surgery, University of Southern California, Los Angeles, CA
| | - Georges Bordage
- Department of Medical Education, University of Illinois at Chicago, Chicago, IL
| |
Collapse
|
20
|
Lang F, Willuth E, Haney CM, Felinska EA, Wennberg E, Kowalewski KF, Schmidt MW, Wagner M, Müller-Stich BP, Nickel F. Serious gaming and virtual reality in the multimodal training of laparoscopic inguinal hernia repair: a randomized crossover study. Surg Endosc 2023; 37:2050-2061. [PMID: 36289083 PMCID: PMC10017619 DOI: 10.1007/s00464-022-09733-6] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Accepted: 10/11/2022] [Indexed: 11/26/2022]
Abstract
BACKGROUND The aim of this study was to assess the transferability of surgical skills for the laparoscopic hernia module between the serious game Touch Surgery™ (TS) and the virtual reality (VR) trainer Lap Mentor™. Furthermore, this study aimed to collect validity evidence and to discuss "sources of validity evidence" for the findings using the laparoscopic inguinal hernia module on TS. METHODS In a randomized crossover study, medical students (n = 40) in their clinical years performed laparoscopic inguinal hernia modules on TS and the VR trainer. TS group started with "Laparoscopic Inguinal Hernia Module" on TS (phase 1: Preparation, phase 2: Port Placement and Hernia Repair), performed the module first in training, then in test mode until proficiency was reached. VR group started with "Inguinal Hernia Module" on the VR trainer (task 1: Anatomy Identification, task 2: Incision and Dissection) and also performed the module until proficiency. Once proficiency reached in the first modality, the groups performed the other training modality until reaching proficiency. Primary endpoint was the number of attempts needed to achieve proficiency for each group for each task/phase. RESULTS Students starting with TS needed significantly less attempts to reach proficiency for task 1 on the VR trainer than students who started with the VR trainer (TS = 2.7 ± 0.6 vs. VR = 3.2 ± 0.7; p = 0.028). No significant differences for task 2 were observed between groups (TS = 2.3 ± 1.1 vs. VR = 2.1 ± 0.8; p = 0.524). For both phases on TS, no significant skill transfer from the VR trainer to TS was observed. Aspects of validity evidence for the module on TS were collected. CONCLUSION The results show that TS brought additional benefit to improve performances on the VR trainer for task 1 but not for task 2. Skill transfer from the VR trainer to TS could not be shown. VR and TS should thus be used in combination with TS first in multimodal training to ensure optimal training conditions.
Collapse
Affiliation(s)
- Franziska Lang
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - E Willuth
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - C M Haney
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - E A Felinska
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - E Wennberg
- Temerty Faculty of Medicine, University of Toronto, Toronto, Canada
| | - K F Kowalewski
- Department of Urology and Urological Surgery, University Medical Center Mannheim, Heidelberg University, Heidelberg, Germany
| | - M W Schmidt
- Department of Gynecology and Obstetrics, University Medical Center of the Johannes Gutenberg University, Mainz, Germany
| | - M Wagner
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - B P Müller-Stich
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany
| | - F Nickel
- Department of General, Visceral and Transplantation Surgery, Heidelberg University Hospital, Heidelberg, Germany.
| |
Collapse
|
21
|
Haug TR, Miskovic D, Ørntoft MBW, Iversen LH, Johnsen SP, Valentin JB, Gomez Ruiz M, Benz S, Storli KE, Stearns AT, Brigic A, Madsen AH. Development of a procedure-specific tool for skill assessment in left- and right-sided laparoscopic complete mesocolic excision. Colorectal Dis 2023; 25:31-43. [PMID: 36031925 PMCID: PMC10087795 DOI: 10.1111/codi.16317] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/13/2022] [Revised: 08/11/2022] [Accepted: 08/14/2022] [Indexed: 02/02/2023]
Abstract
AIM To (1) develop an assessment tool for laparoscopic complete mesocolic excision (LCME) and (2) report evidence of its content validity. METHOD Assessment statements were revealed through (1) semi-structured expert interviews and (2) consensus by the Delphi method, both involving an expert panel of five LCME surgeons. All experts were interviewed and then asked to rate LCME describing statements from 1 (strongly disagree) to 5 (strongly agree). Responses were returned anonymously to the panel until consensus was reached. Statements were directly included as content in the assessment tool if ≥60% of the experts responded "agree" or "strongly agree" (ratings 4 and 5), with the remaining responses being "neither agree nor disagree" (rating 3). Interclass correlation coefficient (ICC) was calculated for expert agreement evaluation. All included statements were subsequently reformulated as tool items and approved by the experts. RESULTS Four Delphi rounds were performed to reach consensus. Disagreement was reported for statements describing instrument handling around pancreas; visualisation of landmarks before inferior mesenteric artery ligation; lymphadenectomy around the inferior mesenteric artery, and division of the terminal ileum and transverse colon. ICC in the last Delphi-round was 0.84. The final tool content included 73 statements, converted to 48 right- and 40 left-sided items for LCME assessment. CONCLUSION A procedure-specific, video-based tool, named complete mesocolic excision competency assessment tool (CMECAT), has been developed for LCME skill assessment. In the future, we hope it can facilitate assessment of LCME surgeons, resulting in improved patient outcome after colon cancer surgery.
Collapse
Affiliation(s)
- Tora Rydtun Haug
- Department of Surgery, Gødstrup Hospital, Herning, Denmark.,Department of Surgery, Aarhus University, Aarhus, Denmark
| | | | | | | | - Søren Paaske Johnsen
- Department of Clinical Medicine, Danish Center for Clinical Health Services Research, Aalborg University, Aalborg, Denmark
| | - Jan Brink Valentin
- Department of Clinical Medicine, Danish Center for Clinical Health Services Research, Aalborg University, Aalborg, Denmark
| | - Marcos Gomez Ruiz
- Colorectal Surgery Unit, Hospital Universitario Marques de Valdecilla and Valdecilla Biomedical Research Institute, IDIVAL, Santander, Spain
| | - Stefan Benz
- Klinik fur Allgemein-und Viszeralchirurgie, Kliniken Boblingen, Germany
| | | | | | - Adela Brigic
- Queen's Medical Centre, Nottingham University Hospitals, Nottingham, UK
| | | |
Collapse
|
22
|
Kurashima Y, Kitagami H, Teramura K, Poudel S, Ebihara Y, Inaki N, Nakamura F, Misawa K, Shibao K, Nagai E, Akashi Y, Takiguchi S, Kobayashi S, Hirano S. Validation study of a skill assessment tool for education and outcome prediction of laparoscopic distal gastrectomy. Surg Endosc 2022; 36:8807-8816. [PMID: 35578050 DOI: 10.1007/s00464-022-09305-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/10/2022] [Accepted: 04/23/2022] [Indexed: 01/06/2023]
Abstract
BACKGROUND The Japanese operative-rating scale for laparoscopic distal gastrectomy (JORS-LDG) was developed through cognitive task analysis together with the Delphi method to measure intraoperative performance during laparoscopic distal gastrectomy. This study aimed to investigate the value of this rating scale as an educational tool and a surgical outcome predictor in laparoscopic distal gastrectomy. METHODS The surgical performance of laparoscopic distal gastrectomy was assessed by the first assistant, through self-evaluation in the operating room and by video raters blind to the case. We evaluated inter-rater reliability, internal consistency, and correlations between the JORS-LDG scores and the evaluation methods, patient characteristics, and surgical outcomes. RESULTS Fifty-four laparoscopic distal gastrectomy procedures performed by 40 surgeons at 16 institutions were evaluated in the operating room and with video recordings using the proposed rating scale. The video inter-rater reliability was > 0.8. Participating surgeons were divided into the low, intermediate, and high groups based on their total scores. The number of laparoscopic surgeries and laparoscopic gastrectomy procedures performed differed significantly among the groups according to laparoscopic distal gastrectomy skill levels. The low, intermediate, and high groups also differed in terms of median operating times (311, 266, and 229 min, respectively, P < 0.001), intraoperative complication rates (27.8, 11.8, and 0%, respectively, P = 0.01), and postoperative complication rates (22.2, 0, and 0%, respectively, P = 0.002). CONCLUSIONS The JORS-LDG is a reliable and valid measure for laparoscopic distal gastrectomy training and could be useful in predicting surgical outcomes.
Collapse
Affiliation(s)
- Yo Kurashima
- Department of Gastroenterological Surgery II, Hokkaido University School of Medicine, Kita-15, Nishi-7, Kita-ku, Sapporo, Hokkaido, 060-8638, Japan.
- Clinical Simulation Center, Hokkaido University School of Medicine, Sapporo, Japan.
| | - Hidehiko Kitagami
- Department of Gastroenterological Surgery, Keiyukai Sapporo Hospital, Sapporo, Japan
| | - Koichi Teramura
- Department of Colorectal Surgery, National Cancer Center Hospital East, Kashiwa, Japan
| | - Saseem Poudel
- Department of Gastroenterological Surgery II, Hokkaido University School of Medicine, Kita-15, Nishi-7, Kita-ku, Sapporo, Hokkaido, 060-8638, Japan
| | - Yuma Ebihara
- Department of Gastroenterological Surgery II, Hokkaido University School of Medicine, Kita-15, Nishi-7, Kita-ku, Sapporo, Hokkaido, 060-8638, Japan
| | - Noriyuki Inaki
- Department of Gastrointestinal Surgery, Kanazawa University Graduate School of Medical Sciences, Kanazawa, Japan
| | - Fumitaka Nakamura
- Department of Gastroenterological Surgery, Teine Keijinkai Hospital, Sapporo, Japan
| | - Kazunari Misawa
- Department of Gastroenterological Surgery, Aichi Cancer Center Hospital, Nagoya, Japan
| | - Kazunori Shibao
- Department of Surgey1, School of Medicine, University of Occupational and Environmental Health, Kitakyushu, Japan
| | - Eishi Nagai
- Department of Gastrointestinal Surgery, Japanese Red Cross Fukuoka Hospital, Fukuoka, Japan
| | - Yoshimasa Akashi
- Department of Gastrointestinal and Hepato-Billiary-Pancreatic Surgery, Faculty of Medicine, University of Tsukuba, Tsukuba, Japan
| | - Shuji Takiguchi
- Department of Gastroenterological Surgery, Nagoya City University Graduate School of Medical Sciences, Nagoya, Japan
| | - Shinichiro Kobayashi
- Department of Surgery, Nagasaki University Graduate School of Biomedical Sciences, Nagasaki, Japan
| | - Satoshi Hirano
- Department of Gastroenterological Surgery II, Hokkaido University School of Medicine, Kita-15, Nishi-7, Kita-ku, Sapporo, Hokkaido, 060-8638, Japan
| |
Collapse
|
23
|
Jogerst KM, Park YS, Anteby R, Sinyard R, Coe TM, Cassidy D, McKinley SK, Petrusa E, Phitayakorn R, Mohapatra A, Gee DW. Impact of Rater Training on Residents Technical Skill Assessments: A Randomized Trial. JOURNAL OF SURGICAL EDUCATION 2022; 79:e225-e234. [PMID: 36333174 DOI: 10.1016/j.jsurg.2022.09.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 08/28/2022] [Accepted: 09/23/2022] [Indexed: 06/16/2023]
Abstract
OBJECTIVE The ACS/APDS Resident Skills Curriculum's Objective Structured Assessment of Technical Skills (OSATS) consists of task-specific checklists and a global rating scale (GRS) completed by raters. Prior work demonstrated a need for rater training. This study evaluates the impact of a rater-training curriculum on scoring discrimination, consistency, and validity for handsewn bowel anastomosis (HBA) and vascular anastomosis (VA). DESIGN/ METHODS A rater training video model was developed, which included a GRS orientation and anchoring performances representing the range of potential scores. Faculty raters were randomized to rater training or no rater training and were asked to score videos of resident HBA/VA. Consensus scores were assigned to each video using a modified Delphi process (Gold Score). Trained and untrained scores were analyzed for discrimination and score spread and compared to the Gold Score for relative agreement. RESULTS Eight general and eight vascular surgery faculty were randomized to score 24 HBA/VA videos. Rater training increased rater discrimination and decreased rating scale shrinkage for both VA (mean trained score: 2.83, variance 1.88; mean untrained score: 3.1, variance 1.14, p = 0.007) and HBA (mean trained score: 3.52, variance 1.44; mean untrained score: 3.42, variance 0.96, p = 0.033). On validity analyses, a comparison between each rater group vs Gold Score revealed a moderate training impact for VA, trained κ=0.65 vs untrained κ=0.57 and no impact for HBA, R1 κ = 0.71 vs R2 κ = 0.73. CONCLUSION A rater-training curriculum improved raters' ability to differentiate performance levels and use a wider range of the scoring scale. However, despite rater training, there was persistent disagreement between faculty GRS scores with no groups reaching the agreement threshold for formative assessment. If technical skill exams are incorporated into high stakes assessments, consensus ratings via a standard setting process are likely a more valid option than individual faculty ratings.
Collapse
Affiliation(s)
- Kristen M Jogerst
- Department of General Surgery, Mayo Clinic Arizona, Phoenix, Arizona; Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Yoon Soo Park
- Department of Emergency Medicine, Massachusetts General Hospital, Boston, Massachusetts
| | - Roi Anteby
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Robert Sinyard
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Taylor M Coe
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Douglas Cassidy
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Sophia K McKinley
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Emil Petrusa
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Roy Phitayakorn
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Abhisekh Mohapatra
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts
| | - Denise W Gee
- Department of Surgery, Massachusetts General Hospital, Boston, Massachusetts.
| |
Collapse
|
24
|
Muacevic A, Adler JR, Patel R, Baxter J. The Validation of Surgical Simulators: A Technical Report on Current Validation Terminology as a Reference for Future Research. Cureus 2022; 14:e31881. [PMID: 36579220 PMCID: PMC9790137 DOI: 10.7759/cureus.31881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/24/2022] [Indexed: 11/27/2022] Open
Abstract
In recent years, surgical trainees have been exposed to a lower volume of operative procedures. In part, this is due to the reduction in working hours and further disruption by the coronavirus disease 2019 pandemic. Much has been done to develop the techniques of surgical skill training outside of the operating theatre. Simulation-based interventions must undergo a process of validation to assess their appropriateness and effectiveness for use in training. The terminology of validation within current literature, however, has not evolved in line with the education community, resulting in varying definitions for the same phrase across domains. This can result in confusion and misinterpretation among researchers and surgeons working within this domain. This technical report describes the "types of validity" definitions used in the traditional framework of surgical simulation literature and the contemporary, unitary framework of validity adopted by educationalist theorists. There is a clear overlap between the traditional "types of validity" and the contemporary, unitary framework. The divergence in the use of those definitions seems, at least partly, influenced by the context of the investigations being conducted. By utilising the contemporary definitions, authors may have struggled to provide the evidence required to justify the use of the multitude of surgical skill simulators developed in the recent past. This report has provided an overview of the current terminology within the validation frameworks and can be used as a reference for future surgical simulation research.
Collapse
|
25
|
Hong M, Rozenblit JW. An Adaptive Force Guidance System for Computer-Guided Laparoscopy Training. IEEE TRANSACTIONS ON CYBERNETICS 2022; 52:8019-8031. [PMID: 33600333 DOI: 10.1109/tcyb.2021.3051837] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/12/2023]
Abstract
We present an adaptive force guidance system for laparoscopic surgery skills training. This system consists of self-adjusting fuzzy sliding-mode controllers and switching mode controllers to provide proper force feedback. Using virtual fixtures, the proposed system restricts motions or guides a trainee to navigate a surgical instrument in a 3-D space in a manner that mimics a human instructor who would teach the trainees by holding their hands. The self-adjusting controllers incorporate human factors, such as different force sensitivity and proficiency levels. The proposed system was implemented and evaluated using the computer-assisted surgical trainer (CAST). The effects of the force guidance system are presented based on the experimental test results.
Collapse
|
26
|
White A, Muller Moran HR, Ryan J, Mador B, Campbell S, Turner SR. Validity Evidence for Procedure-Specific Competency Assessment Tools in Cardiovascular and Thoracic Surgery: A Scoping Review. JOURNAL OF SURGICAL EDUCATION 2022; 79:1016-1023. [PMID: 35491353 DOI: 10.1016/j.jsurg.2022.02.014] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/01/2021] [Revised: 01/19/2022] [Accepted: 02/28/2022] [Indexed: 06/14/2023]
Abstract
OBJECTIVE Surgical education has shifted from a time-based approach to the achievement and demonstration of procedural competency. High quality, objective assessment instruments are required to support this new approach. This study comprehensively reviewed the literature to identify and evaluate available procedure-specific assessment instruments in cardiothoracic and vascular surgery. DESIGN A systematic search of 8 databases identified studies containing procedure-specific operative assessment instruments in cardiothoracic and vascular surgery. Generic global rating scales were excluded, unless modified to be procedure-specific. Two reviewers independently evaluated the validity evidence, methodological rigour and educational utility of each instrument using objective scoring criteria. Validity evidence was evaluated with a scoring tool aligned with the contemporary framework of validity. Methodological rigour was evaluated using the Medical Education Research Study Quality Instrument. Educational utility was evaluated according to the Accreditation Council for Graduate Medical Education (ACGME) framework. RESULTS There were 2130 unique studies describing procedure-specific assessment in surgery. Of these, 9 studies evaluating 8 procedure-specific assessment instruments met inclusion criteria for cardiothoracic and vascular surgery. Four instruments were identified in thoracic surgery, 2 in cardiac surgery, and 2 in vascular surgery. Only 1 instrument was designed to evaluate surgeon performance, with the remainder designed to evaluate residents. No single instrument scored the maximum score of 15 for validity evidence. The highest score was 11, with 62.5% (n = 5) of instruments scoring greater than 10. All tools attained high scores in content validity, with minimal evidence generally presented regarding the consequences of assessment using a particular instrument. All but 1 instrument scored greater than 11 out of a maximum 16.5 points for methodological rigour. Very few studies reported on the ACGME domains of educational utility. CONCLUSIONS In an era where surgical education is shifting towards the demonstration of procedural competency, objective procedure-specific assessment is critical. This review identified that few procedure-specific assessment instruments in cardiothoracic and vascular surgery exist, emphasizing the need for such instruments to ensure the success of competency-based education models.
Collapse
Affiliation(s)
- Abigail White
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | | | - Joanna Ryan
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Brett Mador
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | | | - Simon R Turner
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada.
| |
Collapse
|
27
|
Technical skills in the operating room: Implications for perioperative leadership and patient outcomes. Best Pract Res Clin Anaesthesiol 2022; 36:237-245. [DOI: 10.1016/j.bpa.2022.05.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Accepted: 05/13/2022] [Indexed: 01/02/2023]
|
28
|
Ben Charif A, Zomahoun HTV, Gogovor A, Abdoulaye Samri M, Massougbodji J, Wolfenden L, Ploeg J, Zwarenstein M, Milat AJ, Rheault N, Ousseine YM, Salerno J, Markle-Reid M, Légaré F. Tools for assessing the scalability of innovations in health: a systematic review. Health Res Policy Syst 2022; 20:34. [PMID: 35331260 PMCID: PMC8943495 DOI: 10.1186/s12961-022-00830-5] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2021] [Accepted: 02/16/2022] [Indexed: 01/20/2023] Open
Abstract
BACKGROUND The last decade has seen growing interest in scaling up of innovations to strengthen healthcare systems. However, the lack of appropriate methods for determining their potential for scale-up is an unfortunate global handicap. Thus, we aimed to review tools proposed for assessing the scalability of innovations in health. METHODS We conducted a systematic review following the COSMIN methodology. We included any empirical research which aimed to investigate the creation, validation or interpretability of a scalability assessment tool in health. We searched Embase, MEDLINE, CINAHL, Web of Science, PsycINFO, Cochrane Library and ERIC from their inception to 20 March 2019. We also searched relevant websites, screened the reference lists of relevant reports and consulted experts in the field. Two reviewers independently selected and extracted eligible reports and assessed the methodological quality of tools. We summarized data using a narrative approach involving thematic syntheses and descriptive statistics. RESULTS We identified 31 reports describing 21 tools. Types of tools included criteria (47.6%), scales (33.3%) and checklists (19.0%). Most tools were published from 2010 onwards (90.5%), in open-access sources (85.7%) and funded by governmental or nongovernmental organizations (76.2%). All tools were in English; four were translated into French or Spanish (19.0%). Tool creation involved single (23.8%) or multiple (19.0%) types of stakeholders, or stakeholder involvement was not reported (57.1%). No studies reported involving patients or the public, or reported the sex of tool creators. Tools were created for use in high-income countries (28.6%), low- or middle-income countries (19.0%), or both (9.5%), or for transferring innovations from low- or middle-income countries to high-income countries (4.8%). Healthcare levels included public or population health (47.6%), primary healthcare (33.3%) and home care (4.8%). Most tools provided limited information on content validity (85.7%), and none reported on other measurement properties. The methodological quality of tools was deemed inadequate (61.9%) or doubtful (38.1%). CONCLUSIONS We inventoried tools for assessing the scalability of innovations in health. Existing tools are as yet of limited utility for assessing scalability in health. More work needs to be done to establish key psychometric properties of these tools. Trial registration We registered this review with PROSPERO (identifier: CRD42019107095).
Collapse
Affiliation(s)
| | - Hervé Tchala Vignon Zomahoun
- Department of Social and Preventive Medicine, Université Laval, Quebec City, QC, Canada.,Faculty of Medicine and Health Science, School of Physical and Occupational Therapy, McGill University, Montreal, QC, Canada.,Institut national d'excellence en santé et en services sociaux (INESSS), Quebec City, QC, Canada
| | - Amédé Gogovor
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada.,Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Translation, Université Laval, Quebec City, QC, Canada.,Department of Family Medicine and Emergency Medicine, Université Laval, Quebec City, QC, Canada.,Unité de soutien SSA Québec, Université Laval, Quebec City, QC, Canada
| | - Mamane Abdoulaye Samri
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada.,Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Translation, Université Laval, Quebec City, QC, Canada.,Department of Family Medicine and Emergency Medicine, Université Laval, Quebec City, QC, Canada
| | - José Massougbodji
- Institut national de santé publique du Québec (INSPQ), Quebec City, QC, Canada
| | - Luke Wolfenden
- School of Medicine and Public Health, University of Newcastle, Callaghan, NSW, Australia.,Hunter Medical Research Institute, New Lambton Heights, NSW, Australia.,Hunter New England Population Health, Wallsend, NSW, Australia
| | - Jenny Ploeg
- Aging, Community and Health Research Unit, School of Nursing, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada
| | - Merrick Zwarenstein
- Department of Family Medicine, Centre for Studies in Family Medicine, Schulich School of Medicine and Dentistry, Western University, London, ON, Canada
| | - Andrew J Milat
- School of Public Health, University of Sydney, Sydney, NSW, Australia.,Centre for Epidemiology and Evidence, NSW Ministry of Health, Sydney, Australia
| | - Nathalie Rheault
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada.,Unité de soutien SSA Québec, Université Laval, Quebec City, QC, Canada
| | | | - Jennifer Salerno
- Aging, Community and Health Research Unit, School of Nursing, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada
| | - Maureen Markle-Reid
- Aging, Community and Health Research Unit, School of Nursing, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada.,Canada Research Chair in Person Centred Interventions for Older Adults with Multimorbidity and their Caregivers, McMaster University, Hamilton, ON, Canada
| | - France Légaré
- VITAM - Centre de recherche en santé durable, Université Laval, Quebec City, QC, Canada. .,Tier 1 Canada Research Chair in Shared Decision Making and Knowledge Translation, Université Laval, Quebec City, QC, Canada. .,Department of Family Medicine and Emergency Medicine, Université Laval, Quebec City, QC, Canada. .,Unité de soutien SSA Québec, Université Laval, Quebec City, QC, Canada. .,Population Health and Practice-Changing Research Group, CHU de Québec Research Centre, Quebec City, QC, Canada.
| |
Collapse
|
29
|
Ryan JF, Mador B, Lai K, Campbell S, Hyakutake M, Turner SR. Validity Evidence for Procedure-specific Competence Assessment Tools in General Surgery: A Scoping Review. Ann Surg 2022; 275:482-487. [PMID: 34520421 DOI: 10.1097/sla.0000000000005207] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
Abstract
OBJECTIVE This study aims to provide a focused and detailed assessment of the validity evidence supporting procedure-specific operative assessment tools in general surgery. SUMMARY OF BACKGROUND DATA Competency-based assessment tools should be supported by robust validity evidence to be used reliably for evaluation of operative skills. The contemporary framework of validity relies on five sources of evidence: content, response process, internal structure, relation to other variables, and consequences. METHODS A systematic search of 8 databases was conducted for studies containing procedure-specific operative assessment tools in general surgery. The validity evidence supporting each tool was assessed and scored in alignment with the contemporary framework of validity. Methodological rigour of studies was assessed with the Medical Education Research Study Quality Instrument. The educational utility of each tool was assessed with the Accreditation Council for Graduate Medical Education framework. RESULTS There were 28 studies meeting inclusion criteria and 23 unique tools were assessed. Scores for validity evidence varied widely between tools, ranging from 3 - 14 (maximum 15). Medical Education Research Study Quality Instrument scores assessing the quality of study methodology were also variable (8.5-15.5, maximum 16.5). Direct reporting of educational utility criteria was limited. CONCLUSIONS This study has identified a small group of procedure-specific operative assessment tools in general surgery. Many of these tools have limited validity evidence and have not been studied sufficiently to be used reliably in high-stakes summative assessments. As general surgery transitions to competency-based training, a more robust library of operative assessment tools will be required to support resident education and evaluation.
Collapse
Affiliation(s)
- Joanna F Ryan
- Department of Surgery, University of Alberta, Edmonton, Canada
| | - Brett Mador
- Department of Surgery, University of Alberta, Edmonton, Canada
| | - Krista Lai
- Department of Surgery, University of Alberta, Edmonton, Canada
| | - Sandra Campbell
- John W. Scott Health Sciences Library, University of Alberta, Edmonton, Canada
| | - Momoe Hyakutake
- Department of Obstetrics and Gynecology, University of Alberta, Edmonton, Canada
| | - Simon R Turner
- Department of Surgery, University of Alberta, Edmonton, Canada
| |
Collapse
|
30
|
Carstensen SMD, Velander MJ, Konge L, Østergaard M, Pfeiffer Jensen M, Just SA, Terslev L. Training and assessment of musculoskeletal ultrasound and injection skills-a systematic review. Rheumatology (Oxford) 2022; 61:3889-3901. [PMID: 35218339 DOI: 10.1093/rheumatology/keac119] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/17/2021] [Revised: 02/17/2022] [Indexed: 11/12/2022] Open
Abstract
OBJECTIVES To examine how residents are trained and assessed in musculoskeletal ultrasound (MSUS), MSUS-guided and landmark-guided joint aspiration and injection. Additionally, to present the available assessment tools and examine their supporting validity evidence. METHODS A systematic search of PubMed, Cochrane Library, and Embase was conducted in accordance with the PRISMA guidelines and studies published from January 1, 2000 to May 31, 2021 were included. Two independent reviewers performed the search and data extraction. The studies were evaluated using the Medical Education Research Quality Instrument (MERSQI). RESULTS 9,884 articles were screened, and 43 were included; 3 were randomized studies, 21 pre- and post-test studies, 16 descriptive studies, and 3 studies developing assessment tools. The studies used various theoretical training modalities e.g. lectures, anatomical quizzes and e-learning. The practical training models varied from mannequins and cadavers to healthy volunteers and patients. The majority of studies used subjective "comfort level" as assessment, others used practical examination and/or theoretical examination. All training programs increased trainees' self-confidence, theoretical knowledge, and/or practical performance, however few used validated assessment tools to measure the effect. Only one study met the MERSQI high methodical quality cut-off score of 14. CONCLUSION The included studies were heterogeneous, and most were of poor methodological quality and not based on contemporary educational theories. This review highlights the need for educational studies using validated theoretical and practical assessment tools to ensure optimal MSUS training and assessment in rheumatology.
Collapse
Affiliation(s)
- Stine M D Carstensen
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.,Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Copenhagen, Denmark
| | - Marie Juul Velander
- Section of Rheumatology, Department of Medicine, Svendborg Hospital-OUH Baagøes Allé 15, Svendborg, Denmark
| | - Lars Konge
- Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.,Copenhagen Academy for Medical Education and Simulation, The Capital Region of Denmark, Copenhagen, Denmark
| | - Mikkel Østergaard
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Mogens Pfeiffer Jensen
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Søren Andreas Just
- Section of Rheumatology, Department of Medicine, Svendborg Hospital-OUH Baagøes Allé 15, Svendborg, Denmark
| | - Lene Terslev
- Copenhagen Center for Arthritis Research, Center for Rheumatology and Spine Diseases, Centre for Head and Orthopaedics, Rigshospitalet Glostrup Valdemar Hansens Vej 17, Glostrup, Denmark.,Department of Clinical Medicine, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| |
Collapse
|
31
|
How can surgical skills in laparoscopic colon surgery be objectively assessed?-a scoping review. Surg Endosc 2021; 36:1761-1774. [PMID: 34873653 PMCID: PMC8847271 DOI: 10.1007/s00464-021-08914-z] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2021] [Accepted: 11/21/2021] [Indexed: 11/08/2022]
Abstract
Background In laparoscopic colorectal surgery, higher technical skills have been associated with improved patient outcome. With the growing interest in laparoscopic techniques, pressure on surgeons and certifying bodies is mounting to ensure that operative procedures are performed safely and efficiently. The aim of the present review was to comprehensively identify tools for skill assessment in laparoscopic colon surgery and to assess their validity as reported in the literature. Methods A systematic search was conducted in EMBASE and PubMed/MEDLINE in May 2021 to identify studies examining technical skills assessment tools in laparoscopic colon surgery. Available information on validity evidence (content, response process, internal structure, relation to other variables, and consequences) was evaluated for all included tools. Results Fourteen assessment tools were identified, of which most were procedure-specific and video-based. Most tools reported moderate validity evidence. Commonly not reported were rater training, assessment correlation with variables other than training level, and validity reproducibility and reliability in external educational settings. Conclusion The results of this review show that several tools are available for evaluation of laparoscopic colon cancer surgery, but few authors present substantial validity for tool development and use. As we move towards the implementation of new techniques in laparoscopic colon surgery, it is imperative to establish validity before surgical skill assessment tools can be applied to new procedures and settings. Therefore, future studies ought to examine different aspects of tool validity, especially correlation with other variables, such as patient morbidity and pathological reports, which impact patient survival. Supplementary Information The online version contains supplementary material available at 10.1007/s00464-021-08914-z.
Collapse
|
32
|
Khan R, Zheng E, Wani SB, Scaffidi MA, Jeyalingam T, Gimpaya N, Anderson JT, Grover SC, McCreath G, Walsh CM. Colonoscopy competence assessment tools: a systematic review of validity evidence. Endoscopy 2021; 53:1235-1245. [PMID: 33440438 DOI: 10.1055/a-1352-7293] [Citation(s) in RCA: 16] [Impact Index Per Article: 5.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
BACKGROUND Assessment tools are essential for endoscopy training, being required to support feedback provision, optimize learner capabilities, and document competence. We aimed to evaluate the strength of validity evidence that supports the available colonoscopy direct observation assessment tools using the unified framework of validity. METHODS We systematically searched five databases for studies investigating colonoscopy direct observation assessment tools from inception until 8 April 2020. We extracted data outlining validity evidence (content, response process, internal structure, relations to other variables, and consequences) from the five sources and graded the degree of evidence, with a maximum score of 15. We assessed educational utility using an Accreditation Council for Graduate Medical Education framework and methodological quality using the Medical Education Research Quality Instrument (MERSQI). RESULTS From 10 841 records, we identified 27 studies representing 13 assessment tools (10 adult, 2 pediatric, 1 both). All tools assessed technical skills, while 10 each assessed cognitive and integrative skills. Validity evidence scores ranged from 1-15. The Assessment of Competency in Endoscopy (ACE) tool, the Direct Observation of Procedural Skills (DOPS) tool, and the Gastrointestinal Endoscopy Competency Assessment Tool (GiECAT) had the strongest validity evidence, with scores of 13, 15, and 14, respectively. Most tools were easy to use and interpret, and required minimal resources. MERSQI scores ranged from 9.5-11.5 (maximum score 14.5). CONCLUSIONS The ACE, DOPS, and GiECAT have strong validity evidence compared with other assessments. Future studies should identify barriers to widespread implementation and report on the use of these tools in credentialing examinations.
Collapse
Affiliation(s)
- Rishad Khan
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Eric Zheng
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Canada
| | - Sachin B Wani
- Division of Gastroenterology and Hepatology, University of Colorado Anschutz Medical Campus, Aurora, Colorado, USA
| | - Michael A Scaffidi
- Faculty of Medicine, Queen's University, Kingston, Canada.,Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada
| | - Thurarshen Jeyalingam
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Canada.,The Wilson Centre, University of Toronto, Toronto, Canada
| | - Nikko Gimpaya
- Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada
| | - John T Anderson
- Joint Advisory Group on Gastrointestinal Endoscopy, Royal College of Physicians, London, UK.,Department of Gastroenterology, Gloucestershire Hospitals NHS Foundation Trust, Gloucestershire, UK
| | - Samir C Grover
- Department of Medicine, Faculty of Medicine, University of Toronto, Toronto, Canada.,Division of Gastroenterology, St. Michael's Hospital, Toronto, Canada.,Li Ka Shing Knowledge Institute, St. Michael's Hospital, Toronto, Canada
| | - Graham McCreath
- SickKids Research and Lerning Institutes, The Hospital for Sick Children, Toronto, Canada
| | - Catharine M Walsh
- The Wilson Centre, University of Toronto, Toronto, Canada.,SickKids Research and Lerning Institutes, The Hospital for Sick Children, Toronto, Canada.,Division of Gastroenterology, Hepatology, and Nutrition, The Hospital for Sick Children, Toronto, Canada.,Department of Paediatrics, Faculty of Medicine, University of Toronto, Toronto, Canada
| |
Collapse
|
33
|
Hovgaard LH, Al-Shahrestani F, Andersen SAW. Current Evidence for Simulation-Based Training and Assessment of Myringotomy and Ventilation Tube Insertion: A Systematic Review. Otol Neurotol 2021; 42:e1188-e1196. [PMID: 34267097 DOI: 10.1097/mao.0000000000003268] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVE Myringotomy and ventilation tube insertion (MT) is a key procedure in otorhinolaryngology and can be trained using simulation models. We aimed to systematically review the literature on models for simulation-based training and assessment of MT and supporting educational evidence. DATABASES REVIEWED PubMed, Embase, Cochrane Library, Web of Science, Directory of Open Access Journals. METHODS Inclusion criteria were MT training and/or skills assessment using all types of training modalities and learners. Studies were divided into 1) descriptive and 2) educational interventional/observational in the analysis. For descriptive studies, we provide an overview of available models including materials and cost. Educational studies were appraised using Kirkpatrick's level of educational outcomes, Messick's framework of validity, and a structured quality assessment tool. RESULTS Forty-six studies were included consisting of 21 descriptive studies and 25 educational studies. Thirty-one unique physical and three virtual reality simulation models were identified. The studies report moderate to high realism of the different simulators and trainees and educators perceive them beneficial in training MT skills. Overall, simulation-based training is found to reduce procedure time and errors, and increase performance as measured using different assessment tools. None of the studies used a contemporary validity framework and the current educational evidence is limited. CONCLUSION Numerous simulation models and assessment tools have been described in the literature but educational evidence and systematic implementation into training curricula is scarce. There is especially a need to establish the effect of simulation-based training of MT in transfer to the operating room and on patient outcomes.
Collapse
Affiliation(s)
- Lisette Hvid Hovgaard
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, RegionH
| | - Fahd Al-Shahrestani
- Department of Otorhinolaryngology and Maxillofacial Surgery, Zealand University Hospital, Køge
| | - Steven Arild Wuyts Andersen
- Copenhagen Academy for Medical Education and Simulation (CAMES), Center for HR & Education, RegionH
- Department of Otorhinolaryngology-Head & Neck Surgery, Rigshospitalet, Copenhagen, Denmark
- Department of Otolaryngology, Nationwide Children's Hospital, and the Ohio State University, Columbus, Ohio
| |
Collapse
|
34
|
Ong CW, Tan MCJ, Lam M, Koh VTC. Applications of Extended Reality in Ophthalmology: Systematic Review. J Med Internet Res 2021; 23:e24152. [PMID: 34420929 PMCID: PMC8414293 DOI: 10.2196/24152] [Citation(s) in RCA: 17] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Revised: 12/19/2020] [Accepted: 04/06/2021] [Indexed: 12/27/2022] Open
Abstract
BACKGROUND Virtual reality, augmented reality, and mixed reality make use of a variety of different software and hardware, but they share three main characteristics: immersion, presence, and interaction. The umbrella term for technologies with these characteristics is extended reality. The ability of extended reality to create environments that are otherwise impossible in the real world has practical implications in the medical discipline. In ophthalmology, virtual reality simulators have become increasingly popular as tools for surgical education. Recent developments have also explored diagnostic and therapeutic uses in ophthalmology. OBJECTIVE This systematic review aims to identify and investigate the utility of extended reality in ophthalmic education, diagnostics, and therapeutics. METHODS A literature search was conducted using PubMed, Embase, and Cochrane Register of Controlled Trials. Publications from January 1, 1956 to April 15, 2020 were included. Inclusion criteria were studies evaluating the use of extended reality in ophthalmic education, diagnostics, and therapeutics. Eligible studies were evaluated using the Oxford Centre for Evidence-Based Medicine levels of evidence. Relevant studies were also evaluated using a validity framework. Findings and relevant data from the studies were extracted, evaluated, and compared to determine the utility of extended reality in ophthalmology. RESULTS We identified 12,490 unique records in our literature search; 87 met final eligibility criteria, comprising studies that evaluated the use of extended reality in education (n=54), diagnostics (n=5), and therapeutics (n=28). Of these, 79 studies (91%) achieved evidence levels in the range 2b to 4, indicating poor quality. Only 2 (9%) out of 22 relevant studies addressed all 5 sources of validity evidence. In education, we found that ophthalmic surgical simulators demonstrated efficacy and validity in improving surgical performance and reducing complication rates. Ophthalmoscopy simulators demonstrated efficacy and validity evidence in improving ophthalmoscopy skills in the clinical setting. In diagnostics, studies demonstrated proof-of-concept in presenting ocular imaging data on extended reality platforms and validity in assessing the function of patients with ophthalmic diseases. In therapeutics, heads-up surgical systems had similar complication rates, procedural success rates, and outcomes in comparison with conventional ophthalmic surgery. CONCLUSIONS Extended reality has promising areas of application in ophthalmology, but additional high-quality comparative studies are needed to assess their roles among incumbent methods of ophthalmic education, diagnostics, and therapeutics.
Collapse
Affiliation(s)
- Chee Wui Ong
- Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| | - Marcus Chun Jin Tan
- Department of Ophthalmology, National University Hospital, Singapore, Singapore
| | - Michael Lam
- Department of Ophthalmology, Ng Teng Fong General Hospital, Singapore, Singapore
| | - Victor Teck Chang Koh
- Department of Ophthalmology, National University Hospital, Singapore, Singapore.,Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore
| |
Collapse
|
35
|
Hennings LI, Sørensen JL, Hybscmann J, Strandbygaard J. Tools for measuring technical skills during gynaecologic surgery: a scoping review. BMC MEDICAL EDUCATION 2021; 21:402. [PMID: 34311735 PMCID: PMC8314568 DOI: 10.1186/s12909-021-02790-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 10/01/2020] [Accepted: 06/09/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND Standardised assessment is key to structured surgical training. Currently, there is no consensus on which surgical assessment tool to use in live gynaecologic surgery. The purpose of this review is to identify assessment tools measuring technical skills in gynaecologic surgery and evaluate the measurement characteristics of each tool. METHOD We utilized the scoping review methodology and searched PubMed, Medline, Embase and Cochrane. Inclusion criteria were studies that analysed assessment tools in live gynaecologic surgery. Kane's validity argument was applied to evaluate the assessment tools in the included studies. RESULTS Eight studies out of the 544 identified fulfilled the inclusion criteria. The assessment tools were categorised as global rating scales, global and procedure rating scales combined, procedure-specific rating scales or as a non-procedure-specific error assessment tool. CONCLUSION This scoping review presents the current different tools for observational assessment of technical skills in intraoperative, gynaecologic surgery. This scoping review can serve as a guide for surgical educators who want to apply a scale or a specific tool in surgical assessment.
Collapse
Affiliation(s)
| | - Jette Led Sørensen
- Juliane Marie Centre for children, women and reproduction, Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | - Jane Hybscmann
- Juliane Marie Centre for children, women and reproduction, Rigshospitalet, Copenhagen, Denmark
- Department of Clinical Medicine, University of Copenhagen, Copenhagen, Denmark
| | | |
Collapse
|
36
|
Schmiederer IS, Kearse LE, Lin DT, Anderson TN, Lau JN, Korndorffer JR. Isolating steps instead of learners: Use of deliberate practice and validity evidence in coronavirus disease (COVID)-era procedural assessment. Surgery 2021; 170:1652-1658. [PMID: 34272045 PMCID: PMC8276111 DOI: 10.1016/j.surg.2021.06.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/18/2021] [Revised: 04/25/2021] [Accepted: 06/07/2021] [Indexed: 11/17/2022]
Abstract
Background In surgical training, assessment tools based on strong validity evidence allow for standardized evaluation despite changing external circumstances. At a large academic institution, surgical interns undergo a multimodal curriculum for central line placement that uses a 31-item binary assessment at the start of each academic year. This study evaluated this practice within increased in-person learning restrictions. We hypothesized that external constraints would not affect resident performance nor assessment due to a robust curriculum and assessment checklist. Methods From 2018 to 2020, 81 residents completed central line training and assessment. In 2020, this curriculum was modified to conform to in-person restrictions and social distancing guidelines. Resident score reports were analyzed using multivariate analyses to compare performance, objective scoring parameters, and subjective assessments among “precoronavirus disease” years (2018 and 2019) and 2020. Results There were no significant differences in average scores or objective pass rates over 3 years. Significant differences between 2020 and precoronavirus disease years occurred in subjective pass rates and in first-time success for 4 checklist items: patient positioning, draping, sterile ultrasound probe cover placement, and needle positioning before venipuncture. Conclusion Modifications to procedural training within current restrictions did not adversely affect residents’ overall performance. However, our data suggest that in 2020, expert trainers may not have ensured learner acquisition of automated procedural steps. Additionally, although 2020 raters could have been influenced by logistical barriers leading to more lenient grading, the assessment tool ensured training and assessment integrity.
Collapse
Affiliation(s)
| | | | - Dana T Lin
- Department of Surgery, Stanford University Medical Center, CA
| | | | - James N Lau
- Department of Surgery, Loyola University Medical Center, Maywood, IL
| | | |
Collapse
|
37
|
Martins Neto F, Moura Júnior LGD, Rocha HAL, Castro Neto JVD, Lima AMR, Siqueira RPD, Castillo DLC, Moura RLSD, Valente AS. Development and validation of a simulator for teaching minimally invasive thoracic surgery in Brazil. Acta Cir Bras 2021; 36:e360508. [PMID: 34161435 PMCID: PMC8221800 DOI: 10.1590/acb360508] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2021] [Accepted: 04/01/2021] [Indexed: 12/28/2022] Open
Abstract
Purpose To develop and validate a chest cavity simulator for teaching video-assited
thoracic surgery (VATS). Methods The first phase of the study consisted of developing a chest cavity
simulator. A quasi-experimental study was performed in the second phase, and
25 surgeons and residents participated in a three-stage pulmonary suture
experiment. The videos were recorded and timed. Generalized linear
regression models for repeated measures were used to analyze the outcome
change over time. Results The chest cavity simulator consists of a console simulating the left
hemithorax. Among the participants, 96% rated the design, visual aspect,
positioning ergonomics, and triangulation of the portals as very good or
excellent (face validity). There was a decrease in suturing time in step 1
from 435.7 ± 105 to 355.6 ± 76.8 seconds compared to step 3 (p = 0.001). The
evaluation of the simulation effectiveness and performance (content
validity) was rated as very good or excellent by 96% ofparticipants. The
most experienced surgeon showed significant reduction in procedure time (p =
0.021) (construct validity). Conclusions The thoracic cavity simulator is realistic, showing content and construct
validity, and can be used in VATS training. The simulation model allowed
skill gain in the endoscopic suture.
Collapse
|
38
|
Olsen RG, Bjerrum F, Konge L, Jepsen JV, Azawi NH, Bube SH. Validation of a Novel Simulation-Based Test in Robot-Assisted Radical Prostatectomy. J Endourol 2021; 35:1265-1272. [PMID: 33530867 DOI: 10.1089/end.2020.0986] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Abstract
Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP). Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Ureterovesical Anastomosis. Validity evidence was investigated by using Messick's framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach's alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups' method. Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach's alpha = 0.49, p < 0.001. Test-retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson's r = 0.74, p < 0.001. Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.
Collapse
Affiliation(s)
- Rikke Groth Olsen
- Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark
| | - Flemming Bjerrum
- Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.,Department of Surgery, Herlev/Gentofte Hospital, Herlev, Denmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.,Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark
| | - Jan Viberg Jepsen
- Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.,Department of Urology, Herlev/Gentofte Hospital, Herlev, Denmark
| | - Nessn H Azawi
- Department of Surgery, Herlev/Gentofte Hospital, Herlev, Denmark.,Department of Urology, Zealand University Hospital, Roskilde, Denmark
| | - Sarah Hjartbro Bube
- Copenhagen Academy for Medical Education and Simulation (CAMES), Copenhagen, Denmark.,Department of Urology, Zealand University Hospital, Roskilde, Denmark
| |
Collapse
|
39
|
Crochet P, Netter A, Schmitt A, Garofalo A, Loundou A, Knight S, Rabischong B, Agostini A. Performance Assessment for Total Laparoscopic Hysterectomy in the Operating Room: Validity Evidence of a Procedure-specific Rating Scale. J Minim Invasive Gynecol 2021; 28:1743-1750.e3. [PMID: 33621693 DOI: 10.1016/j.jmig.2021.02.013] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2020] [Revised: 02/09/2021] [Accepted: 02/16/2021] [Indexed: 10/22/2022]
Abstract
STUDY OBJECTIVE The technical conduct of total laparoscopic hysterectomy (LH) is critical to surgical outcomes. This study explored the validity evidence of an objective scale specific to the assessment of technical skills (H-OSATS) for 7 tasks of an LH with salpingo-oophorectomy procedure performed in the operating room. DESIGN Observational cohort study. SETTING Two academic hospitals in Marseille and Montpellier, France. PATIENTS Three groups of operators (novice, intermediate, and experienced surgeons) were video recorded during their live performances of LH on a simple case. For each group, a dozen unedited videos were obtained for the following tasks: division of the round ligament, division of the infundibulopelvic ligament, creation of the bladder flap, opening of the posterior peritoneum, division of the uterine vessels, colpotomy, and closure of the vault. INTERVENTIONS Two qualified raters blindly assessed each video using the H-OSATS rating scale. Inter-rater reliability and test-retest reliability were calculated as measures of internal structure. In a separate round of evaluations, the raters provided a global competent/noncompetent decision for each performance. As a measure of consequential validity, a pass/fail score was set for each task using the contrasting group method. MEASUREMENTS AND MAIN RESULTS Three tasks (creation of the bladder flap, colpotomy, and closure of the vault) displayed sound validity evidence: a meaningful total score difference among the 3 groups of experience as well as between the intermediate and experienced surgeons, reliability outcomes of >0.7, and a pass/fail score with a theoretical false-positive rate of <10%. CONCLUSION The validity evidence of the H-OSATS rating scale differed for separate evaluations of the 7 tasks. Three tasks (i.e., creation of the bladder flap, colpotomy, and closure of the vault) revealed sound validity evidence, including at the level of the attending surgeon, whereas other tasks were more consistent with low-stakes formative evaluation standards.
Collapse
Affiliation(s)
- Patrice Crochet
- Department of Obstetrics and Gynecology, La Conception Hospital, Aix Marseille University (Drs. Crochet, Netter, Schmitt, Garofalo, Knight, and Agostini); Department of Obstetrics and Gynecology, Arnaud de Villeneuve Hospital, University of Montpellier, Montpellier (Dr. Crochet).
| | - Antoine Netter
- Department of Obstetrics and Gynecology, La Conception Hospital, Aix Marseille University (Drs. Crochet, Netter, Schmitt, Garofalo, Knight, and Agostini); Institut Méditerranéen de Biodiversité et d'Écologie marine et continentale, Aix Marseille University, CNRS, IRD, Avignon University (Dr. Netter)
| | - Andy Schmitt
- Department of Obstetrics and Gynecology, La Conception Hospital, Aix Marseille University (Drs. Crochet, Netter, Schmitt, Garofalo, Knight, and Agostini)
| | - Anna Garofalo
- Department of Obstetrics and Gynecology, La Conception Hospital, Aix Marseille University (Drs. Crochet, Netter, Schmitt, Garofalo, Knight, and Agostini)
| | - Anderson Loundou
- Support Unit for Clinical Research and Economic Evaluation, Assistance Publique Hôpitaux de Marseille, Aix Marseille University (Dr. Loundou), Marseille
| | - Sophie Knight
- Department of Obstetrics and Gynecology, La Conception Hospital, Aix Marseille University (Drs. Crochet, Netter, Schmitt, Garofalo, Knight, and Agostini)
| | - Benoit Rabischong
- Department of Gynecological Surgery, Clermont-Ferrand University Hospital Estaing, Clermont-Ferrand (Dr. Rabischong), France
| | - Aubert Agostini
- Department of Obstetrics and Gynecology, La Conception Hospital, Aix Marseille University (Drs. Crochet, Netter, Schmitt, Garofalo, Knight, and Agostini)
| |
Collapse
|
40
|
Podolsky DJ, Fisher DM, Riff KWW, Zuker RM, Drake JM, Forrest CR. Assessing Performance in Simulated Cleft Palate Repair Using a Novel Video Recording Setup. Cleft Palate Craniofac J 2020; 57:687-693. [PMID: 32394745 DOI: 10.1177/1055665620913178] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/23/2022] Open
Abstract
OBJECTIVE To test the feasibility of implementing a high-fidelity cleft palate simulator during a workshop in Santiago, Chile, using a novel video endoscope to assess technical performance. DESIGN Sixteen cleft surgeons from South America participated in a 2-day cleft training workshop. All 16 participants performed a simulated repair, and 13 of them performed a second simulated repair. The repairs were recorded using a low-cost video camera and a newly designed camera mouth retractor attachment. Twenty-nine videos were assessed by 3 cleft surgeons using a previously developed cleft palate objective structured assessment of technical skill (CLOSATS with embedded overall score assessment) and global rating scale. The reliability of the ratings and technical performance in relation to minimum acceptable scores and previous experience was assessed. RESULTS The video setup provided acceptable recording quality for the purpose of assessment. Average intraclass correlation coefficient for the CLOSATS, global, and overall performance score was 0.69, 0.75, and 0.82, respectively. None of the novice surgeons passed the CLOSATS and global score for both sessions. One participant in the intermediate group, and 2 participants in the advanced group passed the CLOSATS and global score for both sessions. There were highly experienced participants who failed to pass the CLOSATS and global score for both sessions. CONCLUSIONS The cleft palate simulator can be practically implemented with video-recording capability to assess performance in cleft palate repair. This technology may be of assistance in assessing surgical competence in cleft palate repair.
Collapse
Affiliation(s)
- Dale J Podolsky
- Division of Plastic & Reconstructive Surgery, University of Toronto, Ontario, Canada.,Center for Image Guided Innovation and Therapeutic Intervention (CIGITI), Toronto, Ontario, Canada
| | - David M Fisher
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Karen W Wong Riff
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Ronald M Zuker
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - James M Drake
- Center for Image Guided Innovation and Therapeutic Intervention (CIGITI), Toronto, Ontario, Canada.,Division of Neurosurgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| | - Christopher R Forrest
- Division of Plastic & Reconstructive Surgery, The Hospital for Sick Children, Toronto, Ontario, Canada
| |
Collapse
|
41
|
Forslund Jacobsen M, Konge L, la Cour M, Holm L, Kjærbo H, Moldow B, Saleh GM, Thomsen ASS. Simulation of advanced cataract surgery - validation of a newly developed test. Acta Ophthalmol 2020; 98:687-692. [PMID: 32304357 PMCID: PMC7687069 DOI: 10.1111/aos.14439] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/17/2020] [Accepted: 03/22/2020] [Indexed: 01/14/2023]
Abstract
PURPOSE To develop and investigate an Eyesi simulator-based test for the more experienced cataract surgeon for evidence of validity. METHODS The study was a prospective interventional cohort study and carried out at the Copenhagen Academy for Medical Education and Simulation. The Eyesi Simulator was used for the test which was developed by three expert cataract surgeons. Ten cataract surgeons (>250 surgeries performed) and ten ophthalmic residents performed two repetitions of the test. The test consisted of four modules: Iris Expansion Ring insertion - level 1, Iris Expansion Ring extraction - level 2, Capsulorhexis - level 3 and Anterior Vitrectomy - level 6. RESULTS Internal consistency reliability showed Cronbach's alpha of 0.63. Test-retest reliabilities were significant for Iris Expansion Ring extraction - level 2 (p = 0.012) and Capsulorhexis - level 3 (p = 0.018). Differences between the two groups were only significant in both repetitions for the Iris Expansion Ring extraction - level 2 (p < 0.001 and p = 0.041, respectively). Furthermore, we found a statistically significant difference between the mean module scores for novices and the more experienced surgeons for Iris Expansion Ring insertion - level 1 (p = 0.021) and Capsulorhexis - level 3 (p = 0.019) in the first repetition. CONCLUSION The investigated modules show evidence of validity within several aspects of Messick's framework. However, the evidence is not strong enough to apply the test for certification purposes of cataract surgeons, but the modules may still be relevant in the training of advanced cataract surgical procedures.
Collapse
Affiliation(s)
- Mads Forslund Jacobsen
- Department of OphthalmologyRigshospitalet – GlostrupCopenhagenDenmark
- Copenhagen Academy for Medical Education and SimulationCentre for HRCapital Region of DenmarkCopenhagenDenmark
| | - Lars Konge
- Copenhagen Academy for Medical Education and SimulationCentre for HRCapital Region of DenmarkCopenhagenDenmark
| | - Morten la Cour
- Department of OphthalmologyRigshospitalet – GlostrupCopenhagenDenmark
| | - Lars Holm
- Department of OphthalmologyRigshospitalet – GlostrupCopenhagenDenmark
| | - Hadi Kjærbo
- Department of OphthalmologyRigshospitalet – GlostrupCopenhagenDenmark
| | - Birgitte Moldow
- Department of OphthalmologyRigshospitalet – GlostrupCopenhagenDenmark
| | - George M. Saleh
- NIHR Biomedical Research Centre at Moorfields Eye Hospital and UCL Institute of OphthalmologyLondonUK
| | - Ann Sofia Skou Thomsen
- Department of OphthalmologyRigshospitalet – GlostrupCopenhagenDenmark
- Copenhagen Academy for Medical Education and SimulationCentre for HRCapital Region of DenmarkCopenhagenDenmark
| |
Collapse
|
42
|
Alvarez-Lopez F, Maina MF, Arango F, Saigí-Rubió F. Use of a Low-Cost Portable 3D Virtual Reality Simulator for Psychomotor Skill Training in Minimally Invasive Surgery: Task Metrics and Score Validity. JMIR Serious Games 2020; 8:e19723. [PMID: 33107833 PMCID: PMC7655469 DOI: 10.2196/19723] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2020] [Revised: 09/06/2020] [Accepted: 10/02/2020] [Indexed: 12/13/2022] Open
Abstract
Background The high cost and low availability of virtual reality simulators in surgical specialty training programs in low- and middle-income countries make it necessary to develop and obtain sources of validity for new models of low-cost portable simulators that enable ubiquitous learning of psychomotor skills in minimally invasive surgery. Objective The aim of this study was to obtain validity evidence for relationships to other variables, internal structure, and consequences of testing for the task scores of a new low-cost portable simulator mediated by gestures for learning basic psychomotor skills in minimally invasive surgery. This new simulator is called SIMISGEST-VR (Simulator of Minimally Invasive Surgery mediated by Gestures - Virtual Reality). Methods In this prospective observational validity study, the authors looked for multiple sources of evidence (known group construct validity, prior videogaming experience, internal structure, test-retest reliability, and consequences of testing) for the proposed SIMISGEST-VR tasks. Undergraduate students (n=100, reference group), surgical residents (n=20), and experts in minimally invasive surgery (n=28) took part in the study. After answering a demographic questionnaire and watching a video of the tasks to be performed, they individually repeated each task 10 times with each hand. The simulator provided concurrent, immediate, and terminal feedback and obtained the task metrics (time and score). From the reference group, 29 undergraduate students were randomly selected to perform the tasks 6 months later in order to determine test-retest reliability. Results Evidence from multiple sources, including strong intrarater reliability and internal consistency, considerable evidence for the hypothesized consequences of testing, and partial confirmation for relations to other variables, supports the validity of the scores and the metrics used to train and teach basic psychomotor skills for minimally invasive surgery via a new low-cost portable simulator that utilizes interaction technology mediated by gestures. Conclusions The results obtained provided multiple sources of evidence to validate SIMISGEST-VR tasks aimed at training novices with no prior experience and enabling them to learn basic psychomotor skills for minimally invasive surgery.
Collapse
Affiliation(s)
- Fernando Alvarez-Lopez
- Faculty of Health Sciences, Universidad de Manizales, Manizales, Colombia.,Faculty of Health Sciences, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Marcelo Fabián Maina
- Faculty of Psychology and Education Sciences, Universitat Oberta de Catalunya, Barcelona, Spain
| | - Fernando Arango
- Faculty of Health Sciences, Universidad de Manizales, Manizales, Colombia
| | | |
Collapse
|
43
|
Gardeck AM, Pu X, Yang Q, Polly DW, Jones KE. The effect of simulation training on resident proficiency in thoracolumbar pedicle screw placement using computer-assisted navigation. J Neurosurg Spine 2020; 34:127-134. [PMID: 32886919 DOI: 10.3171/2020.5.spine2067] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Accepted: 05/15/2020] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Residency work-hour restrictions necessitate efficient, reproducible training. Simulation training for spinal instrumentation placement shows significant benefit to learners' subjective and objective proficiency. Cadaveric laboratories are most effective but have high cost and low availability. The authors' goal was to create a low-cost, efficient, reproducible spinal instrumentation placement simulation curriculum for neurosurgery and orthopedic surgery residents using synthetic models and 3D computer-assisted navigation, assessing subjective and objective proficiency with placement of thoracolumbar pedicle screws. METHODS Fifteen neurosurgery and orthopedic surgery residents participated in a standardized curriculum with lecture followed by two separate sessions of thoracolumbar pedicle screw placement in a synthetic spine model utilizing 3D computer-assisted navigation. Data were collected on premodule experience, time and accuracy of screw placement, and both subjective and objective ratings of proficiency. RESULTS Fifteen of 15 residents demonstrated improvement in subjective (Physician Performance Diagnostic Inventory Scale [PPDIS]) and 14 in objective (Objective Structured Assessment of Technical Skills [OSATS]) measures of proficiency in navigated screw placement with utilization of this curriculum (p < 0.001 for both), regardless of the number of cases of previous experience using thoracolumbar spinal instrumentation. Fourteen of 15 residents demonstrated decreased time per screw placement from session 1 to session 2 (p = 0.006). There was no significant difference in pedicle screw accuracy between session 1 and session 2. CONCLUSIONS A standardized curriculum using synthetic simulation training for navigated thoracolumbar pedicle screw placement results in significantly improved resident subjective and objective proficiency. Development of a nationwide competency curriculum using simulation training for spinal instrumentation placement should be considered for safe, efficient resident training.
Collapse
Affiliation(s)
| | - Xuan Pu
- 2Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota
| | - Qiuyu Yang
- 2Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota
| | - David W Polly
- 3Orthopedic Surgery, University of Minnesota, Minneapolis; and
| | - Kristen E Jones
- 1Departments of Neurosurgery and
- 2Division of Biostatistics, School of Public Health, University of Minnesota, Minneapolis, Minnesota
- 3Orthopedic Surgery, University of Minnesota, Minneapolis; and
| |
Collapse
|
44
|
Gabrysz-Forget F, Bonds M, Lovett M, Alseidi A, Ghaderi I, Nepomnayshy D. Practicing on the Advanced Training in Laparoscopic Suturing Curriculum (ATLAS): Is Mastery Learning in Residency Feasible to Achieve Expert-Level Performance in Laparoscopic Suturing? JOURNAL OF SURGICAL EDUCATION 2020; 77:1138-1145. [PMID: 32184062 DOI: 10.1016/j.jsurg.2020.02.026] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Revised: 01/25/2020] [Accepted: 02/23/2020] [Indexed: 06/10/2023]
Abstract
BACKGROUND Mastery learning assumes that given enough time and appropriate instructional strategies, most trainees will be able to achieve proficiency. Expert-level performance requires numerous hours of intensive and focus practice. We aimed to study whether it was possible for surgical trainees to achieve expert-derived proficiency level in laparoscopic suturing using the Advanced Training in Laparoscopic Suturing (ATLAS) curriculum over a short period of time. STUDY DESIGN A multicenter IRB approved prospective study included surgery residents and minimally invasive fellows. Participants underwent weekly supervised instruction and assessments of ATLAS skills and self-directed practice between sessions over 12 weeks. Participants were asked to practice until they achieved previously established proficiency benchmarks of expert laparoscopic surgeons. RESULTS Fifteen participants, PGY2 to PGY6, from 3 institutions practiced on the ATLAS curriculum. Three participants were able to achieve proficiency on the entire curriculum, with a cumulative practice time varying between 3.4 and 7.6 hours. Individual tasks had varying degrees of difficulty ranging from 85% proficiency on task 1 to 33.3% proficiency for task 6. Using a mixed-method model, the mean cumulative hours of practice to reach the benchmark threshold was estimated for each task and varied from 4.5 to 13.2 hours. The improved performance was associated with higher PGY level and proficiency in FLS. CONCLUSIONS This study demonstrates that it is possible for some senior surgical trainees to achieve proficiency in an expert-level laparoscopic suturing curriculum. This study establishes a learning curve for each ATLAS individual task. Some learners may not be able to achieve proficiency on the entire curriculum over a short period of practice. Additional studies are needed to assess how to shorten the learning curve with effective instructional methods such as expert-guided training with immediate feedback.
Collapse
Affiliation(s)
- Fanny Gabrysz-Forget
- Center for Professional Development and Simulation Training, Lahey Hospital, Burlington, Massachusetts
| | - Morgan Bonds
- Department of Surgery, Virginia Mason Medical Center, Seattle, Washington
| | - Marissa Lovett
- Department of Surgery, Arizona Simulation Technology and Education Center, University of Arizona, Tucson, Arizona
| | - Adnan Alseidi
- Department of Surgery, Virginia Mason Medical Center, Seattle, Washington
| | - Iman Ghaderi
- Deparment of Surgery, University of Arizona, Tucson, Arizona
| | - Dmitry Nepomnayshy
- Center for Professional Development and Simulation Training, Lahey Hospital, Burlington, Massachusetts.
| |
Collapse
|
45
|
Chau JK, Bilgic E, Hada T, Trepanier M, Naghawi H, Kaneva P, Mueller C, Lee L. Development and validation of a transanal endoscopic rectal purse string simulator. Tech Coloproctol 2020; 24:863-871. [PMID: 32504373 DOI: 10.1007/s10151-020-02251-4] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/04/2019] [Accepted: 05/25/2020] [Indexed: 01/02/2023]
Abstract
BACKGROUND Rectal purse string placement for transanal mesorectal excision is challenging, and practice is difficult. The objective of this study is to build an endoscopic rectal purse string simulator and provide evidence for the validity of its use. METHODS A low-cost transanal endoscopic rectal purse string simulator was created and used to measure the performance of participants. Participants included general surgery residents, fellows, and staff surgeons from several Canadian university-affiliated institutions across Canada. The performance of the rectal purse string placement was measured by the time to completion of the task as well scoring with a modified objective structured assessment of technical skills (OSATS) score and a modified advanced laparoscopic suturing (ALS) score. RESULTS Thirty-nine participants were recruited into the study. Participants were split into three groups, based on prior experience with laparoscopic suturing, for the analysis of suturing a rectal purse string on the simulator based on three performance measures. There was a significant difference found in all three measures of performance in the three groups (time to completion p = 0.014, mean blinded OSATS score p = 0.007, mean blinded ALS score p = 0.020). Participants with previous laparoscopic suturing had significantly faster times to completion and higher skills scores when compared to residents (time: 5.1 ± 1.4 min vs 9.0 ± 4.8 min, p = 0.005; OSATS: 19.7 ± 2.8 vs 13.0 ± 5.8, p = 0.00398, ALS: 27.4 ± 4.0 vs 18.9 ± 8.5, p = 0.0151). CONCLUSIONS A transanal endoscopic purse string simulator was constructed and preliminary testing has shown variable performance based on prior laparoscopic suturing experience.
Collapse
Affiliation(s)
- J K Chau
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada.,Department of Surgery, Centre for Colon and Rectal Surgery, McGill University Health Centre, Montreal, QC, Canada
| | - E Bilgic
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada.,Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, QC, Canada
| | - T Hada
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada
| | - M Trepanier
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada
| | - H Naghawi
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada
| | - P Kaneva
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada
| | - C Mueller
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada
| | - L Lee
- Department of Surgery, Steinberg-Bernstein Centre for Minimally Invasive Surgery and Innovation, McGill University Health Centre, 1001 Decarie Boulevard, DS1-3310, Montreal, QC, H4A 3J1, Canada. .,Department of Surgery, Centre for Colon and Rectal Surgery, McGill University Health Centre, Montreal, QC, Canada.
| |
Collapse
|
46
|
St-Louis E, Shaheen M, Mukhtar F, Adessky R, Meterissian S, Boutros M. Towards Development of an Open Surgery Competency Assessment for Residents (OSCAR) Tool - A Systematic Review of the Literature and Delphi Consensus. JOURNAL OF SURGICAL EDUCATION 2020; 77:438-453. [PMID: 31889689 DOI: 10.1016/j.jsurg.2019.10.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 08/16/2019] [Revised: 10/02/2019] [Accepted: 10/06/2019] [Indexed: 06/10/2023]
Abstract
OBJECTIVE Competency-based education has mandated accurate intra-operative assessment tools. We aimed to define consensus-based open surgical skills perceived by experts as critical for assessment. DESIGN A mixed-method design was employed: systematic review and e-Delphi methodology. SETTING The study was performed at McGill University-affiliated large tertiary academic centers in Montreal, Quebec, Canada. PARTICIPANTS Per PRISMA guidelines, a peer-reviewed search strategy was employed. Studies published in English and those describing technical skill assessment of open abdominal surgery were included; subspecialty-specific skills, conference abstracts, academic memoirs were excluded. Most-cited skills were subjected to e-Delphi methodology to identify those deemed essential by experts, based a 3-point Likert scale. Eighteen McGill University-affiliated general surgeons, representing a variety of subspecialties of General Surgery, were invited to answer the questionnaire. RESULTS Around 120 of 4285 references were retained for analysis. The 12 most cited skills included suturing, tissue and instrument handling, movement economy, instrument knowledge, knot tying, flow, knowledge of procedure, completion time, dissection technique, knowledge of anatomy and sterile technique; 6 of these achieved high or perfect scores and agreement after 2 rounds of survey: suturing, sterile technique, knot tying, knowledge of anatomy, knowledge of procedure, and tissue handling. Median standard deviation decreased (0.495 to 0.450) from first to second round, indicating improvement in consensus. CONCLUSION These results will help develop and validate the OSCAR (objective structured clinical assessment rubric) assessment tool for immediate intra-operative feedback of open technical skills for surgical trainees.
Collapse
Affiliation(s)
- Etienne St-Louis
- Colorectal Surgery, Jewish General Hospital, Montreal, Quebec, Canada; Department of General Surgery, McGill University Health Centre, Montreal, Quebec, Canada
| | - Mohammed Shaheen
- Colorectal Surgery, Jewish General Hospital, Montreal, Quebec, Canada; Department of General Surgery, McGill University Health Centre, Montreal, Quebec, Canada
| | - Fareeda Mukhtar
- Center for Medical Education, McGill University, Montreal, Quebec, Canada
| | - Ryan Adessky
- Department of General Surgery, University of Toronto, Toronto, Ontario, Canada
| | - Sarkis Meterissian
- Department of General Surgery, McGill University Health Centre, Montreal, Quebec, Canada
| | - Marylise Boutros
- Colorectal Surgery, Jewish General Hospital, Montreal, Quebec, Canada.
| |
Collapse
|
47
|
Technical Performance as a Predictor of Clinical Outcomes in Laparoscopic Gastric Cancer Surgery. Ann Surg 2020; 270:115-120. [PMID: 29578907 DOI: 10.1097/sla.0000000000002741] [Citation(s) in RCA: 41] [Impact Index Per Article: 10.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/13/2023]
Abstract
OBJECTIVE The purpose of this study was to evaluate the relationship between technical performance and patient outcomes in laparoscopic gastric cancer surgery. BACKGROUND Laparoscopic gastrectomy for cancer is an advanced procedure with high rate of postoperative morbidity and mortality. Many variables including patient, disease, and perioperative management factors have been shown to impact postoperative outcomes; however, the role of surgical performance is insufficiently investigated. METHODS A retrospective review was performed for all patients who had undergone laparoscopic gastrectomy for cancer at 3 teaching institutions between 2009 and 2015. Patients with available, unedited video-recording of their procedure were included in the study. Video files were rated for technical performance, using Objective Structured Assessments of Technical Skills (OSATS) and Generic Error Rating Tool instruments. The main outcome variable was major short-term complications. The effect of technical performance on patient outcomes was assessed using logistic regression analysis with backward selection strategy. RESULTS Sixty-one patients with available video recordings were included in the study. The overall complication rate was 29.5%. The mean Charlson comorbidity index, type of procedure, and the global OSATS score were included in the final predictive model. Lower performance score (OSATS ≤29) remained an independent predictor for major short-term outcomes (odds ratio 6.49), while adjusting for comorbidities and type of procedure. CONCLUSIONS Intraoperative technical performance predicts major short-term outcomes in laparoscopic gastrectomy for cancer. Ongoing assessment and enhancement of surgical skills using modern, evidence-based strategies might improve short-term patient outcomes. Future work should focus on developing and studying the effectiveness of such interventions in laparoscopic gastric cancer surgery.
Collapse
|
48
|
Yokoyama S, Mizunuma K, Kurashima Y, Watanabe Y, Mizota T, Poudel S, Kikuchi T, Kawai F, Shichinohe T, Hirano S. Evaluation methods and impact of simulation-based training in pediatric surgery: a systematic review. Pediatr Surg Int 2019; 35:1085-1094. [PMID: 31396735 DOI: 10.1007/s00383-019-04539-5] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 08/01/2019] [Indexed: 11/28/2022]
Abstract
PURPOSE The aim of this study was to identify (1) the type of skill evaluation methods and (2) how the effect of training was evaluated in simulation-based training (SBT) in pediatric surgery. METHODS Databases of PubMed, Cochrane Library, and Web of Science were searched for articles published from January 2000 to January 2017. Search concepts of Medical Subject Heading terms were "surgery," "pediatrics," "simulation," and "training, evaluation." RESULTS Of 5858 publications identified, 43 were included. Twenty papers described simulators as assessment tools used to evaluate technical skills. Reviewers differentiated between experts and trainees using a scoring system (45%) and/or a checklist (25%). Simulators as training tools were described in 23 papers. While the training's effectiveness was measured using performance assessment scales (52%) and/or surveys (43%), no study investigated the improvement of the clinical outcomes after SBT. CONCLUSION Scoring, time, and motion analysis methods were used for the evaluation of basic techniques of laparoscopic skills. Only a few SBT in pediatric surgery have definite goals with clinical effect. Future research needs to demonstrate the educational effect of simulators as assessment or training tools on SBT in pediatric surgery.
Collapse
Affiliation(s)
- Shinichiro Yokoyama
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Kenichi Mizunuma
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Yo Kurashima
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan. .,Clinical Simulation Center, Hokkaido University, Sapporo, Japan. .,Department of Gastroenterological Surgery II, Hokkaido University Faculty of Medicine, Kita 15, Nishi 7, Kita-ku, Sapporo, 060-8638, Japan.
| | - Yusuke Watanabe
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Tomoko Mizota
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan.,Department of Surgery, National Hospital Organization Hakodate Hospital, Hakodate, Japan
| | - Saseem Poudel
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan.,Department of General Surgery, Steel Memorial Muroran Hospital, Muroran, Japan
| | | | - Fujimi Kawai
- St. Luke's International University Library, Tokyo, Japan
| | - Toshiaki Shichinohe
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| | - Satoshi Hirano
- Department of Gastrointestinal Surgery II, Hokkaido University Graduate School of Medicine, Sapporo, Japan
| |
Collapse
|
49
|
How Do Thresholds of Principle and Preference Influence Surgeon Assessments of Learner Performance? Ann Surg 2019; 268:385-390. [PMID: 28463897 DOI: 10.1097/sla.0000000000002284] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/21/2023]
Abstract
OBJECTIVE The present study asks whether intraoperative principles are shared among faculty in a single residency program and explores how surgeons' individual thresholds between principles and preferences might influence assessment. BACKGROUND Surgical education continues to face significant challenges in the implementation of intraoperative assessment. Competency-based medical education assumes the possibility of a shared standard of competence, but intersurgeon variation is prevalent and, at times, valued in surgical education. Such procedural variation may pose problems for assessment. METHODS An entire surgical division (n = 11) was recruited to participate in video-guided interviews. Each surgeon assessed intraoperative performance in 8 video clips from a single laparoscopic radical left nephrectomy performed by a senior learner (>PGY5). Interviews were audio recorded, transcribed, and analyzed using the constant comparative method of grounded theory. RESULTS Surgeons' responses revealed 5 shared generic principles: choosing the right plane, knowing what comes next, recognizing normal and abnormal, making safe progress, and handling tools and tissues appropriately. The surgeons, however, disagreed both on whether a particular performance upheld a principle and on how the performance could improve. This variation subsequently shaped their reported assessment of the learner's performance. CONCLUSIONS The findings of the present study provide the first empirical evidence to suggest that surgeons' attitudes toward their own procedural variations may be an important influence on the subjectivity of intraoperative assessment in surgical education. Assessment based on intraoperative entrustment may harness such subjectivity for the purpose of implementing competency-based surgical education.
Collapse
|
50
|
Abstract
OBJECTIVE To identify current trends in the use of validity frameworks in surgical simulation, to provide an overview of the evidence behind the assessment of technical skills in all surgical specialties, and to present recommendations and guidelines for future validity studies. SUMMARY OF BACKGROUND DATA Validity evidence for assessment tools used in the evaluation of surgical performance is of paramount importance to ensure valid and reliable assessment of skills. METHODS We systematically reviewed the literature by searching 5 databases (PubMed, EMBASE, Web of Science, PsycINFO, and the Cochrane Library) for studies published from January 1, 2008, to July 10, 2017. We included original studies evaluating simulation-based assessments of health professionals in surgical specialties and extracted data on surgical specialty, simulator modality, participant characteristics, and the validity framework used. Data were synthesized qualitatively. RESULTS We identified 498 studies with a total of 18,312 participants. Publications involving validity assessments in surgical simulation more than doubled from 2008 to 2010 (∼30 studies/year) to 2014 to 2016 (∼70 to 90 studies/year). Only 6.6% of the studies used the recommended contemporary validity framework (Messick). The majority of studies used outdated frameworks such as face validity. Significant differences were identified across surgical specialties. The evaluated assessment tools were mostly inanimate or virtual reality simulation models. CONCLUSION An increasing number of studies have gathered validity evidence for simulation-based assessments in surgical specialties, but the use of outdated frameworks remains common. To address the current practice, this paper presents guidelines on how to use the contemporary validity framework when designing validity studies.
Collapse
|