1
|
Dowling E, Larson D, Carlson ML, Price DL. Development and Validation of Instrument for Operative Competency Assessment in Selective Neck Dissection. Ann Otol Rhinol Laryngol 2023; 132:173-181. [PMID: 35249359 DOI: 10.1177/00034894221081101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
BACKGROUND Instruments to assess surgical skills have been validated for several key indicator procedures in otolaryngology. Selective neck dissection is a core procedure for which trainees must integrate knowledge of complex head and neck anatomy with technical surgical skills. An instrument for assessment of surgical performance in selective neck dissection has not been previously developed. The objective of the current study is to develop and validate an instrument for assessing surgical competency for level II-IV selective neck dissection. DESIGN A Delphi working group comprised of 23 fellowship trained head and neck surgeons from 17 institutions was assembled. The modified Delphi method encompassed a 3-step process, including 2 anonymous voting rounds to successively refine individual items and establish levels of consensus. Thresholds for achieving strong consensus, at >80% agreement, were determined a priori. The resulting instrument was subsequently validated in a prospective cohort of 17 resident surgeons, spanning postgraduate year 1 to 5 training experience. Participants were asked to perform a level II-IV selective neck dissection on fresh-frozen cadaveric specimens. Performance was scored by 2 independent, blinded observers using the devised instrument and construct validity was assessed. RESULTS Through the modified Delphi process a final list of 30 items, considered to be the most essential items for achieving the goals of a level II-IV selective neck dissection, was developed. Construct validity was supported by a positive association between instrument scores compared to both resident postgraduate year level and number of head and neck rotations completed. CONCLUSION The development and validation of a novel instrument for assessment of surgical competency in level II-IV selective neck dissection, a key indicator case in otolaryngology, is described. This new instrument may be used to provide objective feedback on overall and task-specific competency to identify surgical deficiencies and offer granular feedback to enhance surgical training.
Collapse
Affiliation(s)
- Eric Dowling
- Mayo Clinic - Otorhinolaryngology, Rochester, MN, USA
| | | | | | | |
Collapse
|
2
|
Rabski JE, Saha A, Cusimano MD, Dabns F. Resident evaluations in the age of competency-based medical education: faculty perspectives on minimizing burdens. J Neurosurg 2021; 135:949-954. [PMID: 33307525 DOI: 10.3171/2020.7.jns201688] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/07/2020] [Accepted: 07/07/2020] [Indexed: 11/06/2022]
Abstract
OBJECTIVE Competency-based medical education (CBME), an outcomes-based approach to medical education, continues to be implemented across many postgraduate medical education programs worldwide, including a recent introduction into Canadian neurosurgical training programs (July 2019). The success of this educational paradigm shift requires frequent faculty observation and evaluation of residents performing defined tasks of the specialty. A main challenge involves providing residents with frequent performance evaluations and feedback that are feasible for faculty to complete. This study aims to define what is currently happening and what changes are needed to make CBME successful for the certification of neurosurgeons' competence. METHODS A 55-item questionnaire was emailed nationwide to survey Canadian neurosurgical faculty. RESULTS Fifty-two complete responses were received and achieved a distribution highly correlated with the number of faculty neurosurgeons practicing in each Canadian province (Pearson's r = 0.94). Two-thirds (35/52) of faculty reported currently taking a median of 10 minutes to complete evaluation forms at the end of a resident's rotation block. Regardless of the faculty's province of practice (p = 0.50) or years of experience (p = 0.06), they reported 3 minutes (minimum 1 minute, maximum 10 minutes, interquartile range [IQR] 3 minutes) as a feasible amount of time to spend completing an evaluation form following an observation of a resident's performance of an entrustable professional activity (EPA). If evaluation forms took 3 minutes to complete, 85% of respondents (44/52) would complete EPA evaluations weekly or daily. The faculty recommended 5 minutes as a feasible amount of time to provide oral feedback (minimum 1 minute, maximum 20 minutes, IQR 3.25 minutes), which was significantly higher (p = 0.00099) than their recommended amount of time for completing evaluation forms. The majority of faculty (71%) stated they would prefer to access resident evaluation forms through a mobile application compared to a paper form (12%), an evaluation website (8%), or through a URL link sent via email (10%; p = 0.0032). CONCLUSIONS To facilitate the successful implementation of CBME into a neurosurgical training curriculum, resident EPA assessment forms should take 3 minutes or less to complete and be accessible through a mobile application.
Collapse
Affiliation(s)
- Jessica E Rabski
- 1Injury Prevention Research Office, St. Michael's Hospital, Toronto
- 1Injury Prevention Research Office, St. Michael's Hospital, Toronto
| | - Ashirbani Saha
- 1Injury Prevention Research Office, St. Michael's Hospital, Toronto
| | | | - Frcsc Dabns
- 1Injury Prevention Research Office, St. Michael's Hospital, Toronto
- 3Department of Surgery, Division of Neurosurgery, University of Toronto, Ontario, Canada
| |
Collapse
|
3
|
Louridas M, de Montbrun S. Competency-Based Education in Minimally Invasive and Robotic Colorectal Surgery. Clin Colon Rectal Surg 2021; 34:155-162. [PMID: 33814997 DOI: 10.1055/s-0040-1718683] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
Minimally invasive and robotic techniques have become increasingly implemented into surgical practice and are now an essential part of the foundational skills of training colorectal surgeons. Over the past 5 years there has been a shift in the surgical educational paradigm toward competency-based education (CBE). CBE recognizes that trainees learn at different rates but regardless, are required to meet a competent threshold of performance prior to independent practice. Thus, CBE attempts to replace the traditional "time" endpoint of training with "performance." Although conceptually sensible, implementing CBE has proven challenging. This article will define competence, outline appropriate assessment tools to assess technical skill, and review the literature on the number of cases required to achieve competence in colorectal procedures while outlining the barriers to implementing CBE.
Collapse
Affiliation(s)
- Marisa Louridas
- Department of Surgery, University of Toronto, Toronto, Ontario, Canada
| | | |
Collapse
|
4
|
Zhao NW, O'Sullivan PS, Huang E. Enhancing Operative Feedback: A Descriptive Trajectory for Surgical Development in Otolaryngology. JOURNAL OF SURGICAL EDUCATION 2020; 77:572-581. [PMID: 31806447 DOI: 10.1016/j.jsurg.2019.11.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/03/2019] [Revised: 11/13/2019] [Accepted: 11/15/2019] [Indexed: 06/10/2023]
Abstract
OBJECTIVE Developmental frameworks are narrative descriptions of learner performance that may be useful to provide operative feedback tools in surgical education. The authors previously proposed the Technician, Anatomist, Anticipator, Strategist, Executive (TAASE) framework utilizing data collected from general surgeons and conjectured its applicability in other surgical specialties. Here, we construct a developmental framework in otolaryngology using similar techniques, then compare the results to TAASE. DESIGN Ten otolaryngology educators participated in semi-structured, audio-recorded interviews, to explore how otolaryngology faculty characterize surgical learning at different levels of training. Researchers analyzed the transcripts using a thematic analysis approach to build a developmental framework. Results were then qualitatively compared to the TAASE trajectory. SETTING Single tertiary academic medical center. PARTICIPANTS Faculty within the Department of Otolaryngology-Head and Neck Surgery. RESULTS Otolaryngology faculty characterized surgical development as an integrated trajectory of technical and cognitive skills progressing from specific operative tasks to a global understanding of patients and procedures largely consistent with stages outlined in the Technician, Anatomist, Anticipator, Strategist, and Executive model. A new theme identified was learner emotions, which may impact learning and hinder or enhance progression along the trajectory. CONCLUSIONS Otolaryngology and general surgeons agree upon a common arc to operative maturity reflected by the TAASE trajectory. The TAASE framework may be useful as a tool for intraoperative assessment in otolaryngology to help promote quality feedback. Further research is needed to understand how emotions may influence operative skill development.
Collapse
Affiliation(s)
- Nina W Zhao
- Department of Otolaryngology-Head and Neck Surgery, University of California, San Francisco, California.
| | - Patricia S O'Sullivan
- Department of Surgery, University of California, San Francisco, California; Department of Medicine, University of California, San Francisco, California
| | - Emily Huang
- Department of Surgery, University of California, San Francisco, California
| |
Collapse
|
5
|
Vaidya A, Aydin A, Ridgley J, Raison N, Dasgupta P, Ahmed K. Current Status of Technical Skills Assessment Tools in Surgery: A Systematic Review. J Surg Res 2020; 246:342-378. [DOI: 10.1016/j.jss.2019.09.006] [Citation(s) in RCA: 25] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2019] [Revised: 08/29/2019] [Accepted: 09/11/2019] [Indexed: 12/18/2022]
|
6
|
Fahim C, Wagner N, Nousiainen MT, Sonnadara R. Assessment of Technical Skills Competence in the Operating Room: A Systematic and Scoping Review. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2018; 93:794-808. [PMID: 28953567 DOI: 10.1097/acm.0000000000001902] [Citation(s) in RCA: 10] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/07/2023]
Abstract
PURPOSE While academic accreditation bodies continue to promote competency-based medical education (CBME), the feasibility of conducting regular CBME assessments remains challenging. The purpose of this study was to identify evidence pertaining to the practical application of assessments that aim to measure technical competence for surgical trainees in a nonsimulated, operative setting. METHOD In August 2016, the authors systematically searched Medline, Embase, and the Cochrane Database of Systematic Reviews for English-language, peer-reviewed articles published in or after 1996. The title, abstract, and full text of identified articles were screened. Data regarding study characteristics, psychometric and measurement properties, implementation of assessment, competency definitions, and faculty training were extracted. The findings from the systematic review were supplemented by a scoping review to identify key strategies related to faculty uptake and implementation of CBME assessments. RESULTS A total of 32 studies were included. The majority of studies reported reasonable scores of interrater reliability and internal consistency. Seven articles identified minimum scores required to establish competence. Twenty-five articles mentioned faculty training. Many of the faculty training interventions focused on timely completion of assessments or scale calibration. CONCLUSIONS There are a number of diverse tools used to assess competence for intraoperative technical skills and a lack of consensus regarding the definition of technical competence within and across surgical specialties. Further work is required to identify when and how often trainees should be assessed and to identify strategies to train faculty to ensure timely and accurate assessment.
Collapse
Affiliation(s)
- Christine Fahim
- C. Fahim is a PhD candidate, Department of Health Research Methods, Evidence and Impact, McMaster University, Hamilton, Ontario, Canada. N. Wagner is a PhD candidate, Department of Psychology, Neuroscience and Behaviour, McMaster University, Hamilton, Ontario, Canada. M.T. Nousiainen is orthopedic surgeon and assistant professor, Sunnybrook Hospital, Department of Surgery, University of Toronto, Toronto, Ontario, Canada. R. Sonnadara is director of education science and associate professor, Department of Surgery, McMaster University, Hamilton, Ontario, Canada, and associate professor, Department of Surgery, University of Toronto, Toronto, Ontario, Canada; ORCID: http://orcid.org/0000-0001-8318-5714
| | | | | | | |
Collapse
|
7
|
Wu KY, Kim S, Fung K, Roth K. Assessing nontechnical skills in otolaryngology emergencies through simulation-based training. Laryngoscope 2018; 128:2301-2306. [DOI: 10.1002/lary.27174] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2018] [Revised: 02/07/2018] [Accepted: 02/12/2018] [Indexed: 11/11/2022]
Affiliation(s)
- Kitty Y. Wu
- Division of Plastic and Reconstructive Surgery, Department of Surgery; London Ontario Canada
| | - Stephanie Kim
- Schulich School of Medicine and Dentistry; London Ontario Canada
| | - Kevin Fung
- Department of Otolaryngology-Head and Neck Surgery; Western University; London Ontario Canada
| | - Kathryn Roth
- Department of Otolaryngology-Head and Neck Surgery; Western University; London Ontario Canada
| |
Collapse
|
8
|
Hamour AF, Mendez AI, Harris JR, Biron VL, Seikaly H, Côté DWJ. A High-Definition Video Teaching Module for Thyroidectomy Surgery. JOURNAL OF SURGICAL EDUCATION 2018; 75:481-488. [PMID: 28780314 DOI: 10.1016/j.jsurg.2017.07.019] [Citation(s) in RCA: 15] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/23/2017] [Revised: 06/21/2017] [Accepted: 07/18/2017] [Indexed: 06/07/2023]
Abstract
OBJECTIVE With the changing landscape of postgraduate surgical education to competency-based curricula, there emerges a need for alternative forms of training. Video teaching modules have been shown to be effective tools in surgical education, complementing traditional postgraduate curricula. There is a lack of validated modules described in the literature, specifically for teaching thyroidectomy. The primary objective of this study was to develop and validate a high definition video-based teaching module instructing thyroidectomy surgery to Otolaryngology-Head and Neck Surgery trainees. DESIGN This prospective study included intermediate to senior Otolaryngology-Head and Neck Surgery residents. Each participant first performed a thyroid lobectomy, serving as the initial assessment. After a washout period of at least 3 weeks, each participant was given the teaching module. The 15-minute module was developed using a 3-camera system and detailed a step-by-step approach to the surgery. After exposure to the module, each trainee performed the same procedure. Recordings of both procedures were deidentified and reviewed by a blinded, independent evaluator. Scoring was done using the Observational Clinical Human Reliability Assessment (OCHRA) system. SETTING University of Alberta Hospital and Royal Alexandra Hospital, Edmonton, Alberta, Canada. PARTICIPANTS A total of 6 intermediate to senior Otolaryngology-Head and Neck Surgery residents entered and completed the study. RESULTS The mean error rate was 8.8 errors per procedure before module exposure and 4.5 errors per procedure after exposure, representing a 49% decrease in error occurrence (p < 0.05). The mean staff takeover event rate was 10.5 takeovers per procedure prior to module exposure and 5.0 takeovers per procedure after exposure, representing a 52% decrease in error occurrence (p < 0.05). CONCLUSION High-definition video teaching modules are a useful complement to traditional surgical training. In a climate where new innovations for teaching thyroid surgery are needed, properly constructed and validated video teaching modules can serve as important tools in supplementing traditional surgical training.
Collapse
Affiliation(s)
- Amr F Hamour
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada.
| | - Adrian I Mendez
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Jeffrey R Harris
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Vincent L Biron
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Hadi Seikaly
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - David W J Côté
- Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| |
Collapse
|
9
|
Labbé M, Young M, Nguyen LH. Validity evidence as a key marker of quality of technical skill assessment in OTL-HNS. Laryngoscope 2018; 128:2296-2300. [DOI: 10.1002/lary.27085] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2017] [Revised: 12/02/2017] [Accepted: 12/13/2017] [Indexed: 11/08/2022]
Affiliation(s)
- Mathilde Labbé
- Faculty of Medicine; McGill University; Montreal Quebec Canada
| | - Meredith Young
- Center for Medical Education; McGill University; Montreal Quebec Canada
- Department of Medicine; McGill University; Montreal Quebec Canada
| | - Lily H.P. Nguyen
- Center for Medical Education; McGill University; Montreal Quebec Canada
- Department of Otolaryngology; McGill University; Montreal Quebec Canada
| |
Collapse
|
10
|
Labbé M, Young M, Nguyen LHP. Toolbox of assessment tools of technical skills in otolaryngology-head and neck surgery: A systematic review. Laryngoscope 2017; 128:1571-1575. [DOI: 10.1002/lary.26943] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/06/2017] [Revised: 08/25/2017] [Accepted: 09/05/2017] [Indexed: 11/06/2022]
Affiliation(s)
- Mathilde Labbé
- Faculty of Medicine; McGill University; Montreal Quebec Canada
| | - Meredith Young
- Center for Medical Education; McGill University; Montreal Quebec Canada
- Department of Medicine; McGill University; Montreal Quebec Canada
| | - Lily H. P. Nguyen
- Center for Medical Education; McGill University; Montreal Quebec Canada
- Department of Otolaryngology; McGill University; Montreal Quebec Canada
| |
Collapse
|
11
|
|
12
|
Mercier É, Chagnon-Monarque S, Lavigne F, Ayad T. Objective Assessment of Technical Skills in Otorhinolaryngology-Head and Neck Surgery Residents: A Systematic Review. Otolaryngol Head Neck Surg 2017; 158:54-61. [PMID: 28925316 DOI: 10.1177/0194599817729826] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/15/2022]
Abstract
Objectives The primary goal is the indexation of validated methods used to assess surgical competency in otorhinolaryngology-head and neck surgery (ORL-HNS) residents. Secondary goals include assessment of the reliability and validity of these tools, as well as the documentation of specific procedures in ORL-HNS involved. Data Sources MEDBASE, OVID, Medline, CINAHL, and EBM, as well as the printed references, available through the Université de Montréal library. Review Methods The PRISMA method was used to review digital and printed databases. Publications were reviewed by 2 independent reviewers, and selected articles were fully analyzed to classify evaluation methods and categorize them by procedure and subspecialty of ORL-HNS involved. Reliability and validity were assessed and scored for each assessment tool. Results Through the review of 30 studies, 5 evaluation methods were described and validated to assess the surgical competency of ORL-HNS residents. The evaluation method most often described was the combined Global Rating Scale and Task-Specific Checklist tool. Reliability and validity for this tool were overall high; however, considerable data were unavailable. Eleven distinctive surgical procedures were studied, encompassing many subspecialties of ORL-HNS: facial plastics, general ear-nose-throat, laryngology, otology, pediatrics, and rhinology. Conclusions Although assessment tools have been developed for an array of surgical procedures, involving most ORL-HNS subspecialties, the use of combined checklists has been repeatedly validated in the literature and shown to be easily applicable in practice. It has been applied to many ORL-HNS procedures but not in oncologic surgery to date.
Collapse
Affiliation(s)
- Érika Mercier
- 1 Centre Hospitalier Universitaire de Montréal, Montréal, Canada
| | | | - François Lavigne
- 1 Centre Hospitalier Universitaire de Montréal, Montréal, Canada
| | - Tareck Ayad
- 1 Centre Hospitalier Universitaire de Montréal, Montréal, Canada
| |
Collapse
|
13
|
Hamour A, Mendez A, Harris JR, Biron VL, Seikaly H, Côté DW. Thyroidectomy Teaching Module. VideoEndocrinology 2017. [DOI: 10.1089/ve.2017.0092] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/12/2022] Open
Affiliation(s)
- Amr Hamour
- Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, Alberta Head and Neck Centre for Oncology and Reconstruction (AHNCOR), University of Alberta, Edmonton, Alberta, Canada
| | - Adrian Mendez
- Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Jeffrey R. Harris
- Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Vincent L. Biron
- Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - Hadi Seikaly
- Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| | - David W.J. Côté
- Division of Otolaryngology–Head and Neck Surgery, Department of Surgery, University of Alberta, Edmonton, Alberta, Canada
| |
Collapse
|
14
|
Wagner N, Fahim C, Dunn K, Reid D, Sonnadara R. Otolaryngology residency education: a scoping review on the shift towards competency-based medical education. Clin Otolaryngol 2016; 42:564-572. [DOI: 10.1111/coa.12772] [Citation(s) in RCA: 29] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/13/2016] [Indexed: 11/27/2022]
Affiliation(s)
- N. Wagner
- Department of Surgery; McMaster University; Hamilton ON Canada
| | - C. Fahim
- Department of Surgery; McMaster University; Hamilton ON Canada
| | - K. Dunn
- Department of Surgery; McMaster University; Hamilton ON Canada
| | - D. Reid
- Department of Surgery; McMaster University; Hamilton ON Canada
| | - R.R. Sonnadara
- Department of Surgery; McMaster University; Hamilton ON Canada
| |
Collapse
|
15
|
Bhatti NI, Ahmed A. Improving skills development in residency using a deliberate-practice and learner-centered model. Laryngoscope 2015; 125 Suppl 8:S1-14. [DOI: 10.1002/lary.25434] [Citation(s) in RCA: 28] [Impact Index Per Article: 3.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2015] [Accepted: 05/11/2015] [Indexed: 11/09/2022]
Affiliation(s)
- Nasir I. Bhatti
- Department of Otolaryngology-Head and Neck Surgery; Johns Hopkins University School of Medicine; Baltimore Maryland U.S.A
| | - Aadil Ahmed
- Department of Otolaryngology-Head and Neck Surgery; Johns Hopkins University School of Medicine; Baltimore Maryland U.S.A
| |
Collapse
|
16
|
Obeid AA, Al-Qahtani KH, Ashraf M, Alghamdi FR, Marglani O, Alherabi A. Development and testing for an operative competency assessment tool for nasal septoplasty surgery. Am J Rhinol Allergy 2015; 28:e163-7. [PMID: 25197910 DOI: 10.2500/ajra.2014.28.4051] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/20/2022]
Abstract
BACKGROUND Assessing surgical competency in otolaryngology is challenging, and residency programs are now responsible for ensuring the surgical competency of their graduates. Therefore, more objective assessment tools are being incorporated into the evaluation process. Objective structured assessment of technical skills (OSATSs) tools have been developed for multiple otolaryngology procedures. These include tonsillectomy, endoscopic sinus surgery, thyroidectomy, mastoidectomy, direct laryngoscopy, and rigid bronchoscopy. The purpose of this study was to develop and test a new assessment tool for septoplasty surgery and ensuring its feasibility, reliability, and construct validity. This study was designed to develop and test a valid, reliable, and feasible evaluation tool designed to measure the development of trainees' surgical skills in the operating room for septoplasty surgery. METHODS A new OSATSs-based instrument form for septoplasty was developed. During the study period of 2 years, 21 otolaryngology-head and neck surgery residents (ranging from postgraduate year 2 to 5) were evaluated intraoperatively by one faculty member obtaining a total of 175 evaluations. Surgical performance was rated using a seven-item task-specific checklist (TSC) and a global rating scale (GRS). The TSC assessed specific septoplasty technical skills, and the GRS assessed the overall surgical performance. RESULTS Our tool showed construct validity for both components of the assessment instrument, with increasing mean scores with advancing clinical levels. Cronbach's α, a measure of internal consistency, was 0.911 for TSC and 0.898 for GRS. Strong correlation between the TSC and GRS was established (r = 0.955; p < 0.01). CONCLUSION This study proved our educational tool to be a valid, reliable, and feasible method for assessing competency in septoplasty surgery. It can be integrated into surgical training programs to facilitate direct formative feedback. Assessing trainees' learning curves enables insight into their progression, ensuring their appropriate development.
Collapse
Affiliation(s)
- Amani A Obeid
- Department of Otolaryngology and Head and Neck Surgery, College of Medicine, King Saud University, Riyadh, Saudi Arabia
| | | | | | | | | | | |
Collapse
|
17
|
Laeeq K, Francis HW, Varela DADV, Malik MU, Cummings CW, Bhatti NI. The timely completion of objective assessment tools for evaluation of technical skills. Laryngoscope 2012; 122:2418-21. [DOI: 10.1002/lary.23494] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/06/2012] [Revised: 04/29/2012] [Accepted: 05/04/2012] [Indexed: 11/10/2022]
|