Construct validity of a simulator for myringotomy with ventilation tube insertion

Defining the Learning Curve for Endoscopic Ear Skills Using a Modular Trainer: A Multi-Institutional Study

OBJECTIVE

Quantify the learning curve for endoscopic ear skills acquisition in otolaryngology residents using a simulator. The secondary objective was to determine if demographic factors or previous endoscopic experience influenced skill development.

STUDY DESIGN

Prospective, multicenter study. Resident participants each completed 10 amassed trials using a validated endoscopic ear skill trainer.

SETTING

Two academic teaching hospitals.

SUBJECTS

Otolaryngology residents.

MAIN OUTCOME MEASURES

Trial completion times; rate of improvement over time.

RESULTS

Thirty-eight residents completed the study, 26 from program A and 12 from program B. Fifteen participants were women and 23 were men. Mean age was 30 years old (range 26 to 34 years). Previous experience with otoendoscopy (B = -16.7, p = 0.005) and sinus endoscopy (B = -23.4, p = 0.001) independently correlated with lower overall trial times. Age, gender, postgraduate year, handedness, interest in otology, and video gaming were not associated with trial times. On multivariate logistic regression, resident completion times improved with trial number, and residents without previous endoscopy experience improved at a faster rate than those with experience ( p < 0.001).

CONCLUSIONS

Novice surgeons may acquire basic endoscopic ear experience with self-directed simulation training. The learning curve for transcanal endoscopic ear surgery is comparable to those demonstrated for other otologic surgeries, and specific task competencies can be achieved within 10 trials, suggesting that previous experiences, or lack thereof, may not dictate the ability to acquire new skills. There may be a translational value to previous endoscopic sinus experience on learning transcanal endoscopic ear surgery.

Thinking Beyond the Temporal Bone Lab: A Systematic Process for Expanding Surgical Simulation in Otolaryngology Training

The COVID-19 pandemic led to a temporary lapse in the development of otolaryngology trainee operative skills due to the cancellation of elective procedures and redeployment of trainees and attendings to COVID-19 units. Although transient, this disruption provided an opportunity for otolaryngology programs to develop contingency plans and formalize nascent simulation training curricula. Integration of formal simulation training alongside current didactic and surgical education may offset lost exposure during surgically lean times while providing the framework and resources for enhanced baseline training. Here, we provide an up-to-date overview of surgical simulation models in otolaryngology and identify easily implementable, low-cost, low fidelity models for junior trainees. By taking advantage of rapid advancements in technology and a paradigm shift to a more hands-on approach in medical education, formal simulation training may prove to be a beneficial tool at all stages of residency training, allowing for expanded peer-mentored skill development and providing a safe haven during unforeseen disruptions in surgical case volume.

Current Evidence for Simulation-Based Training and Assessment of Myringotomy and Ventilation Tube Insertion: A Systematic Review

OBJECTIVE

Myringotomy and ventilation tube insertion (MT) is a key procedure in otorhinolaryngology and can be trained using simulation models. We aimed to systematically review the literature on models for simulation-based training and assessment of MT and supporting educational evidence.

DATABASES REVIEWED

PubMed, Embase, Cochrane Library, Web of Science, Directory of Open Access Journals.

METHODS

Inclusion criteria were MT training and/or skills assessment using all types of training modalities and learners. Studies were divided into 1) descriptive and 2) educational interventional/observational in the analysis. For descriptive studies, we provide an overview of available models including materials and cost. Educational studies were appraised using Kirkpatrick's level of educational outcomes, Messick's framework of validity, and a structured quality assessment tool.

RESULTS

Forty-six studies were included consisting of 21 descriptive studies and 25 educational studies. Thirty-one unique physical and three virtual reality simulation models were identified. The studies report moderate to high realism of the different simulators and trainees and educators perceive them beneficial in training MT skills. Overall, simulation-based training is found to reduce procedure time and errors, and increase performance as measured using different assessment tools. None of the studies used a contemporary validity framework and the current educational evidence is limited.

CONCLUSION

Numerous simulation models and assessment tools have been described in the literature but educational evidence and systematic implementation into training curricula is scarce. There is especially a need to establish the effect of simulation-based training of MT in transfer to the operating room and on patient outcomes.

Combined web and haptic simulation system: A pilot study

OBJECTIVES

Simulation education is increasingly used in otolaryngology training. The objective of this pilot study is to describe a mixed web-based and haptic modular platform for pediatric airway conditions.

METHODS

A web-based modular case presentation and evaluation system were developed to correspond to a pediatric laryngeal condition represented on the haptic models created using 3D printing and silicone elastomer. The haptic microlaryngoscopy simulation training and web module were evaluated by participants using a Likert scale. The mixed modular apparatus systematized the open web platform and haptic simulator, videos, text, questions, and evaluation to provide a more comprehensive simulation experience.

RESULTS

Seventeen otolaryngology trainees participated, each completing a post-simulation survey. All participants responded 4 or higher (agree or strongly agree) on the Likert scale that the virtual training and case presentations improved their understanding of the pediatric laryngeal conditions presented.

CONCLUSION

A modular system constitutes a variety of web-based cases for evaluating, diagnosing, and performing procedures on a haptic surgical simulator taking the physician in training from diagnosis to treatment thereby expanding our current use of haptic simulators in surgical education to include case based learning and decision making.

How I do it: a simulator of the ear for developing otomicroscopy skills during the coronavirus disease 2019 pandemic

OBJECTIVE

To develop a simulator of the external auditory canal and tympanic membrane that enables surgical trainees to practise their otomicroscopy skills, which is particularly valuable at a time where there is limited patient contact because of the coronavirus disease 2019 lockdown.

METHODS

A simulator of the external auditory canal and tympanic membrane was made using a cardboard bowl, a 2 ml syringe and a latex glove. The simulator was used to practise otomicroscopy skills, including microsuction, foreign body removal, myringotomy and grommet insertion. Five doctors in the ENT department participated, ranging from core surgical training year two doctor to specialty doctor.

RESULTS

The simulator provides an effective tool on which surgical trainees can practise, develop and maintain a variety of otomicroscopy skills.

CONCLUSION

This inexpensive, easy and quick-to-make simulator enables trainees to practise their otomicroscopy skills on an approximately accurate model during a time when there is minimal clinical opportunity to develop these skills, particularly because of the coronavirus disease 2019 pandemic.

Multi-institutional Comparison of Temporal Bone Models: A Collaboration of the AAO-HNSF 3D-Printed Temporal Bone Working Group

Objective

The American Academy of Otolaryngology–Head and Neck Surgery Foundation’s (AAO-HNSF’s) 3D-Printed Temporal Bone Working Group was formed with the goal of sharing information and experience relating to the development of 3D-printed temporal bone models. The group conducted a multi-institutional study to directly compare several recently developed models.

Study Design

Expert opinion survey.

Setting

Temporal bone laboratory.

Methods

The working group convened in 2018. The various methods in which 3D virtual models had been created and printed in physical form were then shared and recorded. This allowed for comparison of the advantages, disadvantages, and costs of each method. In addition, a drilling event was held during the October 2018 AAO-HNSF Annual Meeting. Each model was drilled and evaluated by attending-level working group members using an 15-question Likert scale questionnaire. The models were graded on anatomic accuracy as well as their suitability as a simulation of both cadaveric and operative temporal bone drilling.

Results

The models produced for this study demonstrate significant anatomic detail and a likeness to human cadaver specimens for drilling and dissection. Models printed in standard resin material with a stereolithography printer scored highest in the evaluation, though the margin of difference was negligible in several categories.

Conclusion

Simulated 3D temporal bones created through a number of printing methods have potential benefit in surgical training, preoperative simulation for challenging otologic cases, and the standardized testing of temporal bone surgical skills.

SimTube: A National Simulation Training and Research Project

OBJECTIVE

To test the feasibility and impact of a simulation training program for myringotomy and tube (M&T) placement.

STUDY DESIGN

Prospective randomized controlled.

SETTING

Multi-institutional.

SUBJECTS AND METHODS

An M&T simulator was used to assess the impact of simulation training vs no simulation training on the rate of achieving competency. Novice trainees were assessed using posttest simulator Objective Structured Assessment of Technical Skills (OSATS) scores, OSATS score for initial intraoperative tube insertion, and number of procedures to obtain competency. The effect of simulation training was analyzed using χ² tests, Wilcoxon-Mann-Whitney tests, and Cox proportional hazards regression.

RESULTS

A total of 101 residents and 105 raters from 65 institutions were enrolled; however, just 63 residents had sufficient data to be analyzed due to substantial breaches in protocol. There was no difference in simulator pretest scores between intervention and control groups; however, the intervention group had better OSATS global scores on the simulator (17.4 vs 13.7, P = .0003) and OSATS task scores on the simulator (4.5 vs 3.6, P = .02). No difference in OSATS scores was observed during initial live surgery rating (P = .73 and P = .41). OSATS scores were predictive of the rate at which residents achieved competence in performing myringotomy; however, the intervention was not associated with subsequent OSATS scores during live surgeries (P = .44 and P = .91) or the rate of achieving competence (P = .16).

CONCLUSIONS

A multi-institutional simulation study is feasible. Novices trained using the M&T simulator achieved higher scores on simulator but not initial intraoperative OSATS, and they did not reach competency sooner than those not trained on the simulator.

Low cost, easy-to-replicate myringotomy tube insertion simulation model

INTRODUCTION

Simulation is an established part of modern surgical education. Several training centers have proposed different simulation models for myringotomy tube (MT) placement and validated their effectiveness in medical student and resident training. None is widely used. Early models were simple tubes that lacked important microsurgical elements. Newer simulators are more comprehensive, but are difficult and expensive to build. We present a MT placement simulator that is low cost, easy to construct with basic power tools and allows for acquisition of the most necessary MT placement skills.

METHODS

The model incudes a rotating spherical "head", a 4 mm oval speculum, a drilled-out working shaft similar in size to the external auditory canal, and a realistic paper tympanic membrane target, set at an anatomically correct angle. To evaluate the model's efficacy, we assessed the performance of 10 surgically naïve medical student volunteers before training and after 30 min of instruction with the model. Their speed was recorded and operative performance was assessed using a validated Global Rating Scale.

RESULTS

After 30 min of practice on the model, there was significant improvement in MT placement skill scores and significant decrease in time for tube placement (p < 0.05).

CONCLUSION

This MT placement simulation model is inexpensive and easy to build. Unlike existing planar models, it simulates patient head orientation, and requires realistic hand positioning on a 4 mm speculum. Practice with the model for 30 min resulted in statistically significant improvement in MT placement skill scores for inexperienced student surgeons.

Creating a Validated Simulation Training Curriculum in Otolaryngology

Purpose of Review

Simulation-based training is an integral component of surgical training. It allows practice of technical skills within a safe environment without compromising patient safety. This article seeks to review current virtual and non-virtual reality simulation models within the literature and review their validation status.

Recent Findings

Many simulation models exist within otolaryngology and are currently being used for education. New models are also continuously being developed; however, validity should be proven for the models before incorporating their use for educational purposes. Validity should be determined by experts and trainees themselves.

Summary

A validated simulation curriculum should be incorporated within the otolaryngology training programme. A curriculum based on the current training programme at our institution serves as an exemplar for local adoption.

Evaluation of a low-fidelity ear surgery simulator in a low-resource setting

Objective:

The provision of healthcare education in developing countries is a complex problem that simulation has the potential to help. This study aimed to evaluate the effectiveness of a low-cost ear surgery simulator, the Ear Trainer.

Methods:

The Ear Trainer was assessed in two low-resource environments in Cambodia and Uganda. Participants were video-recorded performing four specific middle-ear procedures, and blindly scored using a validated measurement tool. Face validity, construct validity and objective learning were assessed.

Results:

The Ear Trainer provides a realistic representation of the ear. Construct validity assessment confirmed that experts performed better than novices. Participants displayed improvement in all tasks except foreign body removal, likely because of a ceiling effect.

Conclusion:

This study validates the Ear Trainer as a useful training tool for otological microsurgical skills in developing world settings.

Otologic Skills Training

This article presents a summary of the current simulation training for otologic skills. There is a wide variety of educational approaches, assessment tools, and simulators in use, including simple low-cost task trainers to complex computer-based virtual reality systems. A systematic approach to otologic skills training using adult learning theory concepts, such as repeated and distributed practice, self-directed learning, and mastery learning, is necessary for these educational interventions to be effective. Future directions include development of measures of performance to assess efficacy of simulation training interventions and, for complex procedures, improvement in fidelity based on educational goals.

Current Status of Simulation in Otolaryngology: A Systematic Review

OBJECTIVE

Otolaryngology is a highly technical and demanding specialty and the requirements for surgical trainees to acquire proficiency remains challenging. Simulation has been purported to be an effective tool in assisting with this. The aim of this systematic review is to identify the available otolaryngology simulators, their status of validation, and evaluation the level of evidence behind each training model and thereby establish a level of recommendation.

DESIGN

PubMed, ERIC, and Google Scholar databases were searched for articles that described otolaryngology simulators or training models between 1980 and April 2016. Any validation studies for simulators were also retrieved. Titles and abstracts were screened for relevance using the preferred reporting items for systematic reviews and meta-analysis (PRISMA) guidelines. Level of evidence (LoE) and Level of recommendation (LoR) was awarded to each study and model, respectively.

RESULTS

A total of 70 studies were identified describing 64 simulators. Out of these, at least 54 simulators had 1 validation study. Simulators for the ear and temporal bone surgery were the most common (n = 32), followed by laryngeal and throat (n = 20) and endoscopic sinus surgery (n = 12). Face validity was evaluated by 29 studies, 20 attempted to show construct, 20 assessed content, 20 transfer, and only 2 assessed concurrent validity. Of the validation assessments, 2 were classified as Level 1b, 10 Level 2a, and 48 Level 2b. No simulators received the highest LoR, but 8 simulators received a LoR of 2.

CONCLUSIONS

Despite the lack of evidence in outcome studies and limited number of high-validity otolaryngology simulators, the role of simulation continues to grow across surgical specialties Hence, it is imperative that the simulators are of high validity and construct for trainees to practice and rehearse surgical skills to develop confidence.

Ventilation tube insertion simulation: a literature review and validity assessment of five training models

OBJECTIVE

The objective of this study was to identify and investigate the face and content validity of ventilation tube insertion (VTI) training models described in the literature.

DESIGN

A review of literature was carried out to identify articles describing VTI simulators. Feasible models were replicated and assessed by a group of experts.

SETTING

Postgraduate simulation centre.

PARTICIPANTS

Experts were defined as surgeons who had performed at least 100 VTI on patients. Seventeen experts were participated ensuring sufficient statistical power for analysis.

MAIN OUTCOME MEASURES

A standardised 18-item Likert-scale questionnaire was used. This addressed face validity (realism), global and task-specific content (suitability of the model for teaching) and curriculum recommendation.

RESULTS

The search revealed eleven models, of which only five had associated validity data. Five models were found to be feasible to replicate. None of the tested models achieved face or global content validity. Only one model achieved task-specific validity, and hence, there was no agreement on curriculum recommendation.

CONCLUSIONS

The quality of simulation models is moderate and there is room for improvement. There is a need for new models to be developed or existing ones to be refined in order to construct a more realistic training platform for VTI simulation.

Face and content validity of a virtual-reality simulator for myringotomy with tube placement

BACKGROUND

Myringotomy with tube insertion can be challenging for junior Otolaryngology residents as it is one of the first microscopic procedures they encounter. The Western myringotomy simulator was developed to allow trainees to practice microscope positioning, myringotomy, and tube placement. This virtual-reality simulator is viewed in stereoscopic 3D, and a haptic device is used to manipulate the digital ear model and surgical tools.

OBJECTIVE

To assess the face and content validity of the Western myringotomy simulator.

METHODS

The myringotomy simulator was integrated with new modules to allow speculum placement, manipulation of an operative microscope, and insertion of the ventilation tube through a deformable tympanic membrane. A questionnaire was developed in consultation with instructing surgeons. Fourteen face validity questions focused on the anatomy of the ear, simulation of the operative microscope, appearance and movement of the surgical instruments, deformation and cutting of the eardrum, and myringotomy tube insertion. Six content validity questions focused on training potential on surgical tasks such as speculum placement, microscope positioning, tool navigation, ear anatomy, myringotomy creation and tube insertion. A total of 12 participants from the Department of Otolaryngology-Head and Neck Surgery were recruited for the study. Prior to completing the questionnaire, participants were oriented to the simulator and given unlimited time to practice until they were comfortable with all of its aspects.

RESULTS

Responses to 12 of the 14 questions on face validity were predominantly positive. One issue of concern was with contact modeling related to tube insertion into the eardrum, and the second was with the movement of the blade and forceps. The former could be resolved by using a higher resolution digital model for the eardrum to improve contact localization. The latter could be resolved by using a higher fidelity haptic device. With regard to content validity, 64% of the responses were positive, 21% were neutral, and 15% were negative.

CONCLUSIONS

The Western myringotomy simulator appears to have sufficient face and content validity. Further development with automated metrics and skills transference testing is planned.

Face and content validity of a novel, web-based otoscopy simulator for medical education

Background

Despite the fact that otoscopy is a widely used and taught diagnostic tool during medical training, errors in diagnosis are common. Physical otoscopy simulators have high fidelity, but they can be expensive and only a limited number of students can use them at a given time.

Objectives

1) To develop a purely web-based otoscopy simulator that can easily be distributed to students over the internet. 2) To assess face and content validity of the simulator by surveying experts in otoscopy.

Methods

An otoscopy simulator, OtoTrain™, was developed at Western University using web-based programming and Unity 3D. Eleven experts from academic institutions in North America were recruited to test the simulator and respond to an online questionnaire. A 7-point Likert scale was used to answer questions related to face validity (realism of the simulator), content validity (expert evaluation of subject matter and test items), and applicability to medical training.

Results

The mean responses for the face validity, content validity, and applicability to medical training portions of the questionnaire were all ≤3, falling between the “Agree”, “Mostly Agree”, and “Strongly Agree” categories. The responses suggest good face and content validity of the simulator. Open-ended questions revealed that the primary drawbacks of the simulator were the lack of a haptic arm for force feedback, a need for increased focus on pneumatic otoscopy, and few rare disorders shown on otoscopy.

Conclusion

OtoTrain™ is a novel, web-based otoscopy simulator that can be easily distributed and used by students on a variety of platforms. Initial face and content validity was encouraging, and a skills transference study is planned following further modifications and improvements to the simulator.

Novel Peritonsillar Abscess Task Simulator

The management of peritonsillar abscesses is a skill developed early in residency training. Although drainage is not technically complicated, the procedure is intimidating to the neophyte. Task simulators have become increasingly common to provide training opportunities in a controlled environment. The authors designed a peritonsillar abscess simulator using a latex moulage of the oral cavity and other common materials. Twelve medical professionals of various levels of experience were instructed to expose, anesthetize, aspirate, and drain the simulated abscess. After completion, a questionnaire was completed by each volunteer. Initial impressions were positive that the model adequately replicated the tasks requisite for abscess drainage and was suitable as an instructional device. The initial construct cost was approximately 10 dollars, with disposables costing roughly 25 cents. Further research is under way to formally assess the simulator for face, content, and construct validity.

Interval vs Massed Training

Objective

To compare 2 different training paradigms, massed vs interval training, when novice students learn a surgical procedure, myringotomy with ventilation tube insertion, on a validated surgical simulator.

Study Design

Medical students were randomized into 2 training groups: the interval group (n = 19) was trained to perform the procedure in 5 trials/d over 3 days, and the massed group (n = 21) was trained to perform the procedure in 15 trials all in 1 session. One week later, all students were tested in 5 additional final trials. Pre- and posttest surveys were administered.

Setting

Academic medical center.

Subjects and Methods

Forty medical students: 19 students in the interval group were compared with 21 students in the massed group. Time to complete the procedure and number and type of error made were recorded and compared between groups. Pre- and poststudy surveys examined confidence levels working under a microscope and with the procedure.

Results

Students in both groups had a significant decrease in time between practice and final trials. In the final 5 trials, there was no difference in average time to complete the procedure between the massed and interval training groups. No difference was observed in the number of errors committed per trial between initial and final trials (both groups) or between massed and interval training groups. The students’ confidence levels significantly increased across the trials, regardless of group.

Conclusion

Surgical training improves proficiency, but method of training had little impact on proficiency in performing a simulated surgical procedure in this setting.

A Systematic Review of Simulators in Otolaryngology

Objective

To conduct a systematic review of published articles that describe simulators that could be used in otolaryngology for education, skill acquisition, and/or skill improvement.

Data Sources

Ovid and Embase databases searched July 14, 2011.

Review Methods

Three hundred fifty-three abstracts were independently reviewed by both authors, then 154 eligible articles were reviewed by both authors, and 95 articles were categorized by organ system (eg, otologic); type of simulator (eg, physical, virtual); whether the simulator was a prototype, could be purchased, or was constructed; validation; and level of learning assessment. Discrepancies were resolved by re-review and discussion.

Results

In addition to 11 overview articles, 28 articles described 16 otology simulators, most of which are virtual and prototypes. Ninteen articles described 10 sinus/rhinology simulators; most are virtual surgery simulators and prototypes. Eight articles described 8 oral cavity simulators, and 8 articles described neck simulators. Seventeen articles described 13 bronchoscopy simulators; several are full-body high-technology manikins adapted from other purposes. Five articles described eclectic simulators, including some for learning nontechnical and teamwork skills. Half of the simulators have been validated. Learning levels were often not assessed or assessment was limited to the learners’ perceptions.

Conclusion

A wide variety of simulators are available or under development. Lack of unified validation concepts and limited descriptions restricted our ability to assess model characteristics, availability, and validation. Simulators are emerging as powerful tools to facilitate learning; this review may provide a platform for discussion and refinement of the information reported and analyzed in evaluating simulators.

Development and validation of a novel ear simulator to teach pneumatic otoscopy

INTRODUCTION

Otoscopy is an important skill in diagnosing conditions of the middle ear. This study evaluated the ability of a novel ear simulator to teach medical students diagnostic and pneumatic otoscopy. We hypothesized that exposure to this simulator improves the ability of medical students to apply an appropriate pneumatic pressure during insufflation and accurately identify the presence of a middle ear effusion in a simulated setting.

METHODS

An ear simulator was created to teach otoscopic skills to medical students. Third-year medical students attended a workshop on pediatric otoscopy, including a demonstration, videos, and verbal instruction on otoscope use. A cohort of these students then practiced pneumatic otoscopy with the simulator. All students, as well as a group of experts who had not been exposed to the trainer, then diagnosed the presence or absence of middle ear fluid in six simulator ears and pneumatic pressures generated were recorded.

RESULTS

Interaction with the simulator enabled students to insufflate with pressures in the proper range (0.4-20 in H2O) more often, apply an average pressure (12.7 in H2O) in the appropriate range, and diagnose middle ear fluid more accurately (79.2%) than students who were not exposed to the trainer (57.3%). The students exposed to the simulator also performed closer to the level of the experts who diagnosed presence of effusion 100% accurately with an average insufflation pressure of 3.8 in H2O.

DISCUSSION

Medical students trained with the ear simulator applied appropriate pneumatic pressure more consistently and diagnosed the presence of effusion more accurately and more like experts than students not exposed to the simulator. This ear trainer is a valuable tool for teaching pneumatic otoscopy.

Training and simulation in otolaryngology

This article focuses on key issues surrounding the needs and application of simulation technologies for technical skills training in otolaryngology. The discussion includes an overview of key topics in training and learning, the application of these issues in simulation environments, and the subsequent applications of these simulation environments to otolaryngology. Examples of past applications are presented, with discussion of how the interplay of cultural changes in surgical training in general along with the rapid advancements in technology have shaped and influenced their adoption and adaptation. The authors conclude with emerging trends and potential influences advanced simulation and training will have on technical skills training in otolaryngology.