Ophthalmic Simulated Surgical Competency Assessment Rubric for manual small-incision cataract surgery

Evaluation of the impact of short term manual small incision cataract surgery (MSICS) training program on trainees with varying prior surgical experience using international council of ophthalmology-ophthalmology surgical competency assessment rubrics (ICO-OSCAR)

PURPOSE

To assess the learning curve of MSICS in three different groups of trainees with varying prior MSICS experience. To evaluate the effectiveness of ICO OSCAR for objective assessment of surgical skill transfer.

METHODS

Ninety-five MSICS trainees were divided into three groups as 1st year resident, fellow and external trainee. Each group were evaluated for their surgical skill acquisition during one month MSICS training program using ICO-OSCAR. Each trainee performed an average of 19 surgeries. The progress in the learning curve of the three groups of trainees was analyzed by evaluating the mean scores in sets of five consecutive cases. Complications during the training period were also noted.

RESULTS

The study evaluated a total of 1842 cases. The fellows and external trainees, with prior MSICS experience, had an initial mean score of 57.57 ± 16.16 and 56.86 ± 17.82 respectively, whereas the 1st year resident group had a relatively low initial mean score of 45.91(p = 0.009). The difference in mean scores between the 1st year resident group and other groups significantly reduced towards the end of training. The most common complications made by 1st year residents were in sclero-corneal tunnel construction. The external trainee group had statistically significant higher rates of zonular dialysis in the study.

CONCLUSIONS

ICO-OSCAR is an effective tool for assessing MSICS training program. Structured short term MSICS surgical training program is effective in surgical skill transfer, especially in novice surgeons.

Comparing the impact of surgical expert versus non-ophthalmologist instructors on virtual-reality surgical performance: A randomized controlled trial

PURPOSE

To compare Manual Small Incision Cataract Surgery (MSICS) microsurgical performance in course participants who received virtual reality simulation-based training by either a surgical expert or a non-ophthalmologist instructor.

SETTING

Copenhagen Academy for Medical Education and Simulation, Copenhagen, Denmark.

DESIGN

Randomized controlled trial.

METHODS

Residents and specialists in ophthalmology with no prior MSICS experience were included to receive virtual reality simulation training in MSICS using the HelpMeSee simulator. The participants were randomly allocated to receive training from either an experienced MSICS surgeon or a non-ophthalmologist, also known as near-peer teaching. The performances of the participants were evaluated at baseline and post-training using a MSICS proficiency-based test with evidence of validity.

RESULTS

Thirty participants were included in the study and 29 completed the course. There was no significant difference in final test score between the two groups (p = 0.13). The performance score of both groups of participants increased significantly after receiving the training (p < 0.001). All participants passed the proficiency-based test after receiving the training.

CONCLUSION

We found no significant difference in surgical proficiency-level whether the participants were trained by a surgical expert or a non-ophthalmologist instructor for MSICS in a virtual-reality based setting. The findings of this study suggest that near-peer teaching within microsurgical performance potentially could be applied with teaching outcomes comparable to a surgical expert-instructor.

Skills assessment after a grape-based microsurgical course for ophthalmology residents: randomised controlled trial

AIMS

To introduce and assess a course using grapes as training models for ophthalmology residents to acquire basic microsurgical skills.

METHODS

Ophthalmology residents who were novices at microsurgery were included. Participants were randomised into a 1:1 ratio to a 4-hour training programme based on fruit models (group A) or virtual reality (VR) modulator and silicone suture pads (group B), respectively. Before and after training, questionnaires were designed to measure their self-confidence with ophthalmic operations and with their coming role as surgical assistants. After training, each participant provided their interest in further studying microsurgery and was assessed for their general competence of ophthalmic microsurgery on porcine eyes.

RESULTS

Eighty-three participants were included, with 42 ones in group A and 41 ones in group B. After training, participants in group A performed better in the uniformities of the suture span (p<0.05), suture thickness (p<0.05) and tissue protection (p<0.05) during the corneal suturing assessment. The overall scores of corneal suturing and circular capsulorhexis in the porcine eye in group A were comparable to those in group B (p=0.26 and 0.87, respectively). Group A showed a more positive attitude to withstand the training for more than 4 hours (p<0.001), as well as a higher willingness to receive more times of the training in the future (p<0.001).

CONCLUSIONS

Training models based on grapes are equal to VR simulators and silicon suture pads to provide solid training tasks for ophthalmology residents to master basic microsurgical skills, and might have advantages in lower economic cost, and easy availability.

TRIAL REGISTRATION NUMBER

ChiCTR2000040439.

A validated test has been developed for assessment of manual small incision cataract surgery skills using virtual reality simulation

This study investigates the validity evidence of metrics used for the assessment of surgical skills for Manual Small Incision Cataract Surgery (MSICS) in a virtual reality simulator. MSICS surgery is a low-cost, low-technology cataract surgery technique, which is widely used in low- and middle-income countries. However, there is a lack of cataract surgeons globally, and efficient and evidence-based training of new surgeons is needed. In order to investigate the validity of simulator metrics, we included three groups of participants: (1) MSICS novices who were ophthalmologists with no cataract surgery experience, (2) MSICS novices who were experienced phacoemulsification cataract surgeons, but with no MSICS experience, and (3) experienced phacoemulsification and MSICS surgeons. The evaluation included 11 steps of the MSICS procedure, and all simulator metrics for those steps were reviewed. Of the 55 initial metrics, 30 showed high positive discriminative ability. A test passing score of 20 out of 30 was established, and one of 15 novices with no MSICS experience (mean score 15.5) and 7 out of 10 experienced MSICS surgeons (mean score 22.7) passed the test. We have developed and established validity evidence for a test for MSICS skills in a virtual reality simulator for future use in proficiency-based training and evidence-based testing of training interventions.

Wet Laboratory Training in Ophthalmology as a Tool for Formative Assessment

Surgical skills training and assessment have always remained a critical issue for both trainees and trainers. Ophthalmology being a microsurgical specialty, its training in the operating theater further faces a lot of challenges. Wet laboratory training and simulation training were, therefore, developed so that residents acquired certain basic skills in controlled laboratory settings before they could operate on patients and patient safety could thereby be ensured. Unfortunately, most literature focuses on the benefits such practice has on operating room performance; but not much attention has been paid to the use of such training for formative assessment, feedback, and its importance in effective learning. In this article, we highlight the challenges faced in surgical skill transfer and also give an insight into how wet or dry laboratory training can be of formative value in postgraduate training.

Theoretical Frameworks in Medical Education: Using a Systematic Review of Ophthalmology Education Research to Create a Theory of Change Model

BACKGROUND

Theoretical frameworks provide a lens to examine questions and interpret results; however, they are underutilized in medical education.

OBJECTIVE

To systematically evaluate the use of theoretical frameworks in ophthalmic medical education and present a theory of change model to guide educational initiatives.

METHODS

Six electronic databases were searched for peer-reviewed, English-language studies published between 2016 and 2021 on ophthalmic educational initiatives employing a theoretical framework. Quality of studies was assessed using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach; risk of bias was evaluated using the Medical Education Research Study Quality Instrument (MERSQI) and the Accreditation Council for Graduate Medical Education (ACGME) guidelines for evaluation of assessment methods. Abstracted components of the included studies were used to develop a theory of change model.

RESULTS

The literature search yielded 1661 studies: 666 were duplicates, 834 studies were excluded after abstract review, and 132 after full-text review; 29 studies (19.2%) employing a theoretical framework were included. The theories used most frequently were the Dreyfus model of skill acquisition and Messick's contemporary validity framework. GRADE ratings were predominantly "low," the average MERSQI score was 10.04, and the ACGME recommendation for all assessment development studies was the lowest recommendation. The theory of change model outlined how educators can select, apply, and evaluate theory-based interventions.

CONCLUSIONS

Few ophthalmic medical education studies employed a theoretical framework; their overall rigor was low as assessed by GRADE, MERSQI, and ACGME guidelines. A theory of change model can guide integration of theoretical frameworks into educational initiatives.

Modern Educational Simulation-Based Tools Among Residents of Ophthalmology: A Narrative Review

INTRODUCTION

With no specific overview on simulation-based training for educational purposes among residents in ophthalmology, this narrative review attempts to highlight the current literature on modern educational simulation-based tools used to educate residents in ophthalmology.

METHODS

We searched databases Web of Science and PubMed between March 15 and July 21, 2022. Relevant and accessible articles and abstracts published after 2006 and in English only were included.

RESULTS

Simulation-based cataract surgery is associated with better outcomes in the operating room and faster surgeries. Construct validity has been established across different procedures and levels in simulation-based cataract surgery and simulation-based vitreoretinal surgery. Other simulation-based procedures indicate promising results but in general lack evidence-based validity.

DISCUSSION

This narrative review highlights and evaluates the current and relevant literature of modern educational simulation-based tools to train ophthalmology residents in different fundamental skills like simulation-based ophthalmoscopy and complex surgical procedures like simulation-based cataract surgery and vitreoretinal surgery. Some studies attempt to develop simulators for the use in education of ophthalmology residents. Other studies strive to establish validity of the respective procedures or modern education tools and some studies investigate the effect of simulation-based training. The most validated modern educational simulation-based tool is the Eyesi Surgical Simulator (VRmagic, Germany). However, other modern educational simulation-based tools have also been evaluated, including the HelpMeSee Eye Surgery Simulator (HelpMeSee Inc., New York, USA) and the MicroVisTouch Surgical Simulator (ImmersiveTouch, USA).

CONCLUSION

Simulation-based training has already been established for residents in ophthalmology to benefit the most from skill-demanding procedures resulting in better learning and better patient handling. Future studies should aim to validate more simulation-based procedures for the teaching of ophthalmology residents so that the evidence is kept at a high standard.

Validity of scoring systems for the assessment of technical and non-technical skills in ophthalmic surgery-a systematic review

Evaluation and recommendation of the scoring systems for technical skills (TS) and non-technical skills (NTS) assessments in ophthalmic surgery. A literature search was performed between December 2019 and May 2020. Studies describing the development or validation of TS or NTS scoring systems in ophthalmic surgery were included. Only scoring systems for completion by hand were included. The primary outcome was the validity and reliability status for each scoring system. The secondary outcome was recommendation based on modified Oxford Centre for Evidence-Based Medicine guidelines. Nineteen and five scoring systems were identified for TS and NTS respectively. TS scoring systems exist for cataract surgery (including the steps of phacoemulsification and paediatric cataract surgery) ptosis, strabismus, lateral tarsal strip, vitrectomy, and intraocular surgery in general. NTS scoring systems apply to cataract surgery or ophthalmic surgery in general. No single scoring system satisfied all validity and reliability measures. The recommended TS scoring systems are 'International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubrics' (ICO-OSCAR) for phacoemulsification, strabismus and paediatric cataract surgery, and 'Objective Structured Assessment of Cataract Surgical Skill' (OSACSS). Non-Technical Skills for Surgeons (NOTSS), Observational Teamwork Assessment for Surgery (OTAS) and Anaesthetists Non-Technical Skills (ANTS) are recommended for NTS. There is a paucity of NTS scoring systems. Further research is required to validate all scoring systems to consistent standards. Limitations of the assessment tools included infrequent quantification of face and content validity, and inconsistency in terminology and statistical methods between studies.

Effectiveness of simulation-based training for manual small incision cataract surgery among novice surgeons: a randomized controlled trial

This study was designed to determine the effect of a novel simulation-based training curriculum for scleral tunnel construction in manual small incision cataract surgery (MSICS) compared with traditional training. In this multicenter, investigator-masked, randomized clinical trial, resident surgeons within 3 months of matriculation with minimal or no prior experience with MSICS were assigned either to simulation-based training, the Experimental Group (EG), or to conventional training, the Control Group (CG). EG residents were trained to perform scleral tunnel construction using a simulation-based curriculum (HelpMeSee Eye Surgery Simulator), while residents in the CG followed institution-specific curriculum before progressing to live surgery. Surgical videos of the first 20 attempts at tunnel construction were reviewed by masked video raters. The primary outcome was the total number of any of 9 pre-specified errors. On average, the total number of errors was 9.25 (95% CI 0–18.95) in the EG and 17.56 (95% CI 6.63–28.49) in the CG (P = 0.05); the number of major errors was 4.86 (95% CI 0.13–9.59) in the EG and 10.09 (95% CI 4.76–15.41) in the CG (P = 0.02); and the number of minor errors was 4.39 (95% CI 0–9.75) in the EG and 7.47 (95% CI 1.43–13.51) in the CG (P = 0.16). These results support that novice surgeons trained using the novel simulation-based curriculum performed fewer errors in their first 20 attempts at tunnel construction compared to those trained with a conventional curriculum.

Addressing the MSICS learning curve: identification of instrument-holding techniques used by experienced surgeons

AIM

To identify instrument holding archetypes used by experienced surgeons in order to develop a universal language and set of validated techniques that can be utilized in manual small incision cataract surgery (MSICS) curricula.

METHODS

Experienced cataract surgeons performed five MSICS steps (scleral incision, scleral tunnel, side port, corneal tunnel, and capsulorhexis) in a wet lab to record surgeon hand positions. Images and videos were taken during each step to identify validated hand position archetypes.

RESULTS

For each MSICS step, one or two major archetypes and key modifying variables were observed, including tripod for scleral incision, tripod-thumb bottom for scleral tunnel, underhand-index to thumb grip for side port, index-contact tripod for corneal entry, and tripod-forceps for capsulorhexis. Key differences were noted in thumb placement and number of fingers supporting the instrument, and modifying variables included index finger curvature and amount of flexion.

CONCLUSION

Identification of optimal hand positions and development of a formal nomenclature has the potential to help trainees adopt hand positions in an informed manner, influence instrument design, and improve surgical outcomes.

Current methods for assessing technical skill in cataract surgery

Surgery is a major source of errors in patient care. Preventing complications from surgical errors in the operating room is estimated to lead to reduction of up to 41 846 readmissions and save $620.3 million per year. It is now established that poor technical skill is associated with an increased risk of severe adverse events postoperatively and traditional models to train surgeons are being challenged by rapid advances in technology, an intensified patient-safety culture, and a need for value-driven health systems. This review discusses the current methods available for evaluating technical skills in cataract surgery and the recent technological advancements that have enabled capture and analysis of large amounts of complex surgical data for more automated objective skills assessment.

Online faculty development: An African lusophone ophthalmic society experience during the COVID-19 pandemic

PURPOSE:

Faculty development for procedural specialists aims at developing both their medical education and surgical competence. This has been challenging during the COVID-19 pandemic, especially in under-resourced settings and African Lusophone ophthalmology community has been no exception. The Mozambican College of Ophthalmology (MOC) and the Continuing Professional Development Committee of the International Council of Ophthalmology (ICO) established a collaboration to enhance simulation-based clinical teaching competence in cataract surgery.

METHODS:

Ten Mozambican ophthalmologists experienced in teaching cataract surgery participated in a group mentoring assisted 6 month/11 flipped-learning online experience on curriculum design, which included practice-based and social learning strategies, continuous bidirectional feedback, individual and group reflection opportunities, and the demonstration of declarative and procedural competencies. Program evaluation consisted of pre and post-test knowledge assessment; individual homework, informed by curated reading and a recorded lesson; feedback surveys for each module and one month after the program's conclusion, and a longitudinal project on creating a simulation-based education session on one step of cataract surgery.

RESULTS:

Participants a) highlighted the opportunity to advance their scholarly teaching skills as facilitators; b) showed an increase in knowledge post-test, expressed commitment to improve their learning experiences´ design, include interactive educational methods, and provide constructive feedback; and c) formed a sustained community of practice of ophthalmologists educators (CoP).

CONCLUSION:

This online faculty development program, assisted by group mentoring, held during the COVID-19 pandemic, facilitated the development of a CoP and was effective in enhancing teaching competence in curriculum design to apply in simulation-based learning environments.

Intense Simulation-Based Surgical Education for Manual Small-Incision Cataract Surgery: The Ophthalmic Learning and Improvement Initiative in Cataract Surgery Randomized Clinical Trial in Kenya, Tanzania, Uganda, and Zimbabwe

Importance

Cataracts account for 40% of cases of blindness globally, with surgery the only treatment.

Objective

To determine whether adding simulation-based cataract surgical training to conventional training results in improved acquisition of surgical skills among trainees.

Design, Setting, and Participants

A multicenter, investigator-masked, parallel-group, randomized clinical educational-intervention trial was conducted at 5 university hospital training institutions in Kenya, Tanzania, Uganda, and Zimbabwe from October 1, 2017, to September 30, 2019, with a follow-up of 15 months. Fifty-two trainee ophthalmologists were assessed for eligibility (required no prior cataract surgery as primary surgeon); 50 were recruited and randomized. Those assessing outcomes of surgical competency were masked to group assignment. Analysis was performed on an intention-to-treat basis.

Interventions

The intervention group received a 5-day simulation-based cataract surgical training course, in addition to standard surgical training. The control group received standard training only, without a placebo intervention; however, those in the control group received the intervention training after the initial 12-month follow-up period.

Main Outcomes and Measures

The primary outcome measure was overall surgical competency at 3 months, which was assessed with a validated competency assessment rubric. Secondary outcomes included surgical competence at 1 year and quantity and outcomes (including visual acuity and posterior capsule rupture) of cataract surgical procedures performed during a 1-year period.

Results

Among the 50 participants (26 women [52.0%]; mean [SD] age, 32.3 [4.6] years), 25 were randomized to the intervention group, and 25 were randomized to the control group, with 1 dropout. Forty-nine participants were included in the final intention-to-treat analysis. Baseline characteristics were balanced. The participants in the intervention group had higher scores at 3 months compared with the participants in the control group, after adjusting for baseline assessment rubric score. The participants in the intervention group were estimated to have scores 16.6 points (out of 40) higher (95% CI, 14.4-18.7; P < .001) at 3 months than the participants in the control group. The participants in the intervention group performed a mean of 21.5 cataract surgical procedures in the year after the training, while the participants in the control group performed a mean of 8.5 cataract surgical procedures (mean difference, 13.0; 95% CI, 3.9-22.2; P < .001). Posterior capsule rupture rates (an important complication) were 7.8% (42 of 537) for the intervention group and 26.6% (54 of 203) for the control group (difference, 18.8%; 95% CI, 12.3%-25.3%; P < .001).

Conclusions and Relevance

This randomized clinical trial provides evidence that intense simulation-based cataract surgical education facilitates the rapid acquisition of surgical competence and maximizes patient safety.

Trial Registration

Pan-African Clinical Trial Registry, number PACTR201803002159198.