The learning curve for a colonoscopy simulator in the absence of any feedback: no feedback, no learning

Development and validation of a training module on the use of narrow-band imaging in differentiation of small adenomas from hyperplastic colorectal polyps

BACKGROUND

Experts are accurate in differentiating small adenomas from hyperplastic polyps at colonoscopy by using narrow-band imaging (NBI).

OBJECTIVE

To prospectively evaluate the effectiveness of an NBI training module on individuals with varying colonoscopy experience.

DESIGN

Prospective educational evaluation study.

SETTING

Academic endoscopy unit.

PARTICIPANTS

Twenty-one participants of varying colonoscopy experience (novices, trainees, and experienced gastroenterologists) and 5 experts in NBI.

INTERVENTION

Participants completed a computer-based test module consisting of 30 NBI polyp images. No feedback was given. They then completed a computer-based training module on the use of NBI in the differentiation of adenomas and hyperplastic polyps. The test module was then completed a second time.

MAIN OUTCOME MEASUREMENTS

Construct validity (the difference in baseline accuracy on the test module between different groups of participants) and content validity (difference in accuracy achieved on the test module before and after training) of the training module.

RESULTS

There was a significant difference in the baseline accuracy (P < .001) between experts (0.95; 95% confidence interval [CI], 0.92-0.97), experienced colonoscopists (0.68; 95% CI, 0.68-0.74), trainees (0.75; 95% CI, 0.67-0.82), and novices (0.62; 95% CI, 0.46-0.77). Accuracy increased significantly (P < .001) for all 3 groups after training (novices 0.84; 95% CI, 0.78-0.88, trainees 0.90; 95% CI, 0.84-0.93, and experienced colonoscopists 0.84; 95% CI, 0.76-0.89). After training, the agreement was moderate at least (κ = 0.56 for novices, κ = 0.70 for trainees, and κ = 0.54 for experienced colonoscopists).

LIMITATIONS

This study did not assess the accuracy of optical diagnosis in routine clinical practice.

CONCLUSION

A short, computer-based training module can improve the diagnostic accuracy and interobserver agreement for the use of NBI to differentiate adenomas from hyperplastic polyps and could be used for the initial training in optical diagnosis.

Diagnostic accuracy of narrow-band imaging and pit pattern analysis significantly improved for less-experienced endoscopists after an expanded training program

BACKGROUND

Previous reports assessing diagnostic skill using narrow-band imaging (NBI) and pit pattern analysis for colorectal polyps involved only highly experienced endoscopists.

OBJECTIVE

To evaluate diagnostic skills of less-experienced endoscopists (LEE group) for differentiation of diminutive colorectal polyps by using NBI and pit pattern analysis with and without magnification after an expanded training program.

DESIGN

Prospective study.

PATIENTS

This study involved 32 patients with 44 colorectal polyps (27 adenomas and 17 hyperplastic polyps) of < or =5 mm that were identified and analyzed by using conventional colonoscopy as well as non-magnification and magnification NBI and chromoendoscopy followed by endoscopic removal for histopathological analysis.

INTERVENTION

Before a training course, 220 endoscopic images were distributed in randomized order to residents with no prior endoscopy experience (NEE group) and to the LEE group, who had performed colonoscopies for more than 5 years but had never used NBI. The 220 images were also distributed to highly experienced endoscopists (HEE group) who had routinely used NBI for more than 5 years. The images were distributed to the NEE and LEE groups again after a training class. Magnification NBI and chromoendoscopy images were assessed by using the Sano and Kudo classification systems, respectively.

MAIN OUTCOME MEASUREMENTS

Diagnostic accuracy and interobserver agreement for each endoscopic modality in each group.

RESULTS

Diagnostic accuracy was significantly higher, and kappa (kappa) values improved in the LEE group for NBI with high magnification after expanded training. Diagnostic accuracy and kappa values when using high-magnification NBI were highest among endoscopic techniques for the LEE group after such training and the HEE group (accuracy 90% vs 93%; kappa = 0.79 vs 0.85, respectively).

LIMITATIONS

Study involved only polyps of < or =5 mm.

CONCLUSION

Using high-magnification NBI increased the differential diagnostic skill of the LEE group after expanded training so that it was equivalent to that of the HEE group.

The emerging role of screen based simulators in the training and assessment of colonoscopists

Incorporation of screen based simulators into medical training has recently gained momentum, as advances in technology have coincided with a government led drive to increase the use of medical simulation training to improve patient safety with progressive reductions in working hours available for junior doctors to train. High fidelity screen based simulators hold great appeal for endoscopy training. Potentially, their incorporation into endoscopy training curricula could enhance speed of acquisition of skills and improve patient comfort and safety during the initial phase of learning. They could also be used to demonstrate competence as part of the future relicensing and revalidation of trained endoscopists. Two screen based simulators are widely available for lower gastrointestinal endoscopy training, with a third recently produced in prototype. The utility of these simulators in lower gastrointestinal endoscopy training has been investigated, and construct and expert validity has been shown. Novices demonstrate a learning curve with simulator training that appears to represent real learning of colonoscopy skills. This learning transfers well to the real patient environment, with improvements in performance and patient discomfort scores in subsequent initial live colonoscopy. The significant limitations of currently available screen based simulators include cost implications, and restrictions on a role in certification and revalidation. Many questions remain to be answered by future research, including how best to incorporate screen based simulators into a colonoscopy training programme, their role in training in therapeutic endoscopy and the impact of simulator training on patient safety.

Face and construct validity of a computer-based virtual reality simulator for ERCP

BACKGROUND

Currently, little evidence supports computer-based simulation for ERCP training.

OBJECTIVE

To determine face and construct validity of a computer-based simulator for ERCP and assess its perceived utility as a training tool.

DESIGN

Novice and expert endoscopists completed 2 simulated ERCP cases by using the GI Mentor II.

SETTING

Virtual Education and Surgical Simulation Laboratory, Medical College of Georgia.

MAIN OUTCOME MEASUREMENTS

Outcomes included times to complete the procedure, reach the papilla, and use fluoroscopy; attempts to cannulate the papilla, pancreatic duct, and common bile duct; and number of contrast injections and complications. Subjects assessed simulator graphics, procedural accuracy, difficulty, haptics, overall realism, and training potential.

RESULTS

Only when performance data from cases A and B were combined did the GI Mentor II differentiate novices and experts based on times to complete the procedure, reach the papilla, and use fluoroscopy. Across skill levels, overall opinions were similar regarding graphics (moderately realistic), accuracy (similar to clinical ERCP), difficulty (similar to clinical ERCP), overall realism (moderately realistic), and haptics. Most participants (92%) claimed that the simulator has definite training potential or should be required for training.

LIMITATIONS

Small sample size, single institution.

CONCLUSIONS

The GI Mentor II demonstrated construct validity for ERCP based on select metrics. Most subjects thought that the simulated graphics, procedural accuracy, and overall realism exhibit face validity. Subjects deemed it a useful training tool. Study repetition involving more participants and cases may help confirm results and establish the simulator's ability to differentiate skill levels based on ERCP-specific metrics.

Computer-aided learning in capsule endoscopy leads to improvement in lesion recognition ability

BACKGROUND

The rapid expansion in use of capsule endoscopy (CE) has led to discussion about training needs and provision. The lesion recognition skills required for CE are ideally suited to computer-based training.

OBJECTIVE

The aim of this study was to prospectively evaluate the educational effectiveness of a computer-based CE training and testing module on trainees with varying experience.

DESIGN

This was a prospective educational evaluation study.

SETTING

Academic endoscopy unit.

PATIENTS

This study involved 28 trainees of varying CE experience (medical students, gastroenterology trainees) and 4 CE experts.

INTERVENTION

Trainees (medical students and gastroenterology trainees) without CE experience completed a 60-question, computer-based test module consisting of 30-second video clips and multiple-choice questions. Without feedback, trainees then completed a comprehensive, menu-driven, computer-based CE training module. The test module was then completed a second time and feedback was given. Expert performance on the test module was benchmarked by 4 CE experts.

MAIN OUTCOME MEASUREMENTS

The first measure was the difference in baseline performance on the test module between trainees and experts (to determine construct validity). The second measure was a change in performance on the test module after training (to determine content validity of the training module).

RESULTS

A significant difference was noted in baseline performance (P < .001) among CE experts (mean 73.8% +/- 8.0%), gastroenterology trainees (49.5% +/- 10.9%), and medical students (29.5% +/- 3.3%). Performance improved significantly (P < .001) in both trainee groups after training (gastroenterology trainees' posttraining score 62.1% +/- 7.7%; medical students' 46.7% +/- 6.8%).

CONCLUSION

Computer-based learning has a potentially significant role in the development of a training syllabus for CE and in CE accreditation.

How should we train capsule endoscopy? A pilot study of performance changes during a structured capsule endoscopy training program

There are no validated training/accreditation guidelines for capsule endoscopy. We assessed the utility of a structured training program on two experienced gastroenterologists and one endoscopy nurse. Validated studies were standardized for difficulty in blocks of three. Trainees completed a standardized data sheet for each study reported (12 studies for the physicians, 22 studies for the nurse). After each block the trainer graded performance and highlighted learning points. Statistical analysis was performed. Physician trainees accurately identified landmarks throughout, while the nurse made errors even at the end of training. Improvement in lesion detection and diagnostic accuracy improved in the nurse, but in only one of the physician trainees, highlighting the variability in learning curves. Overall performance improved in all trainees but was most marked for the nurse trainee (correlation coefficient 0.41, P = 0.06). Improvements in lesion recognition and diagnosis can be demonstrated in senior trainees and nurses following a structured training program; however, there is considerable variability.

Assessment of early learning curves among nurses and physicians using a high-fidelity virtual-reality colonoscopy simulator

BACKGROUND

Recently, it has been suggested that nurses can perform diagnostic endoscopy procedures, which traditionally have been a physician's responsibility. The existing studies concerning quality of sigmoidoscopy performed by nurses are small, used assessment tools with insufficient validation and to date there is very little knowledge of the learning curve patterns for physicians and nurses. The aim of a present study was to assess early learning curves on a virtual-reality colonoscopy simulator of untrained residents as compared with that of nurses with and without endoscopy assistance experience.

MATERIALS AND METHODS

Thirty subjects were included in the study: 10 female residents (median age 30.5 years) without colonoscopy experience, 10 female nurses (median age 27.5 years) without endoscopy assistance experience and 10 female nurses (median age 42 years) with endoscopy assistance experience. All participants performed 10 repetitions of task 6 from the "Introduction" colonoscopy module of the Accu Touch Endoscopy simulator. Eight experienced colonoscopists performed three repetitions of task 6 in order to provide the reference expert level of performance.

RESULTS

All subjects completed the virtual colonoscopy without complications. Significant differences existed between residents and nurses with respect to time to complete the procedure. Residents and nurses showed similar learning curve patterns. There were not significant differences between the groups in terms of volume of insufflated air, percentage of time without discomfort, and percentage of mucosa seen. None of the trainee groups achieved expert proficiency level in terms of time and amount of insufflated air by the tenth repetition.

CONCLUSIONS

Nurses performed virtual colonoscopy as accurately and safely as residents. Although the residents performed significantly faster, time differences showed a tendency towards decreasing, and appraisement of the numeric time differences seemed of minor practical importance. From a technical point of view this indicates that nurses may learn to perform colonoscopy after appropriate training.

Feedback in laparoscopic skills acquisition: an observational study during a basic skills training course

This study aimed to obtain insight in the effect of expert feedback during a basic laparoscopic skills training course for residents. A questionnaire was held among participants regarding provided feedback and the self-perceived laparoscopic skills improvement. The participants (n = 24) who completed the questionnaire were in their first to fifth postgraduate year. Most feedback was directed at intracorporeal knot tying (47% reported extensive feedback), while camera navigation and body positioning received the least feedback (40% and 43%, respectively, responded to have received no feedback at all). After the course, the self-perceived competence in intracorporeal knot tying and cutting had improved significantly, while camera navigation, body positioning, pointing, and grasping tasks did not improve. In conclusion, most benefit from expert feedback can be obtained at the start of the learning curve. Therefore, the basic laparoscopic skills course should be attended early in residency. Additionally, it is crucial that training objectives are clear prior to a course for both the expert and the trainee, in order to focus the feedback on all training objectives.

Web-based system for training and dissemination of a magnification chromoendoscopy classification

AIM: To evaluate the use of web-based technologies to assess the learning curve and reassess reproducibility of a simplified version of a classification for gastric magnification chromoendoscopy (MC).

METHODS: As part of a multicenter trial, a hybrid approach was taken using a CD-ROM, with 20 films of MC lasting 5 s each and an “autorun” file triggering a local HTML frameset referenced to a remote questionnaire through an Internet connection. Three endoscopists were asked to prospectively and independently classify 10 of these films randomly selected with at least 3 d apart. The answers were centrally stored and returned to participants together with adequate feedback with the right answer.

RESULTS: For classification in 3 groups, both intra- [Cohen’s kappa (κ) = 0.79-1.00 to 0.89-1.00] and inter-observer agreement increased from 1st (moderate) to 6th observation (κ = 0.94). Also, agreement with reference increased in the last observations (0.90, 1.00 and 1.00, for observers A, B and C, respectively). Validity of 100% was obtained by all observers at their 4th observation. When a 4th (sub)group was considered, inter-observer agreement was almost perfect (κ = 0.92) at 6th observation. The relation with reference clearly improved into κ (0.93-1.00) and sensitivity (75%-100%) at their 6th observations.

CONCLUSION: This MC classification seems to be easily explainable and learnable as shown by excellent intra- and inter-observer agreement, and improved agreement with reference. A web system such as the one used in this study may be useful for endoscopic or other image based diagnostic procedures with respect to definition, education and dissemination.

The perception of gastroenterology fellows towards the relationship between hand size and endoscopic training

Some endoscopic trainees find it difficult to manipulate an endoscope's controls, possibly due to small hand size. To assess this, a survey was mailed to all gastroenterology fellows in the US. Two hundred twenty-seven of 1,295 (17.5%) fellows responded. Median surgical glove size was 7.5. Ninety-three respondents (41.0%) considered their hand too small for a standard endoscope's handle; 176 (78.2%) felt that hand size affects the ability to learn endoscopy. Seventy-seven (34.2%) would use smaller handled endoscopes if available. Of the 38 respondents with glove sizes < or =6.5, 37 (97.4%) were female. These respondents were more likely to consider their hand too small (P < 0.001), want to use smaller handled endoscopes (P < 0.001), and feel that training programs should offer them (P = 0.009). These results suggest that a significant number of trainees, especially women, perceive that their hands are too small for standard endoscopes and believe that hand size plays a role in learning and performing endoscopy.

Computer-based endoscopy simulation: emerging roles in teaching and professional skills assessment

Advances in endoscopy simulation are reviewed with emphasis on applications in teaching and skills assessment. Endoscopy simulation has only been realized recently in a computer-based fashion because of advances in technology, but several studies have been performed both to validate computer-based endoscopy simulators and to assess their potential role in training. Multiple studies have shown that simulators can distinguish between clinicians at different skill levels and also have shown improvement in clinician skill, particularly at the early stages of training. This article summarizes those studies. The cost versus benefit of endoscopic simulators is also discussed, as well as the upcoming role of simulators in judging competence and as a tool in the credentialing process.

Validation of two instruments to assess technical bronchoscopic skill using virtual reality simulation

BACKGROUND

In an era of increased emphasis on patient safety and competency-based education, demonstration of significant variability in the teaching of flexible bronchoscopy has led to initiatives for new standardized curricula and assessment tools.

OBJECTIVES

To evaluate the efficacy of such curricula and to assess bronchoscopic skill, 2 measuring instruments have been developed: the Bronchoscopy Skills and Tasks Assessment Tool (BSTAT) and the Bronchoscopy Step-by-Step Evaluation Tool (BSET). We studied the validity and reliability of these 2 instruments.

METHODS

Two independent testers simultaneously scored 22 volunteer participants at 3 levels [novice (n = 7), fellow (n = 8) and attending (n = 7)] on a virtual reality bronchoscopy simulator using the 2 instruments; each participant was tested twice, in 2 separate sessions. Intertester and test-retest reliability were analyzed with intraclass correlations (ICC); ANOVA was used to assess concurrent validity based on the subjects' expected skill level.

RESULTS

The ICCs between the testers were 0.98 for both the BSTAT and BSET. Comparison of the scores between the sessions showed high test-retest reliability by ICC (0.86 and 0.85 for BSTAT and BSET respectively), with a small yet statistically significant learning effect. The novice group's scores were lower than the fellows' and attendings' (p < 0.001) for both the BSTAT and BSET; the fellows' scores were consistently lower than the attendings' on both tests, yet the differences were not statistically significant.

CONCLUSION

This validation study of 2 objective tests of bronchoscopic skill demonstrated high reliability and concurrent validity. These instruments can now be used to evaluate the effectiveness of new competency-based bronchoscopy curricula.

Training for NOTES

The foundation of skills for the performance of natural orifice translumenal endoscopic surgery (NOTES) lies in the training for general surgery (especially laparoscopy) and flexible gastrointestinal endoscopy. Physicians wishing to practice NOTES need to acquire or have both skill sets, or need to partner together to blend complementary capabilities with colleagues. In the future, however, a new cadre of NOTES specialists may emerge who will have developed individual expertise in the full spectrum of NOTES knowledge base requirements. This article highlights a body of knowledge and skills needed to become a NOTES proceduralist and review the current training paradigms for gastrointestinal endoscopists and surgeons.

Surgical simulator design and development

With the introduction of minimally invasive surgery (MIS), it became necessary to develop training methods to learn skills outside the operating room. Several training simulators have become commercially available, but fundamental research into the requirements for effective and efficient training in MIS is still lacking. Three aspects of developing a training program are investigated here: what should be trained, how it should be trained, and how to assess the results of training. In addition, studies are presented that have investigated the role of force feedback in surgical simulators. Training should be adapted to the level of behavior: skill-based, rule-based, or knowledge-based. These levels can be used to design and structure a training program. Extra motivation for training can be created by assessment. During MIS, force feedback is reduced owing to friction in the laparoscopic instruments and within the trocar. The friction characteristics vary largely among instruments and trocars. When force feedback is incorporated into training, it should include the large variation in force feedback properties as well. Training different levels of behavior requires different training methods. Although force feedback is reduced during MIS, it is needed for tissue manipulation, and therefore force application should be trained as well.

Acquiring basic endoscopy skills by training on the GI Mentor II

BACKGROUND

Achieving proficiency in flexible endoscopy requires a great amount of practice. Virtual reality (VR) simulators could provide an effective alternative for clinical training. This study aimed to gain insight into the proficiency curve for basic endoscope navigation skills with training on the GI Mentor II.

METHODS

For this study, 30 novice endoscopists performed four preset training sessions. In each session, they performed one EndoBubble task and managed multiple VR colonoscopy cases (two in first session and three in subsequent sessions). Virtual reality colonoscopy I-3 was repeatedly performed as the last VR colonoscopy in each session. The assignment for the VR colonoscopies was to visualize the cecum as quickly as possible without causing patient discomfort. Five expert endoscopists also performed the training sessions. Additionally, the performance of the novices was compared with the performance of 20 experienced and 40 expert endoscopists.

RESULTS

The novices progressed significantly, particularly in the time required to accomplish the tasks (p < 0.05, Friedman's analysis of variance [ANOVA], p < 0.05, Wilcoxon signed ranks). The experts did not improve significantly, except in the percentage of time the patient was in excessive pain. For all the runs, the performance of the novices differed significantly from that of both the experienced and the expert endoscopists (p < 0.05, Mann-Whitney U). The performance of the novices in the latter runs differed less from those of both the experienced and the expert endoscopists.

CONCLUSIONS

The study findings demonstrate that training in both VR colonoscopy and EndoBubble tasks on the GI Mentor II improves the basic endoscope navigation skills of novice endoscopists significantly.

Guiding navigation in colonoscopy

BACKGROUND

Nonrigid environments such as the human colon present unique challenges for the navigator in maintaining spatial orientation. Conventional wisdom suggests that a navigational aid, similar to a map, that provides critical shape information would be useful. This article presents a design concept for a colonoscopy navigational aid and the results of an experiment conducted to evaluate the display for supporting navigation and spatial orientation in simulated colonoscopy.

METHODS

A navigational aid was designed to present shape information in an augmented reality display. A total of 14 untrained subjects performed a colonoscopy procedure in rigid and nonrigid colon models, with and without the navigational aid display, in a Latin square design. Performance measures such as time, distance or efficiency of travel, and location and direction error were recorded, together with subjective measures of confidence and workload.

RESULTS

The results showed that, unlike navigating in rigid environments, the subjects spent more time navigating in the nonrigid environment (p < 0.01) and traveled a longer total distance (p = 0.01). The navigational aid had no effect on performance, as compared with the no aid condition. However, subjective measures showed that the subjects were more confident about their determination of location and direction (p < 0.01). They also preferred having the aid during navigation.

CONCLUSION

A navigational aid or map that provides shape information does not seem to improve performance in colonoscopy. In fact, it may lead to a false sense of security about location and orientation in the colon. The value of a map for training purposes remains to be examined.

AccuTouch Endoscopy Simulator: development, applications and early experience

These studies suggest that there are several potential advantages to use of the AccuTouch simulators, including (1) improvement of the endoscopic training of trainees before patient contact, (2) possible evaluation for procedural competency, and (3) possible cost savings, increasing the productivity of faculty while the trainees are developing competency using the trainers. Other potential advantages might include the training of endoscopists in new diagnostic and therapeutic techniques and even new devices before patient contact. With continued improvement in software and hardware, endoscopic simulators will become an integral part of endoscopic training programs.

Evidence-based assessment of endoscopic simulators for training

This article focuses on some important key questions: Can the currently available simulators distinguish between the skill levels of a novice who has never performed endoscopic procedures and an expert who teaches the procedure? Do the cases on the simulator accurately simulate the difficulty of the procedure and the skill required to perform therapeutic maneuvers? What is the evidence that the currently available simulators can supplement or even replace bedside training? When are simulators most useful in the learning period? This article examines the published evidence regarding the efficacy of the current generation of endoscopic simulators in endoscopic training.

Objective assessment of visuospatial and psychomotor ability and flow of residents and senior endoscopists in simulated gastroscopy

BACKGROUND

Advanced medical simulators have predominantly been used to shorten the learning curve of endoscopy for medical students and young residents. Rarely have the effects of visuospatial ability and attitudes of intermediately experienced and experienced specialists been studied with regard to simulator training. The aim of this study was to assess the effects of visuospatial ability and attitude on performance in simulator training.

METHODS

Eighteen surgical residents were included in the study. Prior to the simulated gastroscopy task, they performed a visuospatial test (the card rotation test). After the simulated gastroscopy task, they completed a questionnaire regarding flow experiences. Their results were compared with those of 11 expert endoscopists who performed the same tests.

RESULTS

Total gastroscopy time was significantly shorter for the expert endoscopists compared to residents (2 min 11 sec, p = 0.003). There was also a trend of more mucosa inspected (p = 0.088) and higher efficiency of screening (p = 0.069) by the experts. The residents made fewer errors in the card rotation test than the expert endoscopists (2.5 +/- 0.8 vs 5.5 +/- 1.2, respectively; p = 0.034), and their visuospatial card rotation test results correlated better with their performance in the simulated gastroscopy.

CONCLUSIONS

A virtual gastroscopy task presents more of an emotional as well as a psychomotoric challenge to intermediately experienced endoscopists than to senior experts. Our study demonstrates that these differences can be objectively assessed by the use of visuospatial ability tests, flowsheets, and an endoscopic simulator.

Colonoscopy training--new approaches, old problems

Colonoscopy traditionally has been taught using the apprenticeship approach. Assessment of competence has been, at best, informal and based largely on the number of procedures trainees have performed. The advent of colorectal cancer screening has demanded formalization of training and assessment using structured teaching and state-of-the-art training tools, such as computer simulation. New-generation approaches and technology have the ability to raise the standard of coloscopy provided by competent practitioners.

Advances in colonic imaging: technical improvements in colonoscopy

Colonoscopy has been available since the early 1970s and has become critical to the diagnosis and management of colorectal disorders. Features of the modern colonoscope and its variants are discussed, including the role of paediatric and variable stiffness colonoscopes for difficult insertion. The place of magnetic endoscope imaging systems and simulators in routine colonoscopy and training are examined. Finally, several recent innovations are used to illustrate how colonoscopy may evolve in the future, including new takes on the current instrument as well as potentially revolutionary pain-free, technically-easy, robotic devices for examination of the bowel.