A competency-based test of bronchoscopic knowledge using the Essential Bronchoscopist: an initial concept study

Assessment of Bronchoscopic Dexterity and Procedural Competency in a Low-fidelity Simulation Model

BACKGROUND

Assessment of competency in bronchoscopy has traditionally been undertaken in clinical settings, however, recent recognition of increased procedural complications and learner anxiety have led to interest in simulation-based competency assessment. The aim of this study was to determine if low-fidelity simulation-based assessment allows discrimination of competency based on prior experience between bronchoscopists.

METHODS

Forty-four participants were allocated to 3 groups based on prior bronchoscopic experience [novices (n=31) with no prior experience, intermediates (n=7) with prior experience of 5 to 10 bronchoscopies, and expert bronchoscopists (n=6) with minimum 200 prior bronchoscopies performed]. Participants performed bronchoscopy in a 3D-printed anatomic airway model and were assessed according to time required to navigate to a target bronchus. Bronchoscopic dexterity was measured using a modified version of the validated Bronchoscopy Skills and Tasks Assessment Tool.

RESULTS

Competency based on successful navigation to a target bronchus differed significantly between each group [experts, 12/12 (100%); intermediates, 9/14 (64%); novices, 19/62 (31%); P<0.001]. Bronchoscopic dexterity as measured by modified Bronchoscopy Skills and Tasks Assessment Tool also differed significantly between groups with experts achieving consistently higher scores compared with other 2 groups [median (interquartile range) scores: novices, 3.5 (2.5 to 5); intermediate, 5 (4.5 to 7); experts, 8 (7.5 to 8); P<0.0.001).

CONCLUSIONS

Multiple measures demonstrate that low-fidelity simulation-based assessment may reliably discriminate between different levels of skill in performing bronchoscopic navigation and airway inspection. Procedural dexterity of trainees may be assessed in a 0-risk simulation environment.

A survey of training in pediatric flexible bronchoscopy

OBJECTIVE

There is currently no evidence-based method for defining competency in pediatric flexible bronchoscopy (FB). Based on expert opinion, guidelines using numbers of procedures have been published in defining competency for pediatric FB. The purpose of this study was to formally survey the opinion of USA pediatric pulmonology training directors about the assessment of competency and training experiences in pediatric FB in their programs.

METHODS

An Internet-based Survey Monkey™ of the Pediatric Pulmonary Training Directors Association (PEPTDA) was administered 10/12/10 through 11/1/10 with a supplemental survey 6/1/11-6/30/11.

RESULTS

This survey of US pediatric pulmonology training directors about competency and training in pediatric FB showed that a majority (86%) felt there was a minimum threshold of procedures for developing competency that could be defined, with a median of 50 and an average of 56.4 (SD = 33.0). The actual number of FBs performed by fellows during their 3-year fellowship averaged 89.4 (SD = 45.3) with a range of 10-200. The survey also revealed a variety of teaching techniques used for FB, including simulation technology. Finally many differences were reported in skill assessment, locations for performance of FB, and the range of underlying indications and patient populations. The apprenticeship model is the predominant method of learning FB in the surveyed programs.

CONCLUSIONS

A majority of US pediatric pulmonology training directors felt that a minimum number of procedures could be defined for developing competency in pediatric FB. There was variability in the numbers of procedures performed, training techniques and assessment, and application of FB. This survey represents an initial step in assessing training and defining competency in pediatric FB.

Simulation in flexible bronchoscopy and endobronchial ultrasound: a review

Flexible bronchoscopy is a widely used medical procedure performed by a variety of specialists. Learning flexible bronchoscopy in a clinical setting is associated with increased learner anxiety, introduces significant variability into the learning experience, and increases complications placing the burden of learning on the patients. Bronchoscopy simulation involves the use of inanimate models, wet lab models, or computerized virtual reality simulators to teach and evaluate bronchoscopy skills, with the advantage of standardization of the teaching and assessment techniques without risk to the patient. The aim of this study is to provide a review of the major evidence supporting the use of bronchoscopy simulation to date and to identify limitations in the literature to stimulate further research incorporating these exciting teaching modalities.

Web-based versus traditional lecture: are they equally effective as a flexible bronchoscopy teaching method?

This study compares the traditional live lecture to a web-based approach in the teaching of bronchoscopy and evaluates the positive and negative aspects of both methods. We developed a web-based bronchoscopy curriculum, which integrates texts, images and animations. It was applied to first-year interns, who were later administered a multiple-choice test. Another group of eight first-year interns received the traditional teaching method and the same test. The two groups were compared using the Student's t-test. The mean scores (± SD) of students who used the website were 14.63 ± 1.41 (range 13-17). The test scores of the other group had the same range, with a mean score of 14.75 ± 1. The Student's t-test showed no difference between the test results. The common positive point noted was the presence of multimedia content. The web group cited as positive the ability to review the pages, and the other one the role of the teacher. Web-based bronchoscopy education showed results similar to the traditional live lecture in effectiveness.

Development and organization of an interventional pulmonology department

Development and organization of an interventional pulmonology department requires careful consideration of many elements, including personnel, equipment, logistics, procedures performed, practice management, institutional vision and biases, practice setting, continued medical education, risk management, communication and fiscal responsibility. In this paper, a patient-centric approach is used to begin addressing these issues and present the strengths, weaknesses, advantages and disadvantages of an interventional pulmonology department.

Reliability of a 25-item low-stakes multiple-choice assessment of bronchoscopic knowledge

BACKGROUND

A need for improved patient safety, quality of care, and accountability has prompted the development of competency-based educational processes. Assessment tools related to bronchoscopy training, however, have not yet been developed or validated.

PURPOSES

To determine whether 25 multiple-choice questions (MCQs) extracted from the free, Web-based Essential Bronchoscopist (EB) learning guide qualify in their original form as a preliminary pool of questions for a low-stakes assessment of bronchoscopic knowledge.

MATERIALS AND METHODS

Twenty-five randomly selected MCQs from among the top 70 question-answer sets of the EB were administered to 40 self-declared novice bronchoscopists (n = 13), experienced bronchoscopists (n = 21), and expert bronchoscopists (n = 6). A difficulty index and a discrimination index (DI) were calculated for each item. Internal consistency reliability was calculated using item-total correlation and Cronbach alpha. Content validity was determined by five independent experts. Ideal test items based on a difficulty index and item-total correlation were administered to a different group of 24 bronchoscopists to prospectively reassess internal consistency reliability.

RESULTS

The mean (+/- SD) score for the 40 participants was 16.47 +/- 3.72 (median score, 17; score range, 7 to 22). The mean difficulty index was 0.65 +/- 0.22, and the mean DI was 0.52 +/- 0.28. Item total-correlations ranged from - 0.01 to + 0.71. Test content was unanimously validated. The Cronbach alpha was 0.69. There was no significant correlation between scores and the number of bronchoscopies performed or self-declared expertise. Eleven ideal test MCQs were identified. The internal consistency of these items remained satisfactory (Cronbach alpha = 0.75) when assessed prospectively in a different cohort.

CONCLUSION

Reliable and valid MCQs were identified to initiate a preliminary pool of questions for a low-stakes assessment of bronchoscopic knowledge.